system

Abstract

Provide a system. A means for a user to input personal information; A means for storing the entered personal information in a database; A means of proposing individual services based on the stored data; means for transmitting the proposal to a user terminal; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With the recent increase in inbound tourists, there is a demand for support to help travelers enjoy their stay in Japan comfortably and efficiently. However, travelers often encounter difficulties when they arrive due to language barriers, cultural differences, and a variety of payment methods. Furthermore, there is a lack of comprehensive tools for travelers to efficiently plan and enjoy sightseeing spots and activities in Japan. Therefore, there is a demand for a system that allows travelers to enjoy their stay in Japan to the fullest without stress. [Means for solving the problem]

[0005] To solve the above-mentioned problems, the present invention provides the following means. A system is constructed that includes a means for users to input personal information, a means for saving the input personal information in a database, a means for proposing individual services based on the saved data, and a means for transmitting the proposals to the user's terminal. The system also provides a means for users to input their desired travel itinerary, destination, and budget, a means for proposing appropriate tourist spots and activities based on the input travel information, and a means for automatically generating a travel plan from the proposed tourist spots and activities and transmitting it to the user's terminal, thereby enabling travelers to easily create travel plans. Furthermore, the system provides a means for acquiring the user's current location using the GPS function of the user's terminal, a means for calculating the optimal travel route based on the current location and destination, and a means for displaying the calculated travel route on the user's terminal, thereby enabling travelers to travel smoothly within the local area. This realizes a comprehensive support system that allows travelers to enjoy their stay in Japan to the fullest.

[0006] "User" refers to a traveler who uses this system to create a travel plan and receive on-site information and navigation.

[0007] "Personal information" refers to information entered by the user that identifies the individual, such as name, nationality, emergency contact information, and email address.

[0008] "Database" refers to the server-based system of record for storing and managing personal information, travel information, and travel plans entered by users.

[0009] "Service Suggestion" refers to the system's ability to provide appropriate tourist spots and activities based on the user's personal and travel information.

[0010] "User terminal" refers to an electronic device used by a traveler to use the application of this system, such as a smartphone or tablet.

[0011] "Travel information" refers to information entered by the user, such as travel itinerary, destination, budget, and activities of interest.

[0012] "Tourist attractions" refer to famous places and sights that tourists can visit.

[0013] "Activities" refers to specific actions such as events and experience programs that travelers can experience locally.

[0014] "GPS function" refers to the technology installed in a user's device for obtaining location information.

[0015] "Travel route" refers to the optimal travel route from the current location to the destination. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is an integrated system for enabling inbound tourists to enjoy their stay in Japan smoothly, and functions in cooperation with users, terminals, and servers. Specific embodiments for carrying out the present invention will be described below.

[0038] Account Registration

[0039] User:

[0040] The first step a traveler takes after launching the app is to create an account. Users follow the app's instructions and enter personal information such as their name, nationality, emergency contact details, and email address.

[0041] Device:

[0042] The terminal transmits the personal information input by the user to the server.

[0043] server:

[0044] The server stores the received personal information in a database, and once the storage is complete, a confirmation email is sent to the user's email address, requesting that they activate their account.

[0045] Creating a travel plan

[0046] User:

[0047] Users use the app to input their desired travel dates, destinations, budget, and activities they are interested in.

[0048] Device:

[0049] The terminal transmits the input information to the server.

[0050] server:

[0051] The server compares the travel information with the database, selects appropriate tourist spots and activities, and automatically generates and proposes a travel plan to the user.

[0052] Device:

[0053] The device displays the proposed travel plan to the user, who can review the plan, make adjustments as needed, and finalize the plan.

[0054] Real-time navigation and information

[0055] User:

[0056] When a user travels to a tourist spot or destination, they enter the destination into the app to activate the navigation function.

[0057] Device:

[0058] The device uses GPS to obtain its current location and transmits it to the server.

[0059] server:

[0060] The server calculates the optimal route based on the current location and destination, and generates detailed route guidance including transportation information.

[0061] Device:

[0062] The device displays the route guidance received from the server on a map and provides real-time navigation.

[0063] Payment and cashless payment

[0064] User:

[0065] When using a store or service, the user activates the app's payment function and presents the QR code (registered trademark) to the store.

[0066] Device:

[0067] The terminal sends payment information to the server to initiate the payment process.

[0068] server:

[0069] The server checks the payment information, executes the payment, and sends the result to the terminal once the payment is complete.

[0070] Device:

[0071] The terminal will display a payment completion screen to the user and provide receipts and payment history as needed.

[0072] Emergency support

[0073] User:

[0074] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency.

[0075] Device:

[0076] The terminal transmits emergency information to the server.

[0077] server:

[0078] The server searches for emergency services and hospital information closest to the current location and generates data to provide to the user.

[0079] Device:

[0080] The device displays emergency information received from the server and provides translation functions as needed to help users communicate with local medical staff.

[0081] Specific examples

[0082] Example 1: Creating a travel plan

[0083] User:

[0084] Traveler A starts the app and plans a trip. The trip is for 7 days, with the main tourist destinations being Tokyo and Kyoto. The budget is 100,000 yen.

[0085] Device:

[0086] The user's input information is sent to the server.

[0087] server:

[0088] Based on the information provided by Traveler A, the plan suggests tourist spots such as Sensoji Temple, Skytree, Fushimi Inari Taisha Shrine, and Kiyomizu-dera Temple. In addition, activities such as a maiko experience and sushi-making experience are also included in the plan.

[0089] Device:

[0090] The created travel plan is displayed to Traveler A, prompting them to confirm and make adjustments.

[0091] Example 2: Real-time navigation

[0092] User:

[0093] Traveler B needs navigation to travel from Tokyo to Osaka. He enters "Tokyo to Osaka" into the app.

[0094] Device:

[0095] Use the GPS function to obtain the current location (Tokyo Station) and send it to the server.

[0096] server:

[0097] The system calculates the optimal travel route from Tokyo Station to Osaka Station, including using the Shinkansen, and generates detailed navigation information.

[0098] Device:

[0099] It displays the travel route received from the server and provides real-time information on the next train and transfers.

[0100] In this way, the present invention provides a comprehensive support system to help inbound travelers enjoy their stay in Japan to the fullest.

[0101] The processing flow will be explained below.

[0102] Account Registration

[0103] Step 1:

[0104] The user launches the app and selects "Create a new account" on the initial screen.

[0105] Step 2:

[0106] The user enters their name, nationality, emergency contact, and email address.

[0107] Step 3:

[0108] The terminal transmits the input information to the server.

[0109] Step 4:

[0110] The server stores the received information in a database.

[0111] Step 5:

[0112] The server sends a confirmation email to the user's email address.

[0113] Step 6:

[0114] The user activates their account by clicking the link in the confirmation email.

[0115] Step 7:

[0116] The device will display a screen indicating that the account registration was successful.

[0117] Creating a travel plan

[0118] Step 1:

[0119] The user selects "Create a trip" in the app.

[0120] Step 2:

[0121] The user enters desired travel dates, destinations, budget, and activities of interest.

[0122] Step 3:

[0123] The terminal transmits the input travel information to the server.

[0124] Step 4:

[0125] The server compares the travel information with a database and selects appropriate attractions and activities.

[0126] Step 5:

[0127] The server automatically generates a travel plan based on the selected tourist spots and activities.

[0128] Step 6:

[0129] The server transmits the generated travel plan to the terminal.

[0130] Step 7:

[0131] The device displays the travel plan to the user and prompts them to confirm and adjust it.

[0132] Step 8:

[0133] The user reviews and confirms the plan.

[0134] Real-time navigation

[0135] Step 1:

[0136] The user enters a destination in the app and launches the navigation function.

[0137] Step 2:

[0138] The device uses GPS to obtain its current location and sends it to the server.

[0139] Step 3:

[0140] The server calculates the optimal route based on the current location and destination.

[0141] Step 4:

[0142] The server sends route guidance information to the terminal.

[0143] Step 5:

[0144] The device displays route guidance on a map and provides real-time navigation to the user.

[0145] Payment and cashless payment

[0146] Step 1:

[0147] When the user visits a store or uses a service, they activate the app's payment function.

[0148] Step 2:

[0149] The user confirms the payment amount and scans the QR code.

[0150] Step 3:

[0151] The terminal sends the payment information to the server.

[0152] Step 4:

[0153] The server verifies the payment information and executes the payment.

[0154] Step 5:

[0155] The server sends the payment result to the terminal.

[0156] Step 6:

[0157] The terminal displays a payment completion screen to the user.

[0158] Emergency support

[0159] Step 1:

[0160] The user launches the emergency support function and enters the details of the emergency.

[0161] Step 2:

[0162] The terminal transmits emergency information to the server.

[0163] Step 3:

[0164] The server searches for information about emergency services and hospitals closest to your location.

[0165] Step 4:

[0166] The server sends the search results to the terminal.

[0167] Step 5:

[0168] The terminal displays the emergency information to the user.

[0169] Step 6:

[0170] The device provides translation functionality as needed to help users communicate with local medical staff.

[0171] Example 1

[0172] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0173] Today's travelers are expected to plan their own trips, navigate their way around, make payments, and even respond to emergencies all in one place, but existing systems struggle to provide integrated support for all of these. It's particularly difficult to provide a single platform for planning sightseeing trips, real-time navigation, cashless payments, and emergency response. As a result, travelers are forced to use multiple applications and services, which creates hassle and confusion that can degrade the quality of their trip.

[0174] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0175] In this invention, the server includes a means for a user to input personal information, a means for saving the input personal information in a database, and a means for sending a confirmation email of account activation to the user's email address after saving is complete, thereby enabling a user to manage all travel-related information on one platform and enjoy personalized services.

[0176] "User" refers to a traveler who uses this system.

[0177] "Terminal" refers to an electronic device such as a smartphone or tablet used by a user.

[0178] "Server" refers to a computer system that processes, stores, and manages data for the entire system.

[0179] "Personal information" refers to information entered by the user, such as name, nationality, emergency contact information, and email address.

[0180] "Database" refers to a system for storing and managing information such as personal information and travel plans.

[0181] "Verification Email" means the email sent to a User's email address to activate their account.

[0182] "Travel itinerary" refers to the planned duration of the trip entered by the user.

[0183] "Destinations" refer to places or cities that a user plans to visit during their trip.

[0184] "Budget" refers to the amount of money a user plans to spend on a trip.

[0185] "Activities" refer to experiential tourist events and activities suggested based on the user's interests.

[0186] "GPS Function" refers to the location technology used by a Device to obtain its current location.

[0187] "Travel route" refers to the route from the current location to the destination.

[0188] "Payment functionality" refers to the functionality of the app that allows users to make cashless payments.

[0189] "Payment System" means an online payment platform for processing payment information.

[0190] "Emergency Support Feature" refers to the feature of the App that enables a User to receive necessary support in an emergency.

[0191] "Emergency information" refers to the user's report of an emergency situation.

[0192] "Translation Feature" refers to the automatic translation feature used by Users to communicate in their local language.

[0193] This invention is an integrated system that enables inbound tourists to enjoy their stay in Japan smoothly. The user, terminal, and server work together, allowing the user to manage their entire trip on a single platform.

[0194] First, the user launches the app using a device such as a smartphone or tablet. When the app is launched for the first time, the user enters their personal information, which is then sent from the device to the server. The server stores the received information in a database and sends a confirmation email to the user's email address to activate their account. The account is activated when the user approves this confirmation email.

[0195] Next, the user enters their travel itinerary, destinations, budget, and activities of interest into the app. The device sends this travel information to the server, which compares it with a database and uses a generative AI model (e.g., GPT-4 (registered trademark)) to suggest suitable tourist spots and activities. The proposed plan is automatically generated, saved in JSON format, and then sent to the user's device. The device displays the received plan to the user and makes adjustments as necessary.

[0196] Furthermore, when a user travels to a tourist spot or other destination, the device's GPS function is used to send the user's current location to the server. The server then calculates the optimal route from the user's current location to the destination. Detailed navigation information is generated and sent to the device. The device then displays this information on a map and provides real-time navigation.

[0197] When a user visits a store or uses a service, they activate the app's payment function and present the QR code to the store. This payment information is sent from the device to the server, and the server executes the payment using a payment system (e.g., Stripe, PayPal). The payment completion result is sent to the terminal, which displays it to the user.

[0198] In an emergency, the user enters emergency information using the app's emergency support function. This information is sent from the device to a server, which searches for the most appropriate emergency service or hospital information. The search results are sent to the device and displayed there. Furthermore, a translation function is provided if necessary, allowing the user to communicate with local staff.

[0199] Specific examples

[0200] Example 1: Creating a travel plan

[0201] User A launches the app and creates a seven-day travel plan. The destinations entered are Tokyo and Kyoto, with a budget of 100,000 yen. The entered information is sent to the server, which then suggests tourist spots such as Sensoji Temple, Skytree, Fushimi Inari Taisha Shrine, and Kiyomizu-dera Temple. Activities such as a maiko experience and a sushi-making experience are also incorporated into the plan. The generated plan is displayed on the device to User A, who then reviews the contents and finalizes the plan.

[0202] Example 2: Real-time navigation

[0203] User B uses the app's navigation function when traveling from Tokyo to Osaka. The current location (Tokyo Station) is obtained using the GPS function and sent to the server. The server calculates the optimal route from Tokyo Station to Osaka Station, generates detailed navigation including Shinkansen information, and sends it to the device. The device displays this on a map and provides real-time information on the next train and transfers.

[0204] Prompt Sentence Examples

[0205] "Generate a 7-day travel plan with a budget of 100,000 yen, focusing on Tokyo and Kyoto."

[0206] As described above, the present invention allows users to manage all travel-related information in one system, providing a smooth travel experience.

[0207] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0208] Step 1:

[0209] User information entry and submission

[0210] A user launches the app and navigates to the account creation screen, where they enter personal information such as their name, nationality, emergency contact, and email address. After entering the information, the device encrypts it using the HTTPS protocol and sends it to the server. The input contains personal information, and the output is encrypted data.

[0211] Step 2:

[0212] Server saves information and sends confirmation email

[0213] The server stores the received personal information in a database (e.g., MySQL (registered trademark)). If the storage is successful, the server sends a confirmation email of account activation to the user's email address via the SMTP server. The input is the received encrypted personal information, and the output is the result of storing it in the database and the confirmation email.

[0214] Step 3:

[0215] User's travel information input and submission

[0216] The user inputs their travel itinerary, destinations, budget, and activities of interest into the app. The device sends this information to the server. The input is information about the trip, and the output is data sent to the server.

[0217] Step 4:

[0218] Travel plan generation by server

[0219] The server compares the received travel information with a database (e.g., MongoDB), selects appropriate tourist spots and activities using a generative AI model (e.g., GPT-4), and automatically generates a travel plan. The generated plan is saved in JSON format and sent to the device. The input is travel information, and the output is the generated travel plan.

[0220] Step 5:

[0221] View and adjust travel plans on your device

[0222] The device displays the travel plan received from the server to the user. The user reviews the plan and makes adjustments as necessary. If adjustments are necessary, the device sends the information back to the server. The input is the generated travel plan, and the output is the adjusted plan approval.

[0223] Step 6:

[0224] Launching the navigation function

[0225] When a user travels to a tourist spot or other destination, they input their destination into the app and activate the navigation function. The device uses GPS to obtain current location information and sends it to the server. The input is destination information and current location information, and the output is the data sent to the server.

[0226] Step 7:

[0227] Route calculation by the server

[0228] The server calculates the optimal route based on the received current location and destination information using the Google (registered trademark) Maps API and route calculation algorithms. Detailed route guidance is generated in JSON format and sent to the device. The input is current location information and destination information, and the output is route guidance.

[0229] Step 8:

[0230] Displaying travel route on device

[0231] The device displays the route guidance received from the server on a map and provides real-time navigation. The user moves around while checking the app screen. The input is the route guidance, and the output is the display of navigation information.

[0232] Step 9:

[0233] Activating payment functions and sending information

[0234] When using a store or service, the user activates the app's payment function, selects a payment method, and presents a QR code. The terminal then sends the payment information to the server. The input is the payment information, and the output is the data sent to the server.

[0235] Step 10:

[0236] Server-based payment execution

[0237] The server processes the received payment information via a payment system (e.g., Stripe, PayPal). After the payment is completed, the result is sent to the terminal. The input is the payment information, and the output is the payment result.

[0238] Step 11:

[0239] Terminal indicates payment is complete

[0240] The terminal receives the payment result from the server, displays it to the user, and provides receipts and payment history as needed. The input is the payment result, and the output is a payment completion screen.

[0241] Step 12:

[0242] Enter and submit emergency information

[0243] When a user encounters an emergency, they activate the app's emergency support function and input the details of the emergency. The device then sends the emergency information to the server. The input is the emergency information, and the output is the data sent to the server.

[0244] Step 13:

[0245] Provision of emergency response information by server

[0246] The server searches for emergency services and hospital information nearest to the user's current location based on the received emergency information. The search results are sent to the device. The input is the emergency information, and the output is emergency response information.

[0247] Step 14:

[0248] Emergency information display and translation support on devices

[0249] The terminal displays emergency response information received from the server to the user and provides translation function as needed, allowing the user to communicate with local medical staff. The input is emergency response information, and the output is display of emergency information and translation support.

[0250] (Application example 1)

[0251] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0252] Support systems to help inbound tourists enjoy their stay in Japan include travel plan creation, navigation, cashless payment, and emergency assistance, but these services are often provided separately. Such systems require travelers to use multiple applications and devices, which is inconvenient. These functions are rarely integrated into autonomous transportation, particularly in these modes of transportation, so there is a need for centralized, integrated support.

[0253] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0254] In this invention, the server includes means for a user to input personal information, means for saving the input personal information in a database, means for proposing individual services based on the saved data, means for transmitting the proposals to a user terminal, means for managing the user's itinerary in the autonomous transportation means, means for providing navigation based on the user's location information, means for sending payment information to the server and executing payment, and means for providing information on nearby medical institutions in the event of an emergency. This allows travelers to use a single system to receive a variety of integrated services.

[0255] "User" refers to travelers and end users who use this system.

[0256] "Personal Information" refers to information about you, including your name, nationality, emergency contact details, email address, etc.

[0257] "Database" refers to an information system for storing and managing users' personal information, travel information, payment information, etc.

[0258] A "server" is a computer that manages the entire system and processes and stores data.

[0259] "Autonomous mobility" refers to devices that move without user intervention, such as self-driving vehicles and robots.

[0260] "Location information" refers to a user's current location and past location data obtained using technologies such as GPS.

[0261] "Navigation" refers to the function of calculating the optimal route from the current location to the destination and guiding the user along the route.

[0262] "Payment" refers to the process by which a user pays for goods or services cashlessly.

[0263] An "emergency" refers to a situation in which the user encounters a sudden situation such as an accident or sudden illness.

[0264] "Medical institution information" refers to information about medical facilities such as hospitals and clinics.

[0265] This invention is an integrated system that allows inbound tourists to enjoy their stay in Japan smoothly. The user, terminal, and server work together to provide personal information management, travel plan creation, real-time navigation, cashless payment, and emergency support.

[0266] System Program

[0267] The system includes the following elements:

[0268] 1. Registration of personal information

[0269] The user uses a terminal (for example, a touch panel inside the car) to input personal information (name, nationality, emergency contact, email address).

[0270] The terminal sends the entered personal information to the server, which stores it in a database.

[0271] Once saved, the server will send a confirmation email to the user's email address, asking them to activate their account.

[0272] 2. Create a travel plan

[0273] The user uses the terminal to input the desired travel dates, destinations, budget, and activities of interest.

[0274] The device sends the entered information to a server, which then compares the travel information with a database to select appropriate tourist spots and activities.

[0275] The server automatically generates a travel plan and sends it to the terminal, which displays it to the user for confirmation and adjustment.

[0276] 3. Real-time navigation

[0277] When a user travels to a tourist spot or other destination, the user inputs the destination into the terminal.

[0278] The device uses GPS to obtain its current location and transmits it to the server.

[0279] The server calculates the optimal route based on the current location and destination and generates detailed route guidance.

[0280] The device displays the route guidance received from the server on a map and provides real-time navigation.

[0281] 4. Payment and Cashless Payment

[0282] The user activates the payment function using the terminal and presents the QR code to the store.

[0283] The terminal sends the payment information to the server, which then executes the payment. Once the process is complete, the server sends the result to the terminal, which then displays a payment completion screen to the user.

[0284] 5. Emergency Support

[0285] When a user encounters an emergency, the user activates an emergency support function on the terminal and inputs the details of the emergency.

[0286] The device sends emergency information to the server, which then searches for information on emergency services and medical institutions closest to the current location and generates the data.

[0287] The device displays emergency information received from the server and provides translation functions as needed to help users communicate with local medical staff.

[0288] Specific examples

[0289] 1. Personal information registration:

[0290] Travelers enter personal information such as their name and nationality on a touch panel inside the vehicle and submit the information.

[0291] The server stores the data and sends a confirmation email.

[0292] 2. Create your travel plan:

[0293] Travelers enter their dates of stay and destination and choose from suggested attractions and activities.

[0294] The server automatically generates a travel plan and displays it on the in-car display.

[0295] 3. Real-time navigation:

[0296] The route from the current location to the next tourist spot is calculated, and the autonomous vehicle operates according to that route.

[0297] 4. Payment and Cashless Payment:

[0298] When making a purchase at a store, scan the QR code to complete the payment.

[0299] 5. Emergency Support:

[0300] When someone suddenly falls ill, the system searches for and navigates to the nearest hospital.

[0301] Prompt Sentence Examples

[0302] Design an autonomous mobility application with an integrated tourism support system. The app will have the following features:

[0303] 1. Registering user personal information and sending a confirmation email.

[0304] 2. Creating a travel plan and suggesting suitable tourist spots.

[0305] 3. Real-time navigation.

[0306] 4. Cashless payments within autonomous vehicles.

[0307] 5. Navigation to the nearest medical facility in case of an emergency.

[0308] Please also describe the data flow and the hardware and software used to realize these functions.

[0309] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0310] Step 1:

[0311] The user uses the touch panel in the car to enter personal information (name, nationality, emergency contact, email address), after which the device sends this information to the server and requests a confirmation email.

[0312] Input: Personal information entered by the user on the touch panel

[0313] Output: Personal information sent to the server, confirmation email request

[0314] Step 2:

[0315] The server stores the received personal information in a database, and once the storage is complete, the server sends a confirmation email to the user's email address, requesting that the account be activated.

[0316] Input: Personal information sent from the device, confirmation email request

[0317] Output: Personal information stored in database, confirmation email sent

[0318] Step 3:

[0319] The user uses the device to input their travel itinerary, destination, budget, and activities they are interested in. The device then sends this information to the server.

[0320] Input: Travel itinerary, destination, budget, and activities of interest entered by the user on the touch panel

[0321] Output: Trip information sent to the server

[0322] Step 4:

[0323] The server compares the received travel information with its database and selects appropriate tourist spots and activities, then automatically generates a travel plan and sends it to the device.

[0324] Input: Travel information

[0325] Output: Automatically generated itinerary, itinerary sent to the device

[0326] Step 5:

[0327] The user checks the travel plan and makes adjustments as necessary. The device sends the user's adjustment request to the server. The server then sends the final travel plan reflecting the adjustments to the device.

[0328] Input: User adjustment request

[0329] Output: Final itinerary with adjustments

[0330] Step 6:

[0331] When a user travels to a tourist spot or other destination, they input their destination into the device and activate the navigation function. The device then uses GPS to obtain their current location and sends it to the server.

[0332] Input: Destination or current location entered by the user on the touch panel

[0333] Output: Current location and destination sent to the server

[0334] Step 7:

[0335] The server calculates the optimal route based on the current location and destination, generates detailed route guidance including transportation information, and sends the generated route guidance to the device.

[0336] Input: Current location and destination

[0337] Output: Optimized travel route, directions sent to the device

[0338] Step 8:

[0339] The device displays the route guidance received from the server on a map and provides real-time navigation.

[0340] Input: Directions sent from the server

[0341] Output: Route directions displayed on a map

[0342] Step 9:

[0343] When using a store or service, the user activates the payment function on the device and presents the QR code to the store. The device then sends the payment information to the server and begins the payment process.

[0344] Input: Payment information entered by the user on the touch panel

[0345] Output: Payment information sent to the server

[0346] Step 10:

[0347] The server executes the payment based on the received payment information. Once the payment is complete, it sends the result to the terminal. The terminal then displays a payment completion screen to the user.

[0348] Input: Payment Information

[0349] Output: Payment result, payment result sent to the terminal

[0350] Step 11:

[0351] When a user encounters an emergency, the user activates the emergency support function using the terminal and inputs the details of the emergency, which then transmits the emergency information to the server.

[0352] Input: Emergency details entered by the user on the touch panel

[0353] Output: Urgent information sent to the server

[0354] Step 12:

[0355] The server searches for information on emergency services and medical institutions closest to the user's current location based on the emergency information, and sends it to the device. The device then displays the emergency information received from the server and provides translation functionality as needed.

[0356] Input: Emergency Information

[0357] Output: Information about nearby medical institutions, emergency information sent to the device

[0358] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0359] The present invention is an integrated system for helping inbound tourists have a comfortable stay in Japan, in which users, terminals, a server, and an emotion engine function in cooperation with each other. Specific embodiments for carrying out the present invention will be described below.

[0360] Account Registration

[0361] User:

[0362] A traveler launches the app and selects "Create a new account" on the initial screen. The user follows the app's instructions and enters personal information such as name, nationality, emergency contact, and email address.

[0363] Device:

[0364] The terminal transmits the personal information input by the user to the server.

[0365] server:

[0366] The server stores the received personal information in a database and sends the user a confirmation email, which the user must click to activate their account.

[0367] Creating a travel plan

[0368] User:

[0369] Users use the app to input their desired travel dates, destinations, budget, and activities they are interested in.

[0370] Emotion Engine:

[0371] The emotion engine recognizes emotions from the user's facial expressions, voice, text input, etc. and determines their emotional state.

[0372] Device:

[0373] The terminal transmits the input travel information and emotion information to the server.

[0374] server:

[0375] The server compares the travel information and emotion information with a database to select appropriate tourist spots and activities. The server adjusts the recommendations according to the emotional state recognized by the emotion engine. It then automatically generates a travel plan and presents it to the user.

[0376] Device:

[0377] The device displays the proposed travel plan to the user, prompting them to confirm and adjust it. The user then confirms the plan and finalizes it.

[0378] Real-time navigation and information

[0379] User:

[0380] When a user travels to a tourist spot or destination, they enter the destination into the app to activate the navigation function.

[0381] Device:

[0382] The device uses GPS to obtain its current location and transmits it to the server.

[0383] server:

[0384] The server calculates the optimal route based on the current location and destination, and generates detailed route guidance including transportation information.

[0385] Device:

[0386] The device displays the route guidance received from the server on a map and provides real-time navigation.

[0387] Payment and cashless payment

[0388] User:

[0389] When using a store or service, users activate the app's payment function and present the QR code to the store.

[0390] Device:

[0391] The terminal sends payment information to the server to initiate the payment process.

[0392] server:

[0393] The server checks the payment information, executes the payment, and sends the result to the terminal once the payment is complete.

[0394] Device:

[0395] The terminal will display a payment completion screen to the user and provide receipts and payment history as needed.

[0396] Emergency support

[0397] User:

[0398] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency.

[0399] Device:

[0400] The terminal transmits emergency information to the server.

[0401] server:

[0402] The server searches for information on emergency services and hospitals closest to the user's current location and generates data to provide to the user.

[0403] Device:

[0404] The device displays emergency information received from the server and provides translation functions as needed to help users communicate with local medical staff.

[0405] Recognizing and responding to user emotions

[0406] User:

[0407] A user's emotional state may change in various situations during travel, such as while moving around a tourist spot or participating in an activity.

[0408] Emotion Engine:

[0409] The emotion engine analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine the user's emotional state. If an emotion is recognized, the corresponding data is sent to the server.

[0410] server:

[0411] The server uses the emotion data to understand the user's current emotional state. For example, if the user is feeling stressed, it can suggest relaxing activities.

[0412] Device:

[0413] The terminal displays the suggestions from the server to the user and provides assistance in real time.

[0414] Specific examples

[0415] Example 1: Travel planning and emotional response

[0416] User:

[0417] Traveler A starts the app and plans a trip. The trip is for 7 days, with the main tourist destinations being Tokyo and Kyoto. The budget is 100,000 yen.

[0418] Traveler A selects cultural experiences and nature walks as activities that interest him.

[0419] Emotion Engine:

[0420] The emotion engine recognizes from Traveler A's voice and text input that he wants to relax.

[0421] Device:

[0422] The user's input information and emotion information are sent to the server.

[0423] server:

[0424] Based on traveler A's information and emotional data, the system suggests relaxing tourist spots such as Sensoji Temple, Fushimi Inari Taisha Shrine, and the bamboo forests of Kamakura. It also automatically generates a travel plan that includes a relaxing hot spring experience.

[0425] Device:

[0426] The travel plan is displayed to Traveler A, prompting them to confirm and adjust. Traveler A confirms and confirms the plan.

[0427] Example 2: Real-time emotion recognition and response

[0428] User:

[0429] Tourist B gets lost in the crowds of Tokyo and becomes stressed.

[0430] Emotion Engine:

[0431] The emotion engine recognizes stress from traveler B's facial expressions and tone of voice.

[0432] Device:

[0433] Emotion data is sent to the server.

[0434] server:

[0435] Based on current location data and emotional data, the app suggests nearby cafes and rest spots where Traveler B can relax.

[0436] Device:

[0437] The server displays the suggestions to Traveler B along with a map, and provides navigation to the rest spot.

[0438] In this way, by combining an emotion engine, the present invention is a system that can provide detailed responses according to the emotional state of inbound tourists, making their stay more comfortable.

[0439] The processing flow will be explained below.

[0440] Specific processing flow of the invention that combines the emotion engine

[0441] Account Registration

[0442] Step 1:

[0443] The user launches the app and selects "Create a new account" on the initial screen.

[0444] Step 2:

[0445] The user enters their name, nationality, emergency contact, and email address.

[0446] Step 3:

[0447] The terminal transmits the input information to the server.

[0448] Step 4:

[0449] The server stores the received information in a database.

[0450] Step 5:

[0451] The server sends a confirmation email to the user's email address.

[0452] Step 6:

[0453] The user activates their account by clicking the link in the confirmation email.

[0454] Step 7:

[0455] The device will display a screen indicating that the account registration was successful.

[0456] Travel planning and emotion recognition

[0457] Step 1:

[0458] The user selects "Create a trip" in the app.

[0459] Step 2:

[0460] The user enters desired travel dates, destinations, budget, and activities of interest.

[0461] Step 3:

[0462] The emotion engine recognizes emotions from the user's facial expressions, voice, and text input, and determines their emotional state.

[0463] Step 4:

[0464] The terminal transmits the input travel information and emotion information to the server.

[0465] Step 5:

[0466] The server compares the travel information and emotion information with a database and selects appropriate tourist spots and activities.

[0467] Step 6:

[0468] The server adjusts the content of the suggestions depending on the emotional state recognized by the emotion engine.

[0469] Step 7:

[0470] The server automatically generates an adjusted travel plan and sends it to the terminal.

[0471] Step 8:

[0472] The device displays the travel plan to the user and prompts them to confirm and adjust it.

[0473] Step 9:

[0474] The user reviews the plan and finalizes it.

[0475] Real-time navigation

[0476] Step 1:

[0477] The user enters a destination in the app and launches the navigation function.

[0478] Step 2:

[0479] The device uses GPS to obtain its current location and sends it to the server.

[0480] Step 3:

[0481] The server calculates the optimal route based on the current location and destination.

[0482] Step 4:

[0483] The server sends route guidance information to the terminal.

[0484] Step 5:

[0485] The device displays route guidance on a map and provides real-time navigation to the user.

[0486] Payment and cashless payment

[0487] Step 1:

[0488] When the user visits a store or uses a service, they activate the app's payment function.

[0489] Step 2:

[0490] The user confirms the payment amount and scans the QR code.

[0491] Step 3:

[0492] The terminal sends the payment information to the server.

[0493] Step 4:

[0494] The server verifies the payment information and executes the payment.

[0495] Step 5:

[0496] The server sends the payment result to the terminal.

[0497] Step 6:

[0498] The terminal displays a payment completion screen to the user.

[0499] Emergency support

[0500] Step 1:

[0501] The user launches the emergency support function and enters the details of the emergency.

[0502] Step 2:

[0503] The terminal transmits emergency information to the server.

[0504] Step 3:

[0505] The server searches for information about emergency services and hospitals closest to your location.

[0506] Step 4:

[0507] The server sends the search results to the terminal.

[0508] Step 5:

[0509] The terminal displays the emergency information to the user.

[0510] Step 6:

[0511] The device provides translation functionality as needed to help users communicate with local medical staff.

[0512] Recognizing and responding to user emotions

[0513] Step 1:

[0514] The emotion engine analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine their emotional state.

[0515] Step 2:

[0516] The emotion engine sends the recognized emotion data to the server.

[0517] Step 3:

[0518] The server understands the user's current emotional state based on the emotional data.

[0519] Step 4:

[0520] The server selects appropriate tourist spots and activities based on the user's emotional state and adjusts the suggestions accordingly.

[0521] Step 5:

[0522] The server sends the adjusted proposal to the terminal.

[0523] Step 6:

[0524] The device displays the suggestions to the user and provides assistance in real time.

[0525] Specific examples

[0526] Example 1: Travel planning and emotional response

[0527] Step 1:

[0528] The user launches the app and plans a trip.

[0529] Step 2:

[0530] The user inputs their travel dates, destinations, budget, and activities of interest (e.g., cultural experiences or nature walks).

[0531] Step 3:

[0532] The emotion engine recognizes the user's desire to relax from their voice and text input.

[0533] Step 4:

[0534] The terminal transmits travel information and emotion information to the server.

[0535] Step 5:

[0536] The server selects relaxing tourist spots and activities (e.g., Sensoji Temple, Fushimi Inari Taisha Shrine, hot spring experience) based on travel information and emotional data.

[0537] Step 6:

[0538] The server sends the automatically generated travel plan to the terminal.

[0539] Step 7:

[0540] The device displays the travel plan to the user and prompts them to confirm and adjust it.

[0541] Step 8:

[0542] The user reviews the plan and finalizes it.

[0543] Example 2: Real-time emotion recognition and response

[0544] Step 1:

[0545] Users get lost in the crowds of Tokyo and feel stressed.

[0546] Step 2:

[0547] The emotion engine recognizes stress from the user's facial expressions and tone of voice.

[0548] Step 3:

[0549] The device transmits the emotion data to the server.

[0550] Step 4:

[0551] The server selects a nearby cafe or rest spot where the user can relax based on the user's current location data and emotional data.

[0552] Step 5:

[0553] The server transmits the selection result to the terminal.

[0554] Step 6:

[0555] The device provides navigation to rest spots.

[0556] In this way, by combining an emotion engine, the present invention is a system that can provide detailed responses according to the emotional state of inbound tourists, making their stay more comfortable.

[0557] Example 2

[0558] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0559] Conventional travel support systems have difficulty understanding travelers' individual needs and emotional state in real time and making optimal recommendations based on that information. Furthermore, there is uncertainty about how to respond to troubles and emergencies during travel, and the infrastructure for travelers to enjoy their stay with peace of mind is insufficient. Furthermore, there is a lack of appropriate support and guidance tailored to the location and situation, and there is a need to improve travelers' satisfaction.

[0560] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0561] In this invention, the server includes: means for a user to input personal information; means for transmitting the input personal information to the server; means for storing the transmitted personal information in a database; means for sending a confirmation email based on the stored data; means for activating the account via a link from the confirmation email; means for a user to input travel information; means for analyzing the user's emotions using an emotion recognition engine; means for transmitting the input travel information and the analyzed emotion information to the server; means for suggesting appropriate tourist spots and activities based on the transmitted travel information and emotion information; means for displaying the proposed travel plan to the user; means for the user to use a GPS function to calculate and display an optimal travel route; and means for displaying the travel route proposal on the user terminal. This enables optimal suggestions and support to be provided in real time according to the individual needs and emotional state of the traveler.

[0562] "Personal Information" is data that can individually identify you, including information such as your name, nationality, emergency contact details, and email address.

[0563] A "server" is a computer system that receives, processes, and stores data sent by users.

[0564] A "database" is a system for structuring, storing, and managing users' personal information, travel information, and emotional information.

[0565] A "confirmation email" is an email sent to the email address entered by the user to prompt them to activate their account.

[0566] "Travel information" is data including information entered by the user such as travel itinerary, destination, budget, and activities of interest.

[0567] An "emotion recognition engine" is a system that analyzes a user's facial expressions, tone of voice, text input, etc. to determine the user's emotional state.

[0568] "Suggestion" refers to the automatic selection of appropriate tourist spots and activities based on the travel information and emotional information entered by the user.

[0569] A "travel plan" is a specific travel itinerary automatically generated by the server according to the user's emotional state and requirements.

[0570] A "user terminal" is a device used by a user to input information and display suggested travel plans and navigation information.

[0571] The "GPS function" is a function that uses the Global Positioning System to obtain the user's current location.

[0572] "Navigation" is a function that presents the optimal route based on the current location and destination, and provides real-time guidance.

[0573] The present invention is an integrated system for helping inbound tourists have a comfortable stay in Japan, in which users, terminals, a server, and an emotion engine function in cooperation with each other. Specific embodiments for carrying out the present invention will be described below.

[0574] Account Registration

[0575] User:

[0576] A traveler launches the app and selects "Create a new account" on the initial screen. The user then enters personal information such as name, nationality, emergency contact, and email address.

[0577] Device:

[0578] The personal information entered by the user is sent to the server.

[0579] server:

[0580] The server stores the received personal information in a database and sends a confirmation email to the user, who clicks on a link in the email to activate their account.

[0581] Creating a travel plan

[0582] User:

[0583] Users use the app to input their desired travel dates, destinations, budget, and activities they are interested in.

[0584] Emotion Engine:

[0585] The emotion engine recognizes emotions from the user's facial expressions, voice, text input, etc. and determines their emotional state.

[0586] Device:

[0587] The entered travel information and analyzed emotion information are sent to the server.

[0588] server:

[0589] The system compares travel information and emotion information with a database to select appropriate tourist spots and activities. Based on the analysis results of the emotion engine, the system adjusts the suggestions and automatically generates a travel plan to present to the user.

[0590] Device:

[0591] The travel plan is displayed to the user and they are prompted to confirm and adjust it. The user confirms the plan and finalizes it.

[0592] Real-time navigation and information

[0593] User:

[0594] When traveling to a tourist spot or destination, enter the destination into the app and activate the navigation function.

[0595] Device:

[0596] Use the GPS function to obtain the current location and send it to the server.

[0597] server:

[0598] Calculates the optimal travel route based on your current location and destination, and generates detailed route guidance including transportation information.

[0599] Device:

[0600] Route guidance is displayed on a map and navigation is provided in real time.

[0601] Payment and cashless payment

[0602] User:

[0603] When using a store or service, activate the app's payment function and present the QR code to the store.

[0604] Device:

[0605] Sends payment information to the server to begin the checkout process.

[0606] server:

[0607] Confirm the payment information and execute the payment. Once the payment is complete, the result will be sent to the terminal.

[0608] Device:

[0609] The payment completion screen is displayed to the user, and a receipt and payment history are provided as needed.

[0610] Emergency support

[0611] User:

[0612] When you encounter an emergency, activate the app's emergency support function and enter the details of the emergency.

[0613] Device:

[0614] Send emergency information to the server.

[0615] server:

[0616] Search for information on emergency services and hospitals closest to your current location and generate data to provide to the user.

[0617] Device:

[0618] It displays emergency information and provides translation functionality where necessary to help users communicate with local medical staff.

[0619] Recognizing and responding to user emotions

[0620] User:

[0621] Your emotional state will change depending on the situation you are in while traveling.

[0622] Emotion Engine:

[0623] It analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine their emotional state. If an emotion is recognized, the data is sent to the server.

[0624] server:

[0625] Based on emotional data, the system grasps the user's current emotional state and suggests activities that will help them relax.

[0626] Device:

[0627] The server displays suggestions to the user, providing real-time assistance.

[0628] Specific examples

[0629] Example 1: Travel planning and emotional response

[0630] User:

[0631] Traveler A starts the app and sets a seven-day travel itinerary. The main tourist destinations are Tokyo and Kyoto, and the budget is 100,000 yen. The traveler also enters that they are interested in cultural experiences and nature walks.

[0632] Emotion Engine:

[0633] Recognize Traveler A's desire to relax from his voice and text input.

[0634] Device:

[0635] The input information and emotion information are sent to the server.

[0636] server:

[0637] Based on traveler A's information and emotional data, the system suggests relaxing tourist spots such as Sensoji Temple, Fushimi Inari Taisha Shrine, and the bamboo forests of Kamakura. It also automatically generates a travel plan that includes a relaxing hot spring experience.

[0638] Device:

[0639] The travel plan is displayed and traveler A is prompted to confirm and adjust the plan. Traveler A confirms and confirms the plan.

[0640] Example 2: Real-time emotion recognition and response

[0641] User:

[0642] Tourist B gets lost in the crowds of Tokyo and becomes stressed.

[0643] Emotion Engine:

[0644] Recognize stress from traveler B's facial expressions and tone of voice.

[0645] Device:

[0646] Emotion data is sent to the server.

[0647] server:

[0648] Based on current location data and emotional data, the app suggests nearby cafes and rest spots where Traveler B can relax.

[0649] Device:

[0650] The suggestions are displayed along with a map, and navigation to rest spots is provided.

[0651] Generative AI model prompt

[0652] Example prompt 1:

[0653] "I want to create a 7-day travel plan for Traveler A. The main tourist destinations are Tokyo and Kyoto, and the budget is 100,000 yen. The traveler is interested in cultural experiences and nature walks, and wants to relax. Please automatically generate a travel plan based on these conditions."

[0654] Example prompt 2:

[0655] "Traveller B is lost in the crowds of Tokyo and feeling stressed. Based on their current location and emotional data, please suggest a nearby cafe or rest spot where they can relax."

[0656] In this way, by combining an emotion engine, the system of this invention aims to provide detailed responses in real time according to the emotional state of inbound tourists, making their stay more comfortable. In addition, by utilizing a generative AI model, it is possible to make quick and accurate suggestions based on user needs.

[0657] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0658] Account Registration

[0659] Step 1: Launch the app and select Create a new account

[0660] User:

[0661] Launch the app and select "Create a new account" on the initial screen. Input: None. Output: New account creation screen.

[0662] Step 2: Enter your personal information

[0663] User:

[0664] Enter personal information such as name, nationality, emergency contact, email address, etc. Input: User information. Output: Entered personal information.

[0665] Step 3: Submit your personal information

[0666] Device:

[0667] The entered personal information is sent to the server. Input: Personal information provided by the user. Output: Data sent to the server.

[0668] Step 4: Save to database and send confirmation email

[0669] server:

[0670] The received personal information is stored in a database and a confirmation email is sent to the user. Input: Personal information from the terminal. Output: Data stored in the database and the confirmation email sent.

[0671] Step 5: Activate your account

[0672] User:

[0673] Click the link in the confirmation email to activate your account. Input: Link in confirmation email. Output: Account activation confirmation message.

[0674] Creating a travel plan

[0675] Step 1: Enter your travel information

[0676] User:

[0677] Enter your desired travel dates, destinations, budget, and activities you're interested in into the app. Input: Trip information. Output: Entered trip information.

[0678] Step 2: Analyze emotional state

[0679] Emotion Engine:

[0680] Recognizes emotions from facial expressions, voice, and text input, and determines emotional state. Input: User's facial expressions, voice, and text data. Output: Analyzed emotional information.

[0681] Step 3: Sending information to the server

[0682] Device:

[0683] Send travel information and emotion information to the server. Input: Travel information, emotion information. Output: Data sent to the server.

[0684] Step 4: Collate the data and generate a plan

[0685] server:

[0686] The database is collated based on travel information and emotional information to select appropriate tourist spots and activities. The suggestions are adjusted based on the analysis results of the emotional engine, and a travel plan is automatically generated. Input: Travel information, emotional information. Output: Automatically generated travel plan.

[0687] Step 5: Present and confirm your plan

[0688] Device:

[0689] Display the itinerary to the user and prompt for confirmation and adjustment. Input: Generated itinerary. Output: Itinerary presented to the user.

[0690] Real-time navigation and information

[0691] Step 1: Enter your destination

[0692] User:

[0693] Enter your destination into the app and launch the navigation function. Input: Destination information. Output: Navigation start screen.

[0694] Step 2: Get your current location

[0695] Device:

[0696] Uses GPS to obtain current location and sends it to the server. Input: Current location information from GPS. Output: Data sent to the server.

[0697] Step 3: Calculating the travel route

[0698] server:

[0699] Calculates the optimal travel route based on the current location and destination, and generates detailed route guidance including transportation information. Input: Current location, destination information. Output: Calculated travel route.

[0700] Step 4: Provide directions

[0701] Device:

[0702] It displays route directions on a map and provides real-time navigation. Input: Calculated travel route. Output: Route directions displayed on a map.

[0703] Payment and cashless payment

[0704] Step 1: Launching the payment function

[0705] User:

[0706] When using a store or service, activate the app's payment function and present the QR code to the store. Input: None. Output: QR code display screen.

[0707] Step 2: Submit your payment information

[0708] Device:

[0709] Sends payment information to the server and starts the payment process. Input: QR code information. Output: Data sent to the server.

[0710] Step 3: Confirm payment

[0711] server:

[0712] Confirm the payment information and execute the payment. Once the payment is completed, the result is sent to the terminal. Input: Payment information. Output: Payment result data.

[0713] Step 4: Payment completion screen displayed

[0714] Device:

[0715] Displays a payment completion screen to the user, and optionally provides a receipt and payment history. Input: Payment result. Output: Payment completion screen and receipt.

[0716] Emergency support

[0717] Step 1: Activating the Emergency Support Feature

[0718] User:

[0719] When you encounter an emergency, activate the app's emergency support function and enter the details of the emergency. Input: Details of the emergency. Output: Emergency support screen.

[0720] Step 2: Send emergency information

[0721] Device:

[0722] Send emergency information to the server. Input: Emergency information. Output: Data sent to the server.

[0723] Step 3: Find and provide emergency services

[0724] server:

[0725] Search for information on emergency services and hospitals closest to the current location and generate data to provide to the user. Input: Emergency information, current location information. Output: Emergency service information.

[0726] Step 4: Displaying emergency information and translation assistance

[0727] Device:

[0728] It displays emergency information and provides translation functionality where necessary to help users communicate with local medical staff. Input: Emergency service information. Output: Displayed emergency information, translated results.

[0729] Recognizing and responding to user emotions

[0730] Step 1: Change your emotions

[0731] User:

[0732] The emotional state changes depending on the situation during the trip. Input: The situation of the user during the trip. Output: The changed emotional state.

[0733] Step 2: Determine your emotional state

[0734] Emotion Engine:

[0735] The system analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine their emotional state. If an emotion is recognized, the data is sent to the server. Input: Facial expressions, tone of voice, text data. Output: Analyzed emotional state, data sent to the server.

[0736] Step 3: Emotional information analysis

[0737] server:

[0738] Based on emotion data, the system understands the user's current emotional state and suggests activities that will help them relax. Input: Emotion data. Output: Activity suggestions.

[0739] Step 4: View the proposal

[0740] Device:

[0741] The server displays suggestions to the user, providing real-time assistance. Input: Suggested activity. Output: Displayed suggestions.

[0742] (Application example 2)

[0743] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0744] Today's inbound travelers are increasingly seeking a comfortable travel experience through the use of hardware and software. However, very few systems offer personalized recommendations based on their emotional state. This can lead to stress during travel and difficulty in selecting the right product. Furthermore, the lack of real-time navigation and personalized information can lead to travelers becoming lost or not receiving appropriate assistance.

[0745] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input personal information, means for saving the input personal information in a database, means for proposing individual services based on the saved data, means for recognizing the user's emotional state using an emotion engine, means for suggesting items and rest spots that will help the user relax based on the recognized emotional state, and means for transmitting the suggestions to the user terminal. This makes it possible to provide individual services that are in line with the traveler's emotional state, thereby achieving a comfortable travel experience.

[0746] "Personal information" is information that can identify a specific person, such as a user's name, nationality, contact information, and email address.

[0747] A "database" is a system that systematically stores information such as input personal information and proposal details, allowing for rapid search and use.

[0748] An "emotion engine" is software or an algorithm that analyzes and determines a user's emotional state based on facial expressions, tone of voice, text input, etc.

[0749] "Suggestions" are presentations of individual services such as tourist spots, activities, relaxation items, and rest spots that are generated based on the user's input information and emotional state.

[0750] A "user terminal" is a portable computing device such as a smartphone or tablet used by a traveler.

[0751] The "GPS function" is a global positioning system that allows precise acquisition of location information for any point on Earth.

[0752] A "generative AI model" is an artificial intelligence model that learns from large datasets, extracts certain patterns and features, and generates new data.

[0753] A "prompt sentence" is a sentence that serves as data or instructions to be input into a generative AI model.

[0754]

[0755] The present invention is an integrated system that uses an emotion engine to improve the travel experience of inbound tourists, and functions in cooperation with users, terminals, and servers. Specific embodiments for implementing the present invention will be described below.

[0756] Account Registration

[0757] A user launches the app and enters their personal information. The device sends the information to the server, which stores it in a database and sends a confirmation email. The user clicks on a link in the email to activate their account.

[0758] Creating a travel plan

[0759] Users input their travel itinerary, destinations, budget, and activities of interest into the app. The emotion engine analyzes the user's emotional state from their facial expressions, tone of voice, and text input, and sends this information to the server. The server then selects matching tourist spots and activities based on the database, automatically generates a travel plan, and sends it to the user's device.

[0760] Suggestions and Emotion Recognition

[0761] The emotion engine assesses the user's real-time emotional state and suggests items and resting spots to help them relax if they are feeling stressed. The server sends these suggestions to the user's device, where the user can confirm the information presented.

[0762] Navigating travel routes

[0763] When a user travels to a tourist spot or other destination, the current location is acquired using the GPS function and sent to the server. The server then calculates the optimal route and displays detailed route guidance on the user's device. Route guidance can also be generated in real time using a generative AI model by entering a prompt sentence.

[0764] Payment and cashless payment

[0765] When a user finds a product in the virtual shopping area, they launch the app's payment function and scan the QR code. The device sends the payment information to the server, which then executes the payment process and sends the results back to the device. A payment completion screen is displayed on the user's device, along with a receipt and payment history.

[0766] Emergency Support

[0767] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency. The device then sends the emergency information to a server, which then provides information on the nearest emergency services and medical institutions. The app also provides a translation function to help users communicate with local medical staff.

[0768] Hardware or software used

[0769] The following hardware and software are used to implement this system.

[0770] Smartphone (iOS or ANDROID (registered trademark))

[0771] Emotion engine: Google Cloud's Natural Language API and Microsoft® Azure®'s Text Analytics API

[0772] GPS module: Built-in GPS in smartphone

[0773] Database: Firebase Firestore or Amazon RDS

[0774] Payment system: Stripe or PayPal API

[0775] Specific examples

[0776] Tourist C is exploring a virtual shopping area in Japan. The emotion engine recognizes from his tone of voice that he is tired, and the app suggests items and resting spots to help him relax. The user can use the app's help function to find more information about these items and how to spend their time.

[0777] Prompt Sentence Examples

[0778] A user launches the "Virtual Tourist Shopping Assistant" application and walks around a virtual shopping area. The system detects fatigue from the user's facial expressions and tone of voice. Please suggest items and resting spots that will help them relax.

[0779] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0780] Step 1:

[0781] A user launches the app and enters their personal information (such as name, nationality, contact details, and email address). The device sends the information to the server, which stores it in a database and sends a confirmation email. The user clicks on a link in the confirmation email to activate their account.

[0782] Step 2:

[0783] The user enters travel information such as travel itinerary, destinations, budget, and activities of interest into the app. The device sends this travel information to the server. The emotion engine analyzes the user's emotional state from their facial expressions, tone of voice, and text input, and sends the results to the server. The server compares the received data with a database to select the most suitable tourist spots and activities.

[0784] Step 3:

[0785] The server automatically generates a travel plan based on the selected tourist spots and activities and sends it to the user's device. The user can then review the proposed plan on their device and make adjustments as necessary. The final travel plan is then finalized.

[0786] Step 4:

[0787] When a user travels to a tourist spot or other destination, they activate the navigation function using the app. The device's GPS function acquires the user's current location and sends that information to the server. The server calculates the optimal route based on the current location and destination, and generates the route in real time based on a generation AI model. The generated route guidance is sent to the user's device and displayed on a map.

[0788] Step 5:

[0789] When a user finds a product in the virtual shopping area, they launch the app's payment function and scan the QR code. The terminal sends the payment information to the server, which then executes the payment process and sends the result to the terminal. The user sees a payment completion screen, and is provided with a receipt and payment history.

[0790] Step 6:

[0791] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency. The device then sends this emergency information to a server. The server then searches for information on the nearest emergency service or medical institution based on the user's current location and sends that information to the user's device. The translation function also helps users communicate with local medical staff.

[0792] Step 7:

[0793] If a user experiences emotional changes while traveling or participating in an activity, the emotion engine analyzes them in real time and sends the results to the server. Based on the user's emotional state, the server creates suggestions for relaxing items and rest spots using a generative AI model and sends the suggestions to the user's device. The user can then review the suggestions and receive any support they need.

[0794] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0795] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0796] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0797] [Second embodiment]

[0798] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0799] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0800] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0801] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0802] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0803] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0804] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0805] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0806] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0807] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0808] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0809] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0810] The present invention is an integrated system for enabling inbound tourists to enjoy their stay in Japan smoothly, and functions in cooperation with users, terminals, and servers. Specific embodiments for carrying out the present invention will be described below.

[0811] Account Registration

[0812] User:

[0813] The first step a traveler takes after launching the app is to create an account. Users follow the app's instructions and enter personal information such as their name, nationality, emergency contact details, and email address.

[0814] Device:

[0815] The terminal transmits the personal information input by the user to the server.

[0816] server:

[0817] The server stores the received personal information in a database, and once the storage is complete, a confirmation email is sent to the user's email address, requesting that they activate their account.

[0818] Creating a travel plan

[0819] User:

[0820] Users use the app to input their desired travel dates, destinations, budget, and activities they are interested in.

[0821] Device:

[0822] The terminal transmits the input information to the server.

[0823] server:

[0824] The server compares the travel information with the database, selects appropriate tourist spots and activities, and automatically generates and proposes a travel plan to the user.

[0825] Device:

[0826] The device displays the proposed travel plan to the user, who can review the plan, make adjustments as needed, and finalize the plan.

[0827] Real-time navigation and information

[0828] User:

[0829] When a user travels to a tourist spot or destination, they enter the destination into the app to activate the navigation function.

[0830] Device:

[0831] The device uses GPS to obtain its current location and transmits it to the server.

[0832] server:

[0833] The server calculates the optimal route based on the current location and destination, and generates detailed route guidance including transportation information.

[0834] Device:

[0835] The device displays the route guidance received from the server on a map and provides real-time navigation.

[0836] Payment and cashless payment

[0837] User:

[0838] When using a store or service, users activate the app's payment function and present the QR code to the store.

[0839] Device:

[0840] The terminal sends payment information to the server to initiate the payment process.

[0841] server:

[0842] The server checks the payment information, executes the payment, and sends the result to the terminal once the payment is complete.

[0843] Device:

[0844] The terminal will display a payment completion screen to the user and provide receipts and payment history as needed.

[0845] Emergency support

[0846] User:

[0847] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency.

[0848] Device:

[0849] The terminal transmits emergency information to the server.

[0850] server:

[0851] The server searches for emergency services and hospital information closest to the current location and generates data to provide to the user.

[0852] Device:

[0853] The device displays emergency information received from the server and provides translation functions as needed to help users communicate with local medical staff.

[0854] Specific examples

[0855] Example 1: Creating a travel plan

[0856] User:

[0857] Traveler A starts the app and plans a trip. The trip is for 7 days, with the main tourist destinations being Tokyo and Kyoto. The budget is 100,000 yen.

[0858] Device:

[0859] The user's input information is sent to the server.

[0860] server:

[0861] Based on the information provided by Traveler A, the plan suggests tourist spots such as Sensoji Temple, Skytree, Fushimi Inari Taisha Shrine, and Kiyomizu-dera Temple. In addition, activities such as a maiko experience and sushi-making experience are also included in the plan.

[0862] Device:

[0863] The created travel plan is displayed to Traveler A, prompting them to confirm and make adjustments.

[0864] Example 2: Real-time navigation

[0865] User:

[0866] Traveler B needs navigation to travel from Tokyo to Osaka. He enters "Tokyo to Osaka" into the app.

[0867] Device:

[0868] Use the GPS function to obtain the current location (Tokyo Station) and send it to the server.

[0869] server:

[0870] The system calculates the optimal travel route from Tokyo Station to Osaka Station, including using the Shinkansen, and generates detailed navigation information.

[0871] Device:

[0872] It displays the travel route received from the server and provides real-time information on the next train and transfers.

[0873] In this way, the present invention provides a comprehensive support system to help inbound travelers enjoy their stay in Japan to the fullest.

[0874] The processing flow will be explained below.

[0875] Account Registration

[0876] Step 1:

[0877] The user launches the app and selects "Create a new account" on the initial screen.

[0878] Step 2:

[0879] The user enters their name, nationality, emergency contact, and email address.

[0880] Step 3:

[0881] The terminal transmits the input information to the server.

[0882] Step 4:

[0883] The server stores the received information in a database.

[0884] Step 5:

[0885] The server sends a confirmation email to the user's email address.

[0886] Step 6:

[0887] The user activates their account by clicking the link in the confirmation email.

[0888] Step 7:

[0889] The device will display a screen indicating that the account registration was successful.

[0890] Creating a travel plan

[0891] Step 1:

[0892] The user selects "Create a trip" in the app.

[0893] Step 2:

[0894] The user enters desired travel dates, destinations, budget, and activities of interest.

[0895] Step 3:

[0896] The terminal transmits the input travel information to the server.

[0897] Step 4:

[0898] The server compares the travel information with a database and selects appropriate attractions and activities.

[0899] Step 5:

[0900] The server automatically generates a travel plan based on the selected tourist spots and activities.

[0901] Step 6:

[0902] The server transmits the generated travel plan to the terminal.

[0903] Step 7:

[0904] The device displays the travel plan to the user and prompts them to confirm and adjust it.

[0905] Step 8:

[0906] The user reviews and confirms the plan.

[0907] Real-time navigation

[0908] Step 1:

[0909] The user enters a destination in the app and launches the navigation function.

[0910] Step 2:

[0911] The device uses GPS to obtain its current location and sends it to the server.

[0912] Step 3:

[0913] The server calculates the optimal route based on the current location and destination.

[0914] Step 4:

[0915] The server sends route guidance information to the terminal.

[0916] Step 5:

[0917] The device displays route guidance on a map and provides real-time navigation to the user.

[0918] Payment and cashless payment

[0919] Step 1:

[0920] When the user visits a store or uses a service, they activate the app's payment function.

[0921] Step 2:

[0922] The user confirms the payment amount and scans the QR code.

[0923] Step 3:

[0924] The terminal sends the payment information to the server.

[0925] Step 4:

[0926] The server verifies the payment information and executes the payment.

[0927] Step 5:

[0928] The server sends the payment result to the terminal.

[0929] Step 6:

[0930] The terminal displays a payment completion screen to the user.

[0931] Emergency support

[0932] Step 1:

[0933] The user launches the emergency support function and enters the details of the emergency.

[0934] Step 2:

[0935] The terminal transmits emergency information to the server.

[0936] Step 3:

[0937] The server searches for information about emergency services and hospitals closest to your location.

[0938] Step 4:

[0939] The server sends the search results to the terminal.

[0940] Step 5:

[0941] The terminal displays the emergency information to the user.

[0942] Step 6:

[0943] The device provides translation functionality as needed to help users communicate with local medical staff.

[0944] Example 1

[0945] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0946] Today's travelers are expected to plan their own trips, navigate their way around, make payments, and even respond to emergencies all in one place, but existing systems struggle to provide integrated support for all of these. It's particularly difficult to provide a single platform for planning sightseeing trips, real-time navigation, cashless payments, and emergency response. As a result, travelers are forced to use multiple applications and services, which creates hassle and confusion that can degrade the quality of their trip.

[0947] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0948] In this invention, the server includes a means for a user to input personal information, a means for saving the input personal information in a database, and a means for sending a confirmation email of account activation to the user's email address after saving is complete, thereby enabling a user to manage all travel-related information on one platform and enjoy personalized services.

[0949] "User" refers to a traveler who uses this system.

[0950] "Terminal" refers to an electronic device such as a smartphone or tablet used by a user.

[0951] "Server" refers to a computer system that processes, stores, and manages data for the entire system.

[0952] "Personal information" refers to information entered by the user, such as name, nationality, emergency contact information, and email address.

[0953] "Database" refers to a system for storing and managing information such as personal information and travel plans.

[0954] "Verification Email" means the email sent to a User's email address to activate their account.

[0955] "Travel itinerary" refers to the planned duration of the trip entered by the user.

[0956] "Destinations" refer to places or cities that a user plans to visit during their trip.

[0957] "Budget" refers to the amount of money a user plans to spend on a trip.

[0958] "Activities" refer to experiential tourist events and activities suggested based on the user's interests.

[0959] "GPS Function" refers to the location technology used by a Device to obtain its current location.

[0960] "Travel route" refers to the route from the current location to the destination.

[0961] "Payment functionality" refers to the functionality of the app that allows users to make cashless payments.

[0962] "Payment System" means an online payment platform for processing payment information.

[0963] "Emergency Support Feature" refers to the feature of the App that enables a User to receive necessary support in an emergency.

[0964] "Emergency information" refers to the user's report of an emergency situation.

[0965] "Translation Feature" refers to the automatic translation feature used by Users to communicate in their local language.

[0966] This invention is an integrated system that enables inbound tourists to enjoy their stay in Japan smoothly. The user, terminal, and server work together, allowing the user to manage their entire trip on a single platform.

[0967] First, the user launches the app using a device such as a smartphone or tablet. When the app is launched for the first time, the user enters their personal information, which is then sent from the device to the server. The server stores the received information in a database and sends a confirmation email to the user's email address to activate their account. The account is activated when the user approves this confirmation email.

[0968] Next, the user enters their travel itinerary, destinations, budget, and activities of interest into the app. The device sends this travel information to the server, which compares it with a database and uses a generative AI model (e.g., GPT-4) to suggest suitable attractions and activities. The proposed plan is automatically generated, saved in JSON format, and then sent to the user's device. The device displays the received plan to the user and makes adjustments if necessary.

[0969] Furthermore, when a user travels to a tourist spot or other destination, the device's GPS function is used to send the user's current location to the server. The server then calculates the optimal route from the user's current location to the destination. Detailed navigation information is generated and sent to the device. The device then displays this information on a map and provides real-time navigation.

[0970] When a user visits a store or uses a service, they activate the app's payment function and present the QR code to the store. This payment information is sent from the device to the server, and the server executes the payment using a payment system (e.g., Stripe, PayPal). The payment completion result is sent to the terminal, which displays it to the user.

[0971] In an emergency, the user enters emergency information using the app's emergency support function. This information is sent from the device to a server, which searches for the most appropriate emergency service or hospital information. The search results are sent to the device and displayed there. Furthermore, a translation function is provided if necessary, allowing the user to communicate with local staff.

[0972] Specific examples

[0973] Example 1: Creating a travel plan

[0974] User A launches the app and creates a seven-day travel plan. The destinations entered are Tokyo and Kyoto, with a budget of 100,000 yen. The entered information is sent to the server, which then suggests tourist spots such as Sensoji Temple, Skytree, Fushimi Inari Taisha Shrine, and Kiyomizu-dera Temple. Activities such as a maiko experience and a sushi-making experience are also incorporated into the plan. The generated plan is displayed on the device to User A, who then reviews the contents and finalizes the plan.

[0975] Example 2: Real-time navigation

[0976] User B uses the app's navigation function when traveling from Tokyo to Osaka. The current location (Tokyo Station) is obtained using the GPS function and sent to the server. The server calculates the optimal route from Tokyo Station to Osaka Station, generates detailed navigation including Shinkansen information, and sends it to the device. The device displays this on a map and provides real-time information on the next train and transfers.

[0977] Prompt Sentence Examples

[0978] "Generate a 7-day travel plan with a budget of 100,000 yen, focusing on Tokyo and Kyoto."

[0979] As described above, the present invention allows users to manage all travel-related information in one system, providing a smooth travel experience.

[0980] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0981] Step 1:

[0982] User information entry and submission

[0983] A user launches the app and navigates to the account creation screen, where they enter personal information such as their name, nationality, emergency contact, and email address. After entering the information, the device encrypts it using the HTTPS protocol and sends it to the server. The input contains personal information, and the output is encrypted data.

[0984] Step 2:

[0985] Server saves information and sends confirmation email

[0986] The server stores the received personal information in a database (e.g., MySQL). If the storage is successful, the server sends a confirmation email of account activation to the user's email address via the SMTP server. The input is the received encrypted personal information, and the output is the storage result in the database and the confirmation email.

[0987] Step 3:

[0988] User's travel information input and submission

[0989] The user inputs their travel itinerary, destinations, budget, and activities of interest into the app. The device sends this information to the server. The input is information about the trip, and the output is data sent to the server.

[0990] Step 4:

[0991] Travel plan generation by server

[0992] The server compares the received travel information with a database (e.g., MongoDB), selects appropriate tourist spots and activities using a generative AI model (e.g., GPT-4), and automatically generates a travel plan. The generated plan is saved in JSON format and sent to the device. The input is travel information, and the output is the generated travel plan.

[0993] Step 5:

[0994] View and adjust travel plans on your device

[0995] The device displays the travel plan received from the server to the user. The user reviews the plan and makes adjustments as necessary. If adjustments are necessary, the device sends the information back to the server. The input is the generated travel plan, and the output is the adjusted plan approval.

[0996] Step 6:

[0997] Launching the navigation function

[0998] When a user travels to a tourist spot or other destination, they input their destination into the app and activate the navigation function. The device uses GPS to obtain current location information and sends it to the server. The input is destination information and current location information, and the output is the data sent to the server.

[0999] Step 7:

[1000] Route calculation by the server

[1001] The server calculates the optimal route based on the received current location and destination information using the Google Maps API and route calculation algorithms. Detailed route guidance is generated in JSON format and sent to the device. The input is the current location information and destination information, and the output is the route guidance.

[1002] Step 8:

[1003] Displaying travel route on device

[1004] The device displays the route guidance received from the server on a map and provides real-time navigation. The user moves around while checking the app screen. The input is the route guidance, and the output is the display of navigation information.

[1005] Step 9:

[1006] Activating payment functions and sending information

[1007] When using a store or service, the user activates the app's payment function, selects a payment method, and presents a QR code. The terminal then sends the payment information to the server. The input is the payment information, and the output is the data sent to the server.

[1008] Step 10:

[1009] Server-based payment execution

[1010] The server processes the received payment information via a payment system (e.g., Stripe, PayPal). After the payment is completed, the result is sent to the terminal. The input is the payment information, and the output is the payment result.

[1011] Step 11:

[1012] Terminal indicates payment is complete

[1013] The terminal receives the payment result from the server, displays it to the user, and provides receipts and payment history as needed. The input is the payment result, and the output is a payment completion screen.

[1014] Step 12:

[1015] Enter and submit emergency information

[1016] When a user encounters an emergency, they activate the app's emergency support function and input the details of the emergency. The device then sends the emergency information to the server. The input is the emergency information, and the output is the data sent to the server.

[1017] Step 13:

[1018] Provision of emergency response information by server

[1019] The server searches for emergency services and hospital information nearest to the user's current location based on the received emergency information. The search results are sent to the device. The input is the emergency information, and the output is emergency response information.

[1020] Step 14:

[1021] Emergency information display and translation support on devices

[1022] The terminal displays emergency response information received from the server to the user and provides translation function as needed, allowing the user to communicate with local medical staff. The input is emergency response information, and the output is display of emergency information and translation support.

[1023] (Application example 1)

[1024] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1025] Support systems to help inbound tourists enjoy their stay in Japan include travel plan creation, navigation, cashless payment, and emergency assistance, but these services are often provided separately. Such systems require travelers to use multiple applications and devices, which is inconvenient. These functions are rarely integrated into autonomous transportation, particularly in these modes of transportation, so there is a need for centralized, integrated support.

[1026] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1027] In this invention, the server includes means for a user to input personal information, means for saving the input personal information in a database, means for proposing individual services based on the saved data, means for transmitting the proposals to a user terminal, means for managing the user's itinerary in the autonomous transportation means, means for providing navigation based on the user's location information, means for sending payment information to the server and executing payment, and means for providing information on nearby medical institutions in the event of an emergency. This allows travelers to use a single system to receive a variety of integrated services.

[1028] "User" refers to travelers and end users who use this system.

[1029] "Personal Information" refers to information about you, including your name, nationality, emergency contact details, email address, etc.

[1030] "Database" refers to an information system for storing and managing users' personal information, travel information, payment information, etc.

[1031] A "server" is a computer that manages the entire system and processes and stores data.

[1032] "Autonomous mobility" refers to devices that move without user intervention, such as self-driving vehicles and robots.

[1033] "Location information" refers to a user's current location and past location data obtained using technologies such as GPS.

[1034] "Navigation" refers to the function of calculating the optimal route from the current location to the destination and guiding the user along the route.

[1035] "Payment" refers to the process by which a user pays for goods or services cashlessly.

[1036] An "emergency" refers to a situation in which the user encounters a sudden situation such as an accident or sudden illness.

[1037] "Medical institution information" refers to information about medical facilities such as hospitals and clinics.

[1038] This invention is an integrated system that allows inbound tourists to enjoy their stay in Japan smoothly. The user, terminal, and server work together to provide personal information management, travel plan creation, real-time navigation, cashless payment, and emergency support.

[1039] System Program

[1040] The system includes the following elements:

[1041] 1. Registration of personal information

[1042] The user uses a terminal (for example, a touch panel inside the car) to input personal information (name, nationality, emergency contact, email address).

[1043] The terminal sends the entered personal information to the server, which stores it in a database.

[1044] Once saved, the server will send a confirmation email to the user's email address, asking them to activate their account.

[1045] 2. Create a travel plan

[1046] The user uses the terminal to input the desired travel dates, destinations, budget, and activities of interest.

[1047] The device sends the entered information to a server, which then compares the travel information with a database to select appropriate tourist spots and activities.

[1048] The server automatically generates a travel plan and sends it to the terminal, which displays it to the user for confirmation and adjustment.

[1049] 3. Real-time navigation

[1050] When a user travels to a tourist spot or other destination, the user inputs the destination into the terminal.

[1051] The device uses GPS to obtain its current location and transmits it to the server.

[1052] The server calculates the optimal route based on the current location and destination and generates detailed route guidance.

[1053] The device displays the route guidance received from the server on a map and provides real-time navigation.

[1054] 4. Payment and Cashless Payment

[1055] The user activates the payment function using the terminal and presents the QR code to the store.

[1056] The terminal sends the payment information to the server, which then executes the payment. Once the process is complete, the server sends the result to the terminal, which then displays a payment completion screen to the user.

[1057] 5. Emergency Support

[1058] When a user encounters an emergency, the user activates an emergency support function on the terminal and inputs the details of the emergency.

[1059] The device sends emergency information to the server, which then searches for information on emergency services and medical institutions closest to the current location and generates the data.

[1060] The device displays emergency information received from the server and provides translation functions as needed to help users communicate with local medical staff.

[1061] Specific examples

[1062] 1. Personal information registration:

[1063] Travelers enter personal information such as their name and nationality on a touch panel inside the vehicle and submit the information.

[1064] The server stores the data and sends a confirmation email.

[1065] 2. Create your travel plan:

[1066] Travelers enter their dates of stay and destination and choose from suggested attractions and activities.

[1067] The server automatically generates a travel plan and displays it on the in-car display.

[1068] 3. Real-time navigation:

[1069] The route from the current location to the next tourist spot is calculated, and the autonomous vehicle operates according to that route.

[1070] 4. Payment and Cashless Payment:

[1071] When making a purchase at a store, scan the QR code to complete the payment.

[1072] 5. Emergency Support:

[1073] When someone suddenly falls ill, the system searches for and navigates to the nearest hospital.

[1074] Prompt Sentence Examples

[1075] Design an autonomous mobility application with an integrated tourism support system. The app will have the following features:

[1076] 1. Registering user personal information and sending a confirmation email.

[1077] 2. Creating a travel plan and suggesting suitable tourist spots.

[1078] 3. Real-time navigation.

[1079] 4. Cashless payments within autonomous vehicles.

[1080] 5. Navigation to the nearest medical facility in case of an emergency.

[1081] Please also describe the data flow and the hardware and software used to realize these functions.

[1082] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1083] Step 1:

[1084] The user uses the touch panel in the car to enter personal information (name, nationality, emergency contact, email address), after which the device sends this information to the server and requests a confirmation email.

[1085] Input: Personal information entered by the user on the touch panel

[1086] Output: Personal information sent to the server, confirmation email request

[1087] Step 2:

[1088] The server stores the received personal information in a database, and once the storage is complete, the server sends a confirmation email to the user's email address, requesting that the account be activated.

[1089] Input: Personal information sent from the device, confirmation email request

[1090] Output: Personal information stored in database, confirmation email sent

[1091] Step 3:

[1092] The user uses the device to input their travel itinerary, destination, budget, and activities they are interested in. The device then sends this information to the server.

[1093] Input: Travel itinerary, destination, budget, and activities of interest entered by the user on the touch panel

[1094] Output: Trip information sent to the server

[1095] Step 4:

[1096] The server compares the received travel information with its database and selects appropriate tourist spots and activities, then automatically generates a travel plan and sends it to the device.

[1097] Input: Travel information

[1098] Output: Automatically generated itinerary, itinerary sent to the device

[1099] Step 5:

[1100] The user checks the travel plan and makes adjustments as necessary. The device sends the user's adjustment request to the server. The server then sends the final travel plan reflecting the adjustments to the device.

[1101] Input: User adjustment request

[1102] Output: Final itinerary with adjustments

[1103] Step 6:

[1104] When a user travels to a tourist spot or other destination, they input their destination into the device and activate the navigation function. The device then uses GPS to obtain their current location and sends it to the server.

[1105] Input: Destination or current location entered by the user on the touch panel

[1106] Output: Current location and destination sent to the server

[1107] Step 7:

[1108] The server calculates the optimal route based on the current location and destination, generates detailed route guidance including transportation information, and sends the generated route guidance to the device.

[1109] Input: Current location and destination

[1110] Output: Optimized travel route, directions sent to the device

[1111] Step 8:

[1112] The device displays the route guidance received from the server on a map and provides real-time navigation.

[1113] Input: Directions sent from the server

[1114] Output: Route directions displayed on a map

[1115] Step 9:

[1116] When using a store or service, the user activates the payment function on the device and presents the QR code to the store. The device then sends the payment information to the server and begins the payment process.

[1117] Input: Payment information entered by the user on the touch panel

[1118] Output: Payment information sent to the server

[1119] Step 10:

[1120] The server executes the payment based on the received payment information. Once the payment is complete, it sends the result to the terminal. The terminal then displays a payment completion screen to the user.

[1121] Input: Payment Information

[1122] Output: Payment result, payment result sent to the terminal

[1123] Step 11:

[1124] When a user encounters an emergency, the user activates the emergency support function using the terminal and inputs the details of the emergency, which then transmits the emergency information to the server.

[1125] Input: Emergency details entered by the user on the touch panel

[1126] Output: Urgent information sent to the server

[1127] Step 12:

[1128] The server searches for information on emergency services and medical institutions closest to the user's current location based on the emergency information, and sends it to the device. The device then displays the emergency information received from the server and provides translation functionality as needed.

[1129] Input: Emergency Information

[1130] Output: Information about nearby medical institutions, emergency information sent to the device

[1131] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1132] The present invention is an integrated system for helping inbound tourists have a comfortable stay in Japan, in which users, terminals, a server, and an emotion engine function in cooperation with each other. Specific embodiments for carrying out the present invention will be described below.

[1133] Account Registration

[1134] User:

[1135] A traveler launches the app and selects "Create a new account" on the initial screen. The user follows the app's instructions and enters personal information such as name, nationality, emergency contact, and email address.

[1136] Device:

[1137] The terminal transmits the personal information input by the user to the server.

[1138] server:

[1139] The server stores the received personal information in a database and sends the user a confirmation email, which the user must click to activate their account.

[1140] Creating a travel plan

[1141] User:

[1142] Users use the app to input their desired travel dates, destinations, budget, and activities they are interested in.

[1143] Emotion Engine:

[1144] The emotion engine recognizes emotions from the user's facial expressions, voice, text input, etc. and determines their emotional state.

[1145] Device:

[1146] The terminal transmits the input travel information and emotion information to the server.

[1147] server:

[1148] The server compares the travel information and emotion information with a database to select appropriate tourist spots and activities. The server adjusts the recommendations according to the emotional state recognized by the emotion engine. It then automatically generates a travel plan and presents it to the user.

[1149] Device:

[1150] The device displays the proposed travel plan to the user, prompting them to confirm and adjust it. The user then confirms the plan and finalizes it.

[1151] Real-time navigation and information

[1152] User:

[1153] When a user travels to a tourist spot or destination, they enter the destination into the app to activate the navigation function.

[1154] Device:

[1155] The device uses GPS to obtain its current location and transmits it to the server.

[1156] server:

[1157] The server calculates the optimal route based on the current location and destination, and generates detailed route guidance including transportation information.

[1158] Device:

[1159] The device displays the route guidance received from the server on a map and provides real-time navigation.

[1160] Payment and cashless payment

[1161] User:

[1162] When using a store or service, users activate the app's payment function and present the QR code to the store.

[1163] Device:

[1164] The terminal sends payment information to the server to initiate the payment process.

[1165] server:

[1166] The server checks the payment information, executes the payment, and sends the result to the terminal once the payment is complete.

[1167] Device:

[1168] The terminal will display a payment completion screen to the user and provide receipts and payment history as needed.

[1169] Emergency support

[1170] User:

[1171] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency.

[1172] Device:

[1173] The terminal transmits emergency information to the server.

[1174] server:

[1175] The server searches for information on emergency services and hospitals closest to the user's current location and generates data to provide to the user.

[1176] Device:

[1177] The device displays emergency information received from the server and provides translation functions as needed to help users communicate with local medical staff.

[1178] Recognizing and responding to user emotions

[1179] User:

[1180] A user's emotional state may change in various situations during travel, such as while moving around a tourist spot or participating in an activity.

[1181] Emotion Engine:

[1182] The emotion engine analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine the user's emotional state. If an emotion is recognized, the corresponding data is sent to the server.

[1183] server:

[1184] The server uses the emotion data to understand the user's current emotional state. For example, if the user is feeling stressed, it can suggest relaxing activities.

[1185] Device:

[1186] The terminal displays the suggestions from the server to the user and provides assistance in real time.

[1187] Specific examples

[1188] Example 1: Travel planning and emotional response

[1189] User:

[1190] Traveler A starts the app and plans a trip. The trip is for 7 days, with the main tourist destinations being Tokyo and Kyoto. The budget is 100,000 yen.

[1191] Traveler A selects cultural experiences and nature walks as activities that interest him.

[1192] Emotion Engine:

[1193] The emotion engine recognizes from Traveler A's voice and text input that he wants to relax.

[1194] Device:

[1195] The user's input information and emotion information are sent to the server.

[1196] server:

[1197] Based on traveler A's information and emotional data, the system suggests relaxing tourist spots such as Sensoji Temple, Fushimi Inari Taisha Shrine, and the bamboo forests of Kamakura. It also automatically generates a travel plan that includes a relaxing hot spring experience.

[1198] Device:

[1199] The travel plan is displayed to Traveler A, prompting them to confirm and adjust. Traveler A confirms and confirms the plan.

[1200] Example 2: Real-time emotion recognition and response

[1201] User:

[1202] Tourist B gets lost in the crowds of Tokyo and becomes stressed.

[1203] Emotion Engine:

[1204] The emotion engine recognizes stress from traveler B's facial expressions and tone of voice.

[1205] Device:

[1206] Emotion data is sent to the server.

[1207] server:

[1208] Based on current location data and emotional data, the app suggests nearby cafes and rest spots where Traveler B can relax.

[1209] Device:

[1210] The server displays the suggestions to Traveler B along with a map, and provides navigation to the rest spot.

[1211] In this way, by combining an emotion engine, the present invention is a system that can provide detailed responses according to the emotional state of inbound tourists, making their stay more comfortable.

[1212] The processing flow will be explained below.

[1213] Specific processing flow of the invention that combines the emotion engine

[1214] Account Registration

[1215] Step 1:

[1216] The user launches the app and selects "Create a new account" on the initial screen.

[1217] Step 2:

[1218] The user enters their name, nationality, emergency contact, and email address.

[1219] Step 3:

[1220] The terminal transmits the input information to the server.

[1221] Step 4:

[1222] The server stores the received information in a database.

[1223] Step 5:

[1224] The server sends a confirmation email to the user's email address.

[1225] Step 6:

[1226] The user activates their account by clicking the link in the confirmation email.

[1227] Step 7:

[1228] The device will display a screen indicating that the account registration was successful.

[1229] Travel planning and emotion recognition

[1230] Step 1:

[1231] The user selects "Create a trip" in the app.

[1232] Step 2:

[1233] The user enters desired travel dates, destinations, budget, and activities of interest.

[1234] Step 3:

[1235] The emotion engine recognizes emotions from the user's facial expressions, voice, and text input, and determines their emotional state.

[1236] Step 4:

[1237] The terminal transmits the input travel information and emotion information to the server.

[1238] Step 5:

[1239] The server compares the travel information and emotion information with a database and selects appropriate tourist spots and activities.

[1240] Step 6:

[1241] The server adjusts the content of the suggestions depending on the emotional state recognized by the emotion engine.

[1242] Step 7:

[1243] The server automatically generates an adjusted travel plan and sends it to the terminal.

[1244] Step 8:

[1245] The device displays the travel plan to the user and prompts them to confirm and adjust it.

[1246] Step 9:

[1247] The user reviews the plan and finalizes it.

[1248] Real-time navigation

[1249] Step 1:

[1250] The user enters a destination in the app and launches the navigation function.

[1251] Step 2:

[1252] The device uses GPS to obtain its current location and sends it to the server.

[1253] Step 3:

[1254] The server calculates the optimal route based on the current location and destination.

[1255] Step 4:

[1256] The server sends route guidance information to the terminal.

[1257] Step 5:

[1258] The device displays route guidance on a map and provides real-time navigation to the user.

[1259] Payment and cashless payment

[1260] Step 1:

[1261] When the user visits a store or uses a service, they activate the app's payment function.

[1262] Step 2:

[1263] The user confirms the payment amount and scans the QR code.

[1264] Step 3:

[1265] The terminal sends the payment information to the server.

[1266] Step 4:

[1267] The server verifies the payment information and executes the payment.

[1268] Step 5:

[1269] The server sends the payment result to the terminal.

[1270] Step 6:

[1271] The terminal displays a payment completion screen to the user.

[1272] Emergency support

[1273] Step 1:

[1274] The user launches the emergency support function and enters the details of the emergency.

[1275] Step 2:

[1276] The terminal transmits emergency information to the server.

[1277] Step 3:

[1278] The server searches for information about emergency services and hospitals closest to your location.

[1279] Step 4:

[1280] The server sends the search results to the terminal.

[1281] Step 5:

[1282] The terminal displays the emergency information to the user.

[1283] Step 6:

[1284] The device provides translation functionality as needed to help users communicate with local medical staff.

[1285] Recognizing and responding to user emotions

[1286] Step 1:

[1287] The emotion engine analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine their emotional state.

[1288] Step 2:

[1289] The emotion engine sends the recognized emotion data to the server.

[1290] Step 3:

[1291] The server understands the user's current emotional state based on the emotional data.

[1292] Step 4:

[1293] The server selects appropriate tourist spots and activities based on the user's emotional state and adjusts the suggestions accordingly.

[1294] Step 5:

[1295] The server sends the adjusted proposal to the terminal.

[1296] Step 6:

[1297] The device displays the suggestions to the user and provides assistance in real time.

[1298] Specific examples

[1299] Example 1: Travel planning and emotional response

[1300] Step 1:

[1301] The user launches the app and plans a trip.

[1302] Step 2:

[1303] The user inputs their travel dates, destinations, budget, and activities of interest (e.g., cultural experiences or nature walks).

[1304] Step 3:

[1305] The emotion engine recognizes the user's desire to relax from their voice and text input.

[1306] Step 4:

[1307] The terminal transmits travel information and emotion information to the server.

[1308] Step 5:

[1309] The server selects relaxing tourist spots and activities (e.g., Sensoji Temple, Fushimi Inari Taisha Shrine, hot spring experience) based on travel information and emotional data.

[1310] Step 6:

[1311] The server sends the automatically generated travel plan to the terminal.

[1312] Step 7:

[1313] The device displays the travel plan to the user and prompts them to confirm and adjust it.

[1314] Step 8:

[1315] The user reviews the plan and finalizes it.

[1316] Example 2: Real-time emotion recognition and response

[1317] Step 1:

[1318] Users get lost in the crowds of Tokyo and feel stressed.

[1319] Step 2:

[1320] The emotion engine recognizes stress from the user's facial expressions and tone of voice.

[1321] Step 3:

[1322] The device transmits the emotion data to the server.

[1323] Step 4:

[1324] The server selects a nearby cafe or rest spot where the user can relax based on the user's current location data and emotional data.

[1325] Step 5:

[1326] The server transmits the selection result to the terminal.

[1327] Step 6:

[1328] The device provides navigation to rest spots.

[1329] In this way, by combining an emotion engine, the present invention is a system that can provide detailed responses according to the emotional state of inbound tourists, making their stay more comfortable.

[1330] Example 2

[1331] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1332] Conventional travel support systems have difficulty understanding travelers' individual needs and emotional state in real time and making optimal recommendations based on that information. Furthermore, there is uncertainty about how to respond to troubles and emergencies during travel, and the infrastructure for travelers to enjoy their stay with peace of mind is insufficient. Furthermore, there is a lack of appropriate support and guidance tailored to the location and situation, and there is a need to improve travelers' satisfaction.

[1333] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1334] In this invention, the server includes: means for a user to input personal information; means for transmitting the input personal information to the server; means for storing the transmitted personal information in a database; means for sending a confirmation email based on the stored data; means for activating the account via a link from the confirmation email; means for a user to input travel information; means for analyzing the user's emotions using an emotion recognition engine; means for transmitting the input travel information and the analyzed emotion information to the server; means for suggesting appropriate tourist spots and activities based on the transmitted travel information and emotion information; means for displaying the proposed travel plan to the user; means for the user to use a GPS function to calculate and display an optimal travel route; and means for displaying the travel route proposal on the user terminal. This enables optimal suggestions and support to be provided in real time according to the individual needs and emotional state of the traveler.

[1335] "Personal Information" is data that can individually identify you, including information such as your name, nationality, emergency contact details, and email address.

[1336] A "server" is a computer system that receives, processes, and stores data sent by users.

[1337] A "database" is a system for structuring, storing, and managing users' personal information, travel information, and emotional information.

[1338] A "confirmation email" is an email sent to the email address entered by the user to prompt them to activate their account.

[1339] "Travel information" is data including information entered by the user such as travel itinerary, destination, budget, and activities of interest.

[1340] An "emotion recognition engine" is a system that analyzes a user's facial expressions, tone of voice, text input, etc. to determine the user's emotional state.

[1341] "Suggestion" refers to the automatic selection of appropriate tourist spots and activities based on the travel information and emotional information entered by the user.

[1342] A "travel plan" is a specific travel itinerary automatically generated by the server according to the user's emotional state and requirements.

[1343] A "user terminal" is a device used by a user to input information and display suggested travel plans and navigation information.

[1344] The "GPS function" is a function that uses the Global Positioning System to obtain the user's current location.

[1345] "Navigation" is a function that presents the optimal route based on the current location and destination, and provides real-time guidance.

[1346] The present invention is an integrated system for helping inbound tourists have a comfortable stay in Japan, in which users, terminals, a server, and an emotion engine function in cooperation with each other. Specific embodiments for carrying out the present invention will be described below.

[1347] Account Registration

[1348] User:

[1349] A traveler launches the app and selects "Create a new account" on the initial screen. The user then enters personal information such as name, nationality, emergency contact, and email address.

[1350] Device:

[1351] The personal information entered by the user is sent to the server.

[1352] server:

[1353] The server stores the received personal information in a database and sends a confirmation email to the user, who clicks on a link in the email to activate their account.

[1354] Creating a travel plan

[1355] User:

[1356] Users use the app to input their desired travel dates, destinations, budget, and activities they are interested in.

[1357] Emotion Engine:

[1358] The emotion engine recognizes emotions from the user's facial expressions, voice, text input, etc. and determines their emotional state.

[1359] Device:

[1360] The entered travel information and analyzed emotion information are sent to the server.

[1361] server:

[1362] The system compares travel information and emotion information with a database to select appropriate tourist spots and activities. Based on the analysis results of the emotion engine, the system adjusts the suggestions and automatically generates a travel plan to present to the user.

[1363] Device:

[1364] The travel plan is displayed to the user and they are prompted to confirm and adjust it. The user confirms the plan and finalizes it.

[1365] Real-time navigation and information

[1366] User:

[1367] When traveling to a tourist spot or destination, enter the destination into the app and activate the navigation function.

[1368] Device:

[1369] Use the GPS function to obtain the current location and send it to the server.

[1370] server:

[1371] Calculates the optimal travel route based on your current location and destination, and generates detailed route guidance including transportation information.

[1372] Device:

[1373] Route guidance is displayed on a map and navigation is provided in real time.

[1374] Payment and cashless payment

[1375] User:

[1376] When using a store or service, activate the app's payment function and present the QR code to the store.

[1377] Device:

[1378] Sends payment information to the server to begin the checkout process.

[1379] server:

[1380] Confirm the payment information and execute the payment. Once the payment is complete, the result will be sent to the terminal.

[1381] Device:

[1382] The payment completion screen is displayed to the user, and a receipt and payment history are provided as needed.

[1383] Emergency support

[1384] User:

[1385] When you encounter an emergency, activate the app's emergency support function and enter the details of the emergency.

[1386] Device:

[1387] Send emergency information to the server.

[1388] server:

[1389] Search for information on emergency services and hospitals closest to your current location and generate data to provide to the user.

[1390] Device:

[1391] It displays emergency information and provides translation functionality where necessary to help users communicate with local medical staff.

[1392] Recognizing and responding to user emotions

[1393] User:

[1394] Your emotional state will change depending on the situation you are in while traveling.

[1395] Emotion Engine:

[1396] It analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine their emotional state. If an emotion is recognized, the data is sent to the server.

[1397] server:

[1398] Based on emotional data, the system grasps the user's current emotional state and suggests activities that will help them relax.

[1399] Device:

[1400] The server displays suggestions to the user, providing real-time assistance.

[1401] Specific examples

[1402] Example 1: Travel planning and emotional response

[1403] User:

[1404] Traveler A starts the app and sets a seven-day travel itinerary. The main tourist destinations are Tokyo and Kyoto, and the budget is 100,000 yen. The traveler also enters that they are interested in cultural experiences and nature walks.

[1405] Emotion Engine:

[1406] Recognize Traveler A's desire to relax from his voice and text input.

[1407] Device:

[1408] The input information and emotion information are sent to the server.

[1409] server:

[1410] Based on traveler A's information and emotional data, the system suggests relaxing tourist spots such as Sensoji Temple, Fushimi Inari Taisha Shrine, and the bamboo forests of Kamakura. It also automatically generates a travel plan that includes a relaxing hot spring experience.

[1411] Device:

[1412] The travel plan is displayed and traveler A is prompted to confirm and adjust the plan. Traveler A confirms and confirms the plan.

[1413] Example 2: Real-time emotion recognition and response

[1414] User:

[1415] Tourist B gets lost in the crowds of Tokyo and becomes stressed.

[1416] Emotion Engine:

[1417] Recognize stress from traveler B's facial expressions and tone of voice.

[1418] Device:

[1419] Emotion data is sent to the server.

[1420] server:

[1421] Based on current location data and emotional data, the app suggests nearby cafes and rest spots where Traveler B can relax.

[1422] Device:

[1423] The suggestions are displayed along with a map, and navigation to rest spots is provided.

[1424] Generative AI model prompt

[1425] Example prompt 1:

[1426] "I want to create a 7-day travel plan for Traveler A. The main tourist destinations are Tokyo and Kyoto, and the budget is 100,000 yen. The traveler is interested in cultural experiences and nature walks, and wants to relax. Please automatically generate a travel plan based on these conditions."

[1427] Example prompt 2:

[1428] "Traveller B is lost in the crowds of Tokyo and feeling stressed. Based on their current location and emotional data, please suggest a nearby cafe or rest spot where they can relax."

[1429] In this way, by combining an emotion engine, the system of this invention aims to provide detailed responses in real time according to the emotional state of inbound tourists, making their stay more comfortable. In addition, by utilizing a generative AI model, it is possible to make quick and accurate suggestions based on user needs.

[1430] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1431] Account Registration

[1432] Step 1: Launch the app and select Create a new account

[1433] User:

[1434] Launch the app and select "Create a new account" on the initial screen. Input: None. Output: New account creation screen.

[1435] Step 2: Enter your personal information

[1436] User:

[1437] Enter personal information such as name, nationality, emergency contact, email address, etc. Input: User information. Output: Entered personal information.

[1438] Step 3: Submit your personal information

[1439] Device:

[1440] The entered personal information is sent to the server. Input: Personal information provided by the user. Output: Data sent to the server.

[1441] Step 4: Save to database and send confirmation email

[1442] server:

[1443] The received personal information is stored in a database and a confirmation email is sent to the user. Input: Personal information from the terminal. Output: Data stored in the database and the confirmation email sent.

[1444] Step 5: Activate your account

[1445] User:

[1446] Click the link in the confirmation email to activate your account. Input: Link in confirmation email. Output: Account activation confirmation message.

[1447] Creating a travel plan

[1448] Step 1: Enter your travel information

[1449] User:

[1450] Enter your desired travel dates, destinations, budget, and activities you're interested in into the app. Input: Trip information. Output: Entered trip information.

[1451] Step 2: Analyze emotional state

[1452] Emotion Engine:

[1453] Recognizes emotions from facial expressions, voice, and text input, and determines emotional state. Input: User's facial expressions, voice, and text data. Output: Analyzed emotional information.

[1454] Step 3: Sending information to the server

[1455] Device:

[1456] Send travel information and emotion information to the server. Input: Travel information, emotion information. Output: Data sent to the server.

[1457] Step 4: Collate the data and generate a plan

[1458] server:

[1459] The database is collated based on travel information and emotional information to select appropriate tourist spots and activities. The suggestions are adjusted based on the analysis results of the emotional engine, and a travel plan is automatically generated. Input: Travel information, emotional information. Output: Automatically generated travel plan.

[1460] Step 5: Present and confirm your plan

[1461] Device:

[1462] Display the itinerary to the user and prompt for confirmation and adjustment. Input: Generated itinerary. Output: Itinerary presented to the user.

[1463] Real-time navigation and information

[1464] Step 1: Enter your destination

[1465] User:

[1466] Enter your destination into the app and launch the navigation function. Input: Destination information. Output: Navigation start screen.

[1467] Step 2: Get your current location

[1468] Device:

[1469] Uses GPS to obtain current location and sends it to the server. Input: Current location information from GPS. Output: Data sent to the server.

[1470] Step 3: Calculating the travel route

[1471] server:

[1472] Calculates the optimal travel route based on the current location and destination, and generates detailed route guidance including transportation information. Input: Current location, destination information. Output: Calculated travel route.

[1473] Step 4: Provide directions

[1474] Device:

[1475] It displays route directions on a map and provides real-time navigation. Input: Calculated travel route. Output: Route directions displayed on a map.

[1476] Payment and cashless payment

[1477] Step 1: Launching the payment function

[1478] User:

[1479] When using a store or service, activate the app's payment function and present the QR code to the store. Input: None. Output: QR code display screen.

[1480] Step 2: Submit your payment information

[1481] Device:

[1482] Sends payment information to the server and starts the payment process. Input: QR code information. Output: Data sent to the server.

[1483] Step 3: Confirm payment

[1484] server:

[1485] Confirm the payment information and execute the payment. Once the payment is completed, the result is sent to the terminal. Input: Payment information. Output: Payment result data.

[1486] Step 4: Payment completion screen displayed

[1487] Device:

[1488] Displays a payment completion screen to the user, and optionally provides a receipt and payment history. Input: Payment result. Output: Payment completion screen and receipt.

[1489] Emergency support

[1490] Step 1: Activating the Emergency Support Feature

[1491] User:

[1492] When you encounter an emergency, activate the app's emergency support function and enter the details of the emergency. Input: Details of the emergency. Output: Emergency support screen.

[1493] Step 2: Send emergency information

[1494] Device:

[1495] Send emergency information to the server. Input: Emergency information. Output: Data sent to the server.

[1496] Step 3: Find and provide emergency services

[1497] server:

[1498] Search for information on emergency services and hospitals closest to the current location and generate data to provide to the user. Input: Emergency information, current location information. Output: Emergency service information.

[1499] Step 4: Displaying emergency information and translation assistance

[1500] Device:

[1501] It displays emergency information and provides translation functionality where necessary to help users communicate with local medical staff. Input: Emergency service information. Output: Displayed emergency information, translated results.

[1502] Recognizing and responding to user emotions

[1503] Step 1: Change your emotions

[1504] User:

[1505] The emotional state changes depending on the situation during the trip. Input: The situation of the user during the trip. Output: The changed emotional state.

[1506] Step 2: Determine your emotional state

[1507] Emotion Engine:

[1508] The system analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine their emotional state. If an emotion is recognized, the data is sent to the server. Input: Facial expressions, tone of voice, text data. Output: Analyzed emotional state, data sent to the server.

[1509] Step 3: Emotional information analysis

[1510] server:

[1511] Based on emotion data, the system understands the user's current emotional state and suggests activities that will help them relax. Input: Emotion data. Output: Activity suggestions.

[1512] Step 4: View the proposal

[1513] Device:

[1514] The server displays suggestions to the user, providing real-time assistance. Input: Suggested activity. Output: Displayed suggestions.

[1515] (Application example 2)

[1516] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1517] Today's inbound travelers are increasingly seeking a comfortable travel experience through the use of hardware and software. However, very few systems offer personalized recommendations based on their emotional state. This can lead to stress during travel and difficulty in selecting the right product. Furthermore, the lack of real-time navigation and personalized information can lead to travelers becoming lost or not receiving appropriate assistance.

[1518] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input personal information, means for saving the input personal information in a database, means for proposing individual services based on the saved data, means for recognizing the user's emotional state using an emotion engine, means for suggesting items and rest spots that will help the user relax based on the recognized emotional state, and means for transmitting the suggestions to the user terminal. This makes it possible to provide individual services that are in line with the traveler's emotional state, thereby achieving a comfortable travel experience.

[1519] "Personal information" is information that can identify a specific person, such as a user's name, nationality, contact information, and email address.

[1520] A "database" is a system that systematically stores information such as input personal information and proposal details, allowing for rapid search and use.

[1521] An "emotion engine" is software or an algorithm that analyzes and determines a user's emotional state based on facial expressions, tone of voice, text input, etc.

[1522] "Suggestions" are presentations of individual services such as tourist spots, activities, relaxation items, and rest spots that are generated based on the user's input information and emotional state.

[1523] A "user terminal" is a portable computing device such as a smartphone or tablet used by a traveler.

[1524] The "GPS function" is a global positioning system that allows precise acquisition of location information for any point on Earth.

[1525] A "generative AI model" is an artificial intelligence model that learns from large datasets, extracts certain patterns and features, and generates new data.

[1526] A "prompt sentence" is a sentence that serves as data or instructions to be input into a generative AI model.

[1527]

[1528] The present invention is an integrated system that uses an emotion engine to improve the travel experience of inbound tourists, and functions in cooperation with users, terminals, and servers. Specific embodiments for implementing the present invention will be described below.

[1529] Account Registration

[1530] A user launches the app and enters their personal information. The device sends the information to the server, which stores it in a database and sends a confirmation email. The user clicks on a link in the email to activate their account.

[1531] Creating a travel plan

[1532] Users input their travel itinerary, destinations, budget, and activities of interest into the app. The emotion engine analyzes the user's emotional state from their facial expressions, tone of voice, and text input, and sends this information to the server. The server then selects matching tourist spots and activities based on the database, automatically generates a travel plan, and sends it to the user's device.

[1533] Suggestions and Emotion Recognition

[1534] The emotion engine assesses the user's real-time emotional state and suggests items and resting spots to help them relax if they are feeling stressed. The server sends these suggestions to the user's device, where the user can confirm the information presented.

[1535] Navigating travel routes

[1536] When a user travels to a tourist spot or other destination, the current location is acquired using the GPS function and sent to the server. The server then calculates the optimal route and displays detailed route guidance on the user's device. Route guidance can also be generated in real time using a generative AI model by entering a prompt sentence.

[1537] Payment and cashless payment

[1538] When a user finds a product in the virtual shopping area, they launch the app's payment function and scan the QR code. The device sends the payment information to the server, which then executes the payment process and sends the results back to the device. A payment completion screen is displayed on the user's device, along with a receipt and payment history.

[1539] Emergency Support

[1540] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency. The device then sends the emergency information to a server, which then provides information on the nearest emergency services and medical institutions. The app also provides a translation function to help users communicate with local medical staff.

[1541] Hardware or software used

[1542] The following hardware and software are used to implement this system.

[1543] Smartphone (iOS or Android)

[1544] Sentiment engine: Google Cloud's Natural Language API and Microsoft Azure's Text Analytics API

[1545] GPS module: Built-in GPS in smartphone

[1546] Database: Firebase Firestore or Amazon RDS

[1547] Payment system: Stripe or PayPal API

[1548] Specific examples

[1549] Tourist C is exploring a virtual shopping area in Japan. The emotion engine recognizes from his tone of voice that he is tired, and the app suggests items and resting spots to help him relax. The user can use the app's help function to find more information about these items and how to spend their time.

[1550] Prompt Sentence Examples

[1551] A user launches the "Virtual Tourist Shopping Assistant" application and walks around a virtual shopping area. The system detects fatigue from the user's facial expressions and tone of voice. Please suggest items and resting spots that will help them relax.

[1552] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1553] Step 1:

[1554] A user launches the app and enters their personal information (such as name, nationality, contact details, and email address). The device sends the information to the server, which stores it in a database and sends a confirmation email. The user clicks on a link in the confirmation email to activate their account.

[1555] Step 2:

[1556] The user enters travel information such as travel itinerary, destinations, budget, and activities of interest into the app. The device sends this travel information to the server. The emotion engine analyzes the user's emotional state from their facial expressions, tone of voice, and text input, and sends the results to the server. The server compares the received data with a database to select the most suitable tourist spots and activities.

[1557] Step 3:

[1558] The server automatically generates a travel plan based on the selected tourist spots and activities and sends it to the user's device. The user can then review the proposed plan on their device and make adjustments as necessary. The final travel plan is then finalized.

[1559] Step 4:

[1560] When a user travels to a tourist spot or other destination, they activate the navigation function using the app. The device's GPS function acquires the user's current location and sends that information to the server. The server calculates the optimal route based on the current location and destination, and generates the route in real time based on a generation AI model. The generated route guidance is sent to the user's device and displayed on a map.

[1561] Step 5:

[1562] When a user finds a product in the virtual shopping area, they launch the app's payment function and scan the QR code. The terminal sends the payment information to the server, which then executes the payment process and sends the result to the terminal. The user sees a payment completion screen, and is provided with a receipt and payment history.

[1563] Step 6:

[1564] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency. The device then sends this emergency information to a server. The server then searches for information on the nearest emergency service or medical institution based on the user's current location and sends that information to the user's device. The translation function also helps users communicate with local medical staff.

[1565] Step 7:

[1566] If a user experiences emotional changes while traveling or participating in an activity, the emotion engine analyzes them in real time and sends the results to the server. Based on the user's emotional state, the server creates suggestions for relaxing items and rest spots using a generative AI model and sends the suggestions to the user's device. The user can then review the suggestions and receive any support they need.

[1567] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1568] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1569] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1570] [Third embodiment]

[1571] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1572] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1573] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1574] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1575] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1576] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1577] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1578] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1579] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1580] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1581] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1582] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1583] The present invention is an integrated system for enabling inbound tourists to enjoy their stay in Japan smoothly, and functions in cooperation with users, terminals, and servers. Specific embodiments for carrying out the present invention will be described below.

[1584] Account Registration

[1585] User:

[1586] The first step a traveler takes after launching the app is to create an account. Users follow the app's instructions and enter personal information such as their name, nationality, emergency contact details, and email address.

[1587] Device:

[1588] The terminal transmits the personal information input by the user to the server.

[1589] server:

[1590] The server stores the received personal information in a database, and once the storage is complete, a confirmation email is sent to the user's email address, requesting that they activate their account.

[1591] Creating a travel plan

[1592] User:

[1593] Users use the app to input their desired travel dates, destinations, budget, and activities they are interested in.

[1594] Device:

[1595] The terminal transmits the input information to the server.

[1596] server:

[1597] The server compares the travel information with the database, selects appropriate tourist spots and activities, and automatically generates and proposes a travel plan to the user.

[1598] Device:

[1599] The device displays the proposed travel plan to the user, who can review the plan, make adjustments as needed, and finalize the plan.

[1600] Real-time navigation and information

[1601] User:

[1602] When a user travels to a tourist spot or destination, they enter the destination into the app to activate the navigation function.

[1603] Device:

[1604] The device uses GPS to obtain its current location and transmits it to the server.

[1605] server:

[1606] The server calculates the optimal route based on the current location and destination, and generates detailed route guidance including transportation information.

[1607] Device:

[1608] The device displays the route guidance received from the server on a map and provides real-time navigation.

[1609] Payment and cashless payment

[1610] User:

[1611] When using a store or service, users activate the app's payment function and present the QR code to the store.

[1612] Device:

[1613] The terminal sends payment information to the server to initiate the payment process.

[1614] server:

[1615] The server checks the payment information, executes the payment, and sends the result to the terminal once the payment is complete.

[1616] Device:

[1617] The terminal will display a payment completion screen to the user and provide receipts and payment history as needed.

[1618] Emergency support

[1619] User:

[1620] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency.

[1621] Device:

[1622] The terminal transmits emergency information to the server.

[1623] server:

[1624] The server searches for emergency services and hospital information closest to the current location and generates data to provide to the user.

[1625] Device:

[1626] The device displays emergency information received from the server and provides translation functions as needed to help users communicate with local medical staff.

[1627] Specific examples

[1628] Example 1: Creating a travel plan

[1629] User:

[1630] Traveler A starts the app and plans a trip. The trip is for 7 days, with the main tourist destinations being Tokyo and Kyoto. The budget is 100,000 yen.

[1631] Device:

[1632] The user's input information is sent to the server.

[1633] server:

[1634] Based on the information provided by Traveler A, the plan suggests tourist spots such as Sensoji Temple, Skytree, Fushimi Inari Taisha Shrine, and Kiyomizu-dera Temple. In addition, activities such as a maiko experience and sushi-making experience are also included in the plan.

[1635] Device:

[1636] The created travel plan is displayed to Traveler A, prompting them to confirm and make adjustments.

[1637] Example 2: Real-time navigation

[1638] User:

[1639] Traveler B needs navigation to travel from Tokyo to Osaka. He enters "Tokyo to Osaka" into the app.

[1640] Device:

[1641] Use the GPS function to obtain the current location (Tokyo Station) and send it to the server.

[1642] server:

[1643] The system calculates the optimal travel route from Tokyo Station to Osaka Station, including using the Shinkansen, and generates detailed navigation information.

[1644] Device:

[1645] It displays the travel route received from the server and provides real-time information on the next train and transfers.

[1646] In this way, the present invention provides a comprehensive support system to help inbound travelers enjoy their stay in Japan to the fullest.

[1647] The processing flow will be explained below.

[1648] Account Registration

[1649] Step 1:

[1650] The user launches the app and selects "Create a new account" on the initial screen.

[1651] Step 2:

[1652] The user enters their name, nationality, emergency contact, and email address.

[1653] Step 3:

[1654] The terminal transmits the input information to the server.

[1655] Step 4:

[1656] The server stores the received information in a database.

[1657] Step 5:

[1658] The server sends a confirmation email to the user's email address.

[1659] Step 6:

[1660] The user activates their account by clicking the link in the confirmation email.

[1661] Step 7:

[1662] The device will display a screen indicating that the account registration was successful.

[1663] Creating a travel plan

[1664] Step 1:

[1665] The user selects "Create a trip" in the app.

[1666] Step 2:

[1667] The user enters desired travel dates, destinations, budget, and activities of interest.

[1668] Step 3:

[1669] The terminal transmits the input travel information to the server.

[1670] Step 4:

[1671] The server compares the travel information with a database and selects appropriate attractions and activities.

[1672] Step 5:

[1673] The server automatically generates a travel plan based on the selected tourist spots and activities.

[1674] Step 6:

[1675] The server transmits the generated travel plan to the terminal.

[1676] Step 7:

[1677] The device displays the travel plan to the user and prompts them to confirm and adjust it.

[1678] Step 8:

[1679] The user reviews and confirms the plan.

[1680] Real-time navigation

[1681] Step 1:

[1682] The user enters a destination in the app and launches the navigation function.

[1683] Step 2:

[1684] The device uses GPS to obtain its current location and sends it to the server.

[1685] Step 3:

[1686] The server calculates the optimal route based on the current location and destination.

[1687] Step 4:

[1688] The server sends route guidance information to the terminal.

[1689] Step 5:

[1690] The device displays route guidance on a map and provides real-time navigation to the user.

[1691] Payment and cashless payment

[1692] Step 1:

[1693] When the user visits a store or uses a service, they activate the app's payment function.

[1694] Step 2:

[1695] The user confirms the payment amount and scans the QR code.

[1696] Step 3:

[1697] The terminal sends the payment information to the server.

[1698] Step 4:

[1699] The server verifies the payment information and executes the payment.

[1700] Step 5:

[1701] The server sends the payment result to the terminal.

[1702] Step 6:

[1703] The terminal displays a payment completion screen to the user.

[1704] Emergency support

[1705] Step 1:

[1706] The user launches the emergency support function and enters the details of the emergency.

[1707] Step 2:

[1708] The terminal transmits emergency information to the server.

[1709] Step 3:

[1710] The server searches for information about emergency services and hospitals closest to your location.

[1711] Step 4:

[1712] The server sends the search results to the terminal.

[1713] Step 5:

[1714] The terminal displays the emergency information to the user.

[1715] Step 6:

[1716] The device provides translation functionality as needed to help users communicate with local medical staff.

[1717] Example 1

[1718] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1719] Today's travelers are expected to plan their own trips, navigate their way around, make payments, and even respond to emergencies all in one place, but existing systems struggle to provide integrated support for all of these. It's particularly difficult to provide a single platform for planning sightseeing trips, real-time navigation, cashless payments, and emergency response. As a result, travelers are forced to use multiple applications and services, which creates hassle and confusion that can degrade the quality of their trip.

[1720] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1721] In this invention, the server includes a means for a user to input personal information, a means for saving the input personal information in a database, and a means for sending a confirmation email of account activation to the user's email address after saving is complete, thereby enabling a user to manage all travel-related information on one platform and enjoy personalized services.

[1722] "User" refers to a traveler who uses this system.

[1723] "Terminal" refers to an electronic device such as a smartphone or tablet used by a user.

[1724] "Server" refers to a computer system that processes, stores, and manages data for the entire system.

[1725] "Personal information" refers to information entered by the user, such as name, nationality, emergency contact information, and email address.

[1726] "Database" refers to a system for storing and managing information such as personal information and travel plans.

[1727] "Verification Email" means the email sent to a User's email address to activate their account.

[1728] "Travel itinerary" refers to the planned duration of the trip entered by the user.

[1729] "Destinations" refer to places or cities that a user plans to visit during their trip.

[1730] "Budget" refers to the amount of money a user plans to spend on a trip.

[1731] "Activities" refer to experiential tourist events and activities suggested based on the user's interests.

[1732] "GPS Function" refers to the location technology used by a Device to obtain its current location.

[1733] "Travel route" refers to the route from the current location to the destination.

[1734] "Payment functionality" refers to the functionality of the app that allows users to make cashless payments.

[1735] "Payment System" means an online payment platform for processing payment information.

[1736] "Emergency Support Feature" refers to the feature of the App that enables a User to receive necessary support in an emergency.

[1737] "Emergency information" refers to the user's report of an emergency situation.

[1738] "Translation Feature" refers to the automatic translation feature used by Users to communicate in their local language.

[1739] This invention is an integrated system that enables inbound tourists to enjoy their stay in Japan smoothly. The user, terminal, and server work together, allowing the user to manage their entire trip on a single platform.

[1740] First, the user launches the app using a device such as a smartphone or tablet. When the app is launched for the first time, the user enters their personal information, which is then sent from the device to the server. The server stores the received information in a database and sends a confirmation email to the user's email address to activate their account. The account is activated when the user approves this confirmation email.

[1741] Next, the user enters their travel itinerary, destinations, budget, and activities of interest into the app. The device sends this travel information to the server, which compares it with a database and uses a generative AI model (e.g., GPT-4) to suggest suitable attractions and activities. The proposed plan is automatically generated, saved in JSON format, and then sent to the user's device. The device displays the received plan to the user and makes adjustments if necessary.

[1742] Furthermore, when a user travels to a tourist spot or other destination, the device's GPS function is used to send the user's current location to the server. The server then calculates the optimal route from the user's current location to the destination. Detailed navigation information is generated and sent to the device. The device then displays this information on a map and provides real-time navigation.

[1743] When a user visits a store or uses a service, they activate the app's payment function and present the QR code to the store. This payment information is sent from the device to the server, and the server executes the payment using a payment system (e.g., Stripe, PayPal). The payment completion result is sent to the terminal, which displays it to the user.

[1744] In an emergency, the user enters emergency information using the app's emergency support function. This information is sent from the device to a server, which searches for the most appropriate emergency service or hospital information. The search results are sent to the device and displayed there. Furthermore, a translation function is provided if necessary, allowing the user to communicate with local staff.

[1745] Specific examples

[1746] Example 1: Creating a travel plan

[1747] User A launches the app and creates a seven-day travel plan. The destinations entered are Tokyo and Kyoto, with a budget of 100,000 yen. The entered information is sent to the server, which then suggests tourist spots such as Sensoji Temple, Skytree, Fushimi Inari Taisha Shrine, and Kiyomizu-dera Temple. Activities such as a maiko experience and a sushi-making experience are also incorporated into the plan. The generated plan is displayed on the device to User A, who then reviews the contents and finalizes the plan.

[1748] Example 2: Real-time navigation

[1749] User B uses the app's navigation function when traveling from Tokyo to Osaka. The current location (Tokyo Station) is obtained using the GPS function and sent to the server. The server calculates the optimal route from Tokyo Station to Osaka Station, generates detailed navigation including Shinkansen information, and sends it to the device. The device displays this on a map and provides real-time information on the next train and transfers.

[1750] Prompt Sentence Examples

[1751] "Generate a 7-day travel plan with a budget of 100,000 yen, focusing on Tokyo and Kyoto."

[1752] As described above, the present invention allows users to manage all travel-related information in one system, providing a smooth travel experience.

[1753] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1754] Step 1:

[1755] User information entry and submission

[1756] A user launches the app and navigates to the account creation screen, where they enter personal information such as their name, nationality, emergency contact, and email address. After entering the information, the device encrypts it using the HTTPS protocol and sends it to the server. The input contains personal information, and the output is encrypted data.

[1757] Step 2:

[1758] Server saves information and sends confirmation email

[1759] The server stores the received personal information in a database (e.g., MySQL). If the storage is successful, the server sends a confirmation email of account activation to the user's email address via the SMTP server. The input is the received encrypted personal information, and the output is the storage result in the database and the confirmation email.

[1760] Step 3:

[1761] User's travel information input and submission

[1762] The user inputs their travel itinerary, destinations, budget, and activities of interest into the app. The device sends this information to the server. The input is information about the trip, and the output is data sent to the server.

[1763] Step 4:

[1764] Travel plan generation by server

[1765] The server compares the received travel information with a database (e.g., MongoDB), selects appropriate tourist spots and activities using a generative AI model (e.g., GPT-4), and automatically generates a travel plan. The generated plan is saved in JSON format and sent to the device. The input is travel information, and the output is the generated travel plan.

[1766] Step 5:

[1767] View and adjust travel plans on your device

[1768] The device displays the travel plan received from the server to the user. The user reviews the plan and makes adjustments as necessary. If adjustments are necessary, the device sends the information back to the server. The input is the generated travel plan, and the output is the adjusted plan approval.

[1769] Step 6:

[1770] Launching the navigation function

[1771] When a user travels to a tourist spot or other destination, they input their destination into the app and activate the navigation function. The device uses GPS to obtain current location information and sends it to the server. The input is destination information and current location information, and the output is the data sent to the server.

[1772] Step 7:

[1773] Route calculation by the server

[1774] The server calculates the optimal route based on the received current location and destination information using the Google Maps API and route calculation algorithms. Detailed route guidance is generated in JSON format and sent to the device. The input is the current location information and destination information, and the output is the route guidance.

[1775] Step 8:

[1776] Displaying travel route on device

[1777] The device displays the route guidance received from the server on a map and provides real-time navigation. The user moves around while checking the app screen. The input is the route guidance, and the output is the display of navigation information.

[1778] Step 9:

[1779] Activating payment functions and sending information

[1780] When using a store or service, the user activates the app's payment function, selects a payment method, and presents a QR code. The terminal then sends the payment information to the server. The input is the payment information, and the output is the data sent to the server.

[1781] Step 10:

[1782] Server-based payment execution

[1783] The server processes the received payment information via a payment system (e.g., Stripe, PayPal). After the payment is completed, the result is sent to the terminal. The input is the payment information, and the output is the payment result.

[1784] Step 11:

[1785] Terminal indicates payment is complete

[1786] The terminal receives the payment result from the server, displays it to the user, and provides receipts and payment history as needed. The input is the payment result, and the output is a payment completion screen.

[1787] Step 12:

[1788] Enter and submit emergency information

[1789] When a user encounters an emergency, they activate the app's emergency support function and input the details of the emergency. The device then sends the emergency information to the server. The input is the emergency information, and the output is the data sent to the server.

[1790] Step 13:

[1791] Provision of emergency response information by server

[1792] The server searches for emergency services and hospital information nearest to the user's current location based on the received emergency information. The search results are sent to the device. The input is the emergency information, and the output is emergency response information.

[1793] Step 14:

[1794] Emergency information display and translation support on devices

[1795] The terminal displays emergency response information received from the server to the user and provides translation function as needed, allowing the user to communicate with local medical staff. The input is emergency response information, and the output is display of emergency information and translation support.

[1796] (Application example 1)

[1797] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1798] Support systems to help inbound tourists enjoy their stay in Japan include travel plan creation, navigation, cashless payment, and emergency assistance, but these services are often provided separately. Such systems require travelers to use multiple applications and devices, which is inconvenient. These functions are rarely integrated into autonomous transportation, particularly in these modes of transportation, so there is a need for centralized, integrated support.

[1799] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1800] In this invention, the server includes means for a user to input personal information, means for saving the input personal information in a database, means for proposing individual services based on the saved data, means for transmitting the proposals to a user terminal, means for managing the user's itinerary in the autonomous transportation means, means for providing navigation based on the user's location information, means for sending payment information to the server and executing payment, and means for providing information on nearby medical institutions in the event of an emergency. This allows travelers to use a single system to receive a variety of integrated services.

[1801] "User" refers to travelers and end users who use this system.

[1802] "Personal Information" refers to information about you, including your name, nationality, emergency contact details, email address, etc.

[1803] "Database" refers to an information system for storing and managing users' personal information, travel information, payment information, etc.

[1804] A "server" is a computer that manages the entire system and processes and stores data.

[1805] "Autonomous mobility" refers to devices that move without user intervention, such as self-driving vehicles and robots.

[1806] "Location information" refers to a user's current location and past location data obtained using technologies such as GPS.

[1807] "Navigation" refers to the function of calculating the optimal route from the current location to the destination and guiding the user along the route.

[1808] "Payment" refers to the process by which a user pays for goods or services cashlessly.

[1809] An "emergency" refers to a situation in which the user encounters a sudden situation such as an accident or sudden illness.

[1810] "Medical institution information" refers to information about medical facilities such as hospitals and clinics.

[1811] This invention is an integrated system that allows inbound tourists to enjoy their stay in Japan smoothly. The user, terminal, and server work together to provide personal information management, travel plan creation, real-time navigation, cashless payment, and emergency support.

[1812] System Program

[1813] The system includes the following elements:

[1814] 1. Registration of personal information

[1815] The user uses a terminal (for example, a touch panel inside the car) to input personal information (name, nationality, emergency contact, email address).

[1816] The terminal sends the entered personal information to the server, which stores it in a database.

[1817] Once saved, the server will send a confirmation email to the user's email address, asking them to activate their account.

[1818] 2. Create a travel plan

[1819] The user uses the terminal to input the desired travel dates, destinations, budget, and activities of interest.

[1820] The device sends the entered information to a server, which then compares the travel information with a database to select appropriate tourist spots and activities.

[1821] The server automatically generates a travel plan and sends it to the terminal, which displays it to the user for confirmation and adjustment.

[1822] 3. Real-time navigation

[1823] When a user travels to a tourist spot or other destination, the user inputs the destination into the terminal.

[1824] The device uses GPS to obtain its current location and transmits it to the server.

[1825] The server calculates the optimal route based on the current location and destination and generates detailed route guidance.

[1826] The device displays the route guidance received from the server on a map and provides real-time navigation.

[1827] 4. Payment and Cashless Payment

[1828] The user activates the payment function using the terminal and presents the QR code to the store.

[1829] The terminal sends the payment information to the server, which then executes the payment. Once the process is complete, the server sends the result to the terminal, which then displays a payment completion screen to the user.

[1830] 5. Emergency Support

[1831] When a user encounters an emergency, the user activates an emergency support function on the terminal and inputs the details of the emergency.

[1832] The device sends emergency information to the server, which then searches for information on emergency services and medical institutions closest to the current location and generates the data.

[1833] The device displays emergency information received from the server and provides translation functions as needed to help users communicate with local medical staff.

[1834] Specific examples

[1835] 1. Personal information registration:

[1836] Travelers enter personal information such as their name and nationality on a touch panel inside the vehicle and submit the information.

[1837] The server stores the data and sends a confirmation email.

[1838] 2. Create your travel plan:

[1839] Travelers enter their dates of stay and destination and choose from suggested attractions and activities.

[1840] The server automatically generates a travel plan and displays it on the in-car display.

[1841] 3. Real-time navigation:

[1842] The route from the current location to the next tourist spot is calculated, and the autonomous vehicle operates according to that route.

[1843] 4. Payment and Cashless Payment:

[1844] When making a purchase at a store, scan the QR code to complete the payment.

[1845] 5. Emergency Support:

[1846] When someone suddenly falls ill, the system searches for and navigates to the nearest hospital.

[1847] Prompt Sentence Examples

[1848] Design an autonomous mobility application with an integrated tourism support system. The app will have the following features:

[1849] 1. Registering user personal information and sending a confirmation email.

[1850] 2. Creating a travel plan and suggesting suitable tourist spots.

[1851] 3. Real-time navigation.

[1852] 4. Cashless payments within autonomous vehicles.

[1853] 5. Navigation to the nearest medical facility in case of an emergency.

[1854] Please also describe the data flow and the hardware and software used to realize these functions.

[1855] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1856] Step 1:

[1857] The user uses the touch panel in the car to enter personal information (name, nationality, emergency contact, email address), after which the device sends this information to the server and requests a confirmation email.

[1858] Input: Personal information entered by the user on the touch panel

[1859] Output: Personal information sent to the server, confirmation email request

[1860] Step 2:

[1861] The server stores the received personal information in a database, and once the storage is complete, the server sends a confirmation email to the user's email address, requesting that the account be activated.

[1862] Input: Personal information sent from the device, confirmation email request

[1863] Output: Personal information stored in database, confirmation email sent

[1864] Step 3:

[1865] The user uses the device to input their travel itinerary, destination, budget, and activities they are interested in. The device then sends this information to the server.

[1866] Input: Travel itinerary, destination, budget, and activities of interest entered by the user on the touch panel

[1867] Output: Trip information sent to the server

[1868] Step 4:

[1869] The server compares the received travel information with its database and selects appropriate tourist spots and activities, then automatically generates a travel plan and sends it to the device.

[1870] Input: Travel information

[1871] Output: Automatically generated itinerary, itinerary sent to the device

[1872] Step 5:

[1873] The user checks the travel plan and makes adjustments as necessary. The device sends the user's adjustment request to the server. The server then sends the final travel plan reflecting the adjustments to the device.

[1874] Input: User adjustment request

[1875] Output: Final itinerary with adjustments

[1876] Step 6:

[1877] When a user travels to a tourist spot or other destination, they input their destination into the device and activate the navigation function. The device then uses GPS to obtain their current location and sends it to the server.

[1878] Input: Destination or current location entered by the user on the touch panel

[1879] Output: Current location and destination sent to the server

[1880] Step 7:

[1881] The server calculates the optimal route based on the current location and destination, generates detailed route guidance including transportation information, and sends the generated route guidance to the device.

[1882] Input: Current location and destination

[1883] Output: Optimized travel route, directions sent to the device

[1884] Step 8:

[1885] The device displays the route guidance received from the server on a map and provides real-time navigation.

[1886] Input: Directions sent from the server

[1887] Output: Route directions displayed on a map

[1888] Step 9:

[1889] When using a store or service, the user activates the payment function on the device and presents the QR code to the store. The device then sends the payment information to the server and begins the payment process.

[1890] Input: Payment information entered by the user on the touch panel

[1891] Output: Payment information sent to the server

[1892] Step 10:

[1893] The server executes the payment based on the received payment information. Once the payment is complete, it sends the result to the terminal. The terminal then displays a payment completion screen to the user.

[1894] Input: Payment Information

[1895] Output: Payment result, payment result sent to the terminal

[1896] Step 11:

[1897] When a user encounters an emergency, the user activates the emergency support function using the terminal and inputs the details of the emergency, which then transmits the emergency information to the server.

[1898] Input: Emergency details entered by the user on the touch panel

[1899] Output: Urgent information sent to the server

[1900] Step 12:

[1901] The server searches for information on emergency services and medical institutions closest to the user's current location based on the emergency information, and sends it to the device. The device then displays the emergency information received from the server and provides translation functionality as needed.

[1902] Input: Emergency Information

[1903] Output: Information about nearby medical institutions, emergency information sent to the device

[1904] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1905] The present invention is an integrated system for helping inbound tourists have a comfortable stay in Japan, in which users, terminals, a server, and an emotion engine function in cooperation with each other. Specific embodiments for carrying out the present invention will be described below.

[1906] Account Registration

[1907] User:

[1908] A traveler launches the app and selects "Create a new account" on the initial screen. The user follows the app's instructions and enters personal information such as name, nationality, emergency contact, and email address.

[1909] Device:

[1910] The terminal transmits the personal information input by the user to the server.

[1911] server:

[1912] The server stores the received personal information in a database and sends the user a confirmation email, which the user must click to activate their account.

[1913] Creating a travel plan

[1914] User:

[1915] Users use the app to input their desired travel dates, destinations, budget, and activities they are interested in.

[1916] Emotion Engine:

[1917] The emotion engine recognizes emotions from the user's facial expressions, voice, text input, etc. and determines their emotional state.

[1918] Device:

[1919] The terminal transmits the input travel information and emotion information to the server.

[1920] server:

[1921] The server compares the travel information and emotion information with a database to select appropriate tourist spots and activities. The server adjusts the recommendations according to the emotional state recognized by the emotion engine. It then automatically generates a travel plan and presents it to the user.

[1922] Device:

[1923] The device displays the proposed travel plan to the user, prompting them to confirm and adjust it. The user then confirms the plan and finalizes it.

[1924] Real-time navigation and information

[1925] User:

[1926] When a user travels to a tourist spot or destination, they enter the destination into the app to activate the navigation function.

[1927] Device:

[1928] The device uses GPS to obtain its current location and transmits it to the server.

[1929] server:

[1930] The server calculates the optimal route based on the current location and destination, and generates detailed route guidance including transportation information.

[1931] Device:

[1932] The device displays the route guidance received from the server on a map and provides real-time navigation.

[1933] Payment and cashless payment

[1934] User:

[1935] When using a store or service, users activate the app's payment function and present the QR code to the store.

[1936] Device:

[1937] The terminal sends payment information to the server to initiate the payment process.

[1938] server:

[1939] The server checks the payment information, executes the payment, and sends the result to the terminal once the payment is complete.

[1940] Device:

[1941] The terminal will display a payment completion screen to the user and provide receipts and payment history as needed.

[1942] Emergency support

[1943] User:

[1944] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency.

[1945] Device:

[1946] The terminal transmits emergency information to the server.

[1947] server:

[1948] The server searches for information on emergency services and hospitals closest to the user's current location and generates data to provide to the user.

[1949] Device:

[1950] The device displays emergency information received from the server and provides translation functions as needed to help users communicate with local medical staff.

[1951] Recognizing and responding to user emotions

[1952] User:

[1953] A user's emotional state may change in various situations during travel, such as while moving around a tourist spot or participating in an activity.

[1954] Emotion Engine:

[1955] The emotion engine analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine the user's emotional state. If an emotion is recognized, the corresponding data is sent to the server.

[1956] server:

[1957] The server uses the emotion data to understand the user's current emotional state. For example, if the user is feeling stressed, it can suggest relaxing activities.

[1958] Device:

[1959] The terminal displays the suggestions from the server to the user and provides assistance in real time.

[1960] Specific examples

[1961] Example 1: Travel planning and emotional response

[1962] User:

[1963] Traveler A starts the app and plans a trip. The trip is for 7 days, with the main tourist destinations being Tokyo and Kyoto. The budget is 100,000 yen.

[1964] Traveler A selects cultural experiences and nature walks as activities that interest him.

[1965] Emotion Engine:

[1966] The emotion engine recognizes from Traveler A's voice and text input that he wants to relax.

[1967] Device:

[1968] The user's input information and emotion information are sent to the server.

[1969] server:

[1970] Based on traveler A's information and emotional data, the system suggests relaxing tourist spots such as Sensoji Temple, Fushimi Inari Taisha Shrine, and the bamboo forests of Kamakura. It also automatically generates a travel plan that includes a relaxing hot spring experience.

[1971] Device:

[1972] The travel plan is displayed to Traveler A, prompting them to confirm and adjust. Traveler A confirms and confirms the plan.

[1973] Example 2: Real-time emotion recognition and response

[1974] User:

[1975] Tourist B gets lost in the crowds of Tokyo and becomes stressed.

[1976] Emotion Engine:

[1977] The emotion engine recognizes stress from traveler B's facial expressions and tone of voice.

[1978] Device:

[1979] Emotion data is sent to the server.

[1980] server:

[1981] Based on current location data and emotional data, the app suggests nearby cafes and rest spots where Traveler B can relax.

[1982] Device:

[1983] The server displays the suggestions to Traveler B along with a map, and provides navigation to the rest spot.

[1984] In this way, by combining an emotion engine, the present invention is a system that can provide detailed responses according to the emotional state of inbound tourists, making their stay more comfortable.

[1985] The processing flow will be explained below.

[1986] Specific processing flow of the invention that combines the emotion engine

[1987] Account Registration

[1988] Step 1:

[1989] The user launches the app and selects "Create a new account" on the initial screen.

[1990] Step 2:

[1991] The user enters their name, nationality, emergency contact, and email address.

[1992] Step 3:

[1993] The terminal transmits the input information to the server.

[1994] Step 4:

[1995] The server stores the received information in a database.

[1996] Step 5:

[1997] The server sends a confirmation email to the user's email address.

[1998] Step 6:

[1999] The user activates their account by clicking the link in the confirmation email.

[2000] Step 7:

[2001] The device will display a screen indicating that the account registration was successful.

[2002] Travel planning and emotion recognition

[2003] Step 1:

[2004] The user selects "Create a trip" in the app.

[2005] Step 2:

[2006] The user enters desired travel dates, destinations, budget, and activities of interest.

[2007] Step 3:

[2008] The emotion engine recognizes emotions from the user's facial expressions, voice, and text input, and determines their emotional state.

[2009] Step 4:

[2010] The terminal transmits the input travel information and emotion information to the server.

[2011] Step 5:

[2012] The server compares the travel information and emotion information with a database and selects appropriate tourist spots and activities.

[2013] Step 6:

[2014] The server adjusts the content of the suggestions depending on the emotional state recognized by the emotion engine.

[2015] Step 7:

[2016] The server automatically generates an adjusted travel plan and sends it to the terminal.

[2017] Step 8:

[2018] The device displays the travel plan to the user and prompts them to confirm and adjust it.

[2019] Step 9:

[2020] The user reviews the plan and finalizes it.

[2021] Real-time navigation

[2022] Step 1:

[2023] The user enters a destination in the app and launches the navigation function.

[2024] Step 2:

[2025] The device uses GPS to obtain its current location and sends it to the server.

[2026] Step 3:

[2027] The server calculates the optimal route based on the current location and destination.

[2028] Step 4:

[2029] The server sends route guidance information to the terminal.

[2030] Step 5:

[2031] The device displays route guidance on a map and provides real-time navigation to the user.

[2032] Payment and cashless payment

[2033] Step 1:

[2034] When the user visits a store or uses a service, they activate the app's payment function.

[2035] Step 2:

[2036] The user confirms the payment amount and scans the QR code.

[2037] Step 3:

[2038] The terminal sends the payment information to the server.

[2039] Step 4:

[2040] The server verifies the payment information and executes the payment.

[2041] Step 5:

[2042] The server sends the payment result to the terminal.

[2043] Step 6:

[2044] The terminal displays a payment completion screen to the user.

[2045] Emergency support

[2046] Step 1:

[2047] The user launches the emergency support function and enters the details of the emergency.

[2048] Step 2:

[2049] The terminal transmits emergency information to the server.

[2050] Step 3:

[2051] The server searches for information about emergency services and hospitals closest to your location.

[2052] Step 4:

[2053] The server sends the search results to the terminal.

[2054] Step 5:

[2055] The terminal displays the emergency information to the user.

[2056] Step 6:

[2057] The device provides translation functionality as needed to help users communicate with local medical staff.

[2058] Recognizing and responding to user emotions

[2059] Step 1:

[2060] The emotion engine analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine their emotional state.

[2061] Step 2:

[2062] The emotion engine sends the recognized emotion data to the server.

[2063] Step 3:

[2064] The server understands the user's current emotional state based on the emotional data.

[2065] Step 4:

[2066] The server selects appropriate tourist spots and activities based on the user's emotional state and adjusts the suggestions accordingly.

[2067] Step 5:

[2068] The server sends the adjusted proposal to the terminal.

[2069] Step 6:

[2070] The device displays the suggestions to the user and provides assistance in real time.

[2071] Specific examples

[2072] Example 1: Travel planning and emotional response

[2073] Step 1:

[2074] The user launches the app and plans a trip.

[2075] Step 2:

[2076] The user inputs their travel dates, destinations, budget, and activities of interest (e.g., cultural experiences or nature walks).

[2077] Step 3:

[2078] The emotion engine recognizes the user's desire to relax from their voice and text input.

[2079] Step 4:

[2080] The terminal transmits travel information and emotion information to the server.

[2081] Step 5:

[2082] The server selects relaxing tourist spots and activities (e.g., Sensoji Temple, Fushimi Inari Taisha Shrine, hot spring experience) based on travel information and emotional data.

[2083] Step 6:

[2084] The server sends the automatically generated travel plan to the terminal.

[2085] Step 7:

[2086] The device displays the travel plan to the user and prompts them to confirm and adjust it.

[2087] Step 8:

[2088] The user reviews the plan and finalizes it.

[2089] Example 2: Real-time emotion recognition and response

[2090] Step 1:

[2091] Users get lost in the crowds of Tokyo and feel stressed.

[2092] Step 2:

[2093] The emotion engine recognizes stress from the user's facial expressions and tone of voice.

[2094] Step 3:

[2095] The device transmits the emotion data to the server.

[2096] Step 4:

[2097] The server selects a nearby cafe or rest spot where the user can relax based on the user's current location data and emotional data.

[2098] Step 5:

[2099] The server transmits the selection result to the terminal.

[2100] Step 6:

[2101] The device provides navigation to rest spots.

[2102] In this way, by combining an emotion engine, the present invention is a system that can provide detailed responses according to the emotional state of inbound tourists, making their stay more comfortable.

[2103] Example 2

[2104] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2105] Conventional travel support systems have difficulty understanding travelers' individual needs and emotional state in real time and making optimal recommendations based on that information. Furthermore, there is uncertainty about how to respond to troubles and emergencies during travel, and the infrastructure for travelers to enjoy their stay with peace of mind is insufficient. Furthermore, there is a lack of appropriate support and guidance tailored to the location and situation, and there is a need to improve travelers' satisfaction.

[2106] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2107] In this invention, the server includes: means for a user to input personal information; means for transmitting the input personal information to the server; means for storing the transmitted personal information in a database; means for sending a confirmation email based on the stored data; means for activating the account via a link from the confirmation email; means for a user to input travel information; means for analyzing the user's emotions using an emotion recognition engine; means for transmitting the input travel information and the analyzed emotion information to the server; means for suggesting appropriate tourist spots and activities based on the transmitted travel information and emotion information; means for displaying the proposed travel plan to the user; means for the user to use a GPS function to calculate and display an optimal travel route; and means for displaying the travel route proposal on the user terminal. This enables optimal suggestions and support to be provided in real time according to the individual needs and emotional state of the traveler.

[2108] "Personal Information" is data that can individually identify you, including information such as your name, nationality, emergency contact details, and email address.

[2109] A "server" is a computer system that receives, processes, and stores data sent by users.

[2110] A "database" is a system for structuring, storing, and managing users' personal information, travel information, and emotional information.

[2111] A "confirmation email" is an email sent to the email address entered by the user to prompt them to activate their account.

[2112] "Travel information" is data including information entered by the user such as travel itinerary, destination, budget, and activities of interest.

[2113] An "emotion recognition engine" is a system that analyzes a user's facial expressions, tone of voice, text input, etc. to determine the user's emotional state.

[2114] "Suggestion" refers to the automatic selection of appropriate tourist spots and activities based on the travel information and emotional information entered by the user.

[2115] A "travel plan" is a specific travel itinerary automatically generated by the server according to the user's emotional state and requirements.

[2116] A "user terminal" is a device used by a user to input information and display suggested travel plans and navigation information.

[2117] The "GPS function" is a function that uses the Global Positioning System to obtain the user's current location.

[2118] "Navigation" is a function that presents the optimal route based on the current location and destination, and provides real-time guidance.

[2119] The present invention is an integrated system for helping inbound tourists have a comfortable stay in Japan, in which users, terminals, a server, and an emotion engine function in cooperation with each other. Specific embodiments for carrying out the present invention will be described below.

[2120] Account Registration

[2121] User:

[2122] A traveler launches the app and selects "Create a new account" on the initial screen. The user then enters personal information such as name, nationality, emergency contact, and email address.

[2123] Device:

[2124] The personal information entered by the user is sent to the server.

[2125] server:

[2126] The server stores the received personal information in a database and sends a confirmation email to the user, who clicks on a link in the email to activate their account.

[2127] Creating a travel plan

[2128] User:

[2129] Users use the app to input their desired travel dates, destinations, budget, and activities they are interested in.

[2130] Emotion Engine:

[2131] The emotion engine recognizes emotions from the user's facial expressions, voice, text input, etc. and determines their emotional state.

[2132] Device:

[2133] The entered travel information and analyzed emotion information are sent to the server.

[2134] server:

[2135] The system compares travel information and emotion information with a database to select appropriate tourist spots and activities. Based on the analysis results of the emotion engine, the system adjusts the suggestions and automatically generates a travel plan to present to the user.

[2136] Device:

[2137] The travel plan is displayed to the user and they are prompted to confirm and adjust it. The user confirms the plan and finalizes it.

[2138] Real-time navigation and information

[2139] User:

[2140] When traveling to a tourist spot or destination, enter the destination into the app and activate the navigation function.

[2141] Device:

[2142] Use the GPS function to obtain the current location and send it to the server.

[2143] server:

[2144] Calculates the optimal travel route based on your current location and destination, and generates detailed route guidance including transportation information.

[2145] Device:

[2146] Route guidance is displayed on a map and navigation is provided in real time.

[2147] Payment and cashless payment

[2148] User:

[2149] When using a store or service, activate the app's payment function and present the QR code to the store.

[2150] Device:

[2151] Sends payment information to the server to begin the checkout process.

[2152] server:

[2153] Confirm the payment information and execute the payment. Once the payment is complete, the result will be sent to the terminal.

[2154] Device:

[2155] The payment completion screen is displayed to the user, and a receipt and payment history are provided as needed.

[2156] Emergency support

[2157] User:

[2158] When you encounter an emergency, activate the app's emergency support function and enter the details of the emergency.

[2159] Device:

[2160] Send emergency information to the server.

[2161] server:

[2162] Search for information on emergency services and hospitals closest to your current location and generate data to provide to the user.

[2163] Device:

[2164] It displays emergency information and provides translation functionality where necessary to help users communicate with local medical staff.

[2165] Recognizing and responding to user emotions

[2166] User:

[2167] Your emotional state will change depending on the situation you are in while traveling.

[2168] Emotion Engine:

[2169] It analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine their emotional state. If an emotion is recognized, the data is sent to the server.

[2170] server:

[2171] Based on emotional data, the system grasps the user's current emotional state and suggests activities that will help them relax.

[2172] Device:

[2173] The server displays suggestions to the user, providing real-time assistance.

[2174] Specific examples

[2175] Example 1: Travel planning and emotional response

[2176] User:

[2177] Traveler A starts the app and sets a seven-day travel itinerary. The main tourist destinations are Tokyo and Kyoto, and the budget is 100,000 yen. The traveler also enters that they are interested in cultural experiences and nature walks.

[2178] Emotion Engine:

[2179] Recognize Traveler A's desire to relax from his voice and text input.

[2180] Device:

[2181] The input information and emotion information are sent to the server.

[2182] server:

[2183] Based on traveler A's information and emotional data, the system suggests relaxing tourist spots such as Sensoji Temple, Fushimi Inari Taisha Shrine, and the bamboo forests of Kamakura. It also automatically generates a travel plan that includes a relaxing hot spring experience.

[2184] Device:

[2185] The travel plan is displayed and traveler A is prompted to confirm and adjust the plan. Traveler A confirms and confirms the plan.

[2186] Example 2: Real-time emotion recognition and response

[2187] User:

[2188] Tourist B gets lost in the crowds of Tokyo and becomes stressed.

[2189] Emotion Engine:

[2190] Recognize stress from traveler B's facial expressions and tone of voice.

[2191] Device:

[2192] Emotion data is sent to the server.

[2193] server:

[2194] Based on current location data and emotional data, the app suggests nearby cafes and rest spots where Traveler B can relax.

[2195] Device:

[2196] The suggestions are displayed along with a map, and navigation to rest spots is provided.

[2197] Generative AI model prompt

[2198] Example prompt 1:

[2199] "I want to create a 7-day travel plan for Traveler A. The main tourist destinations are Tokyo and Kyoto, and the budget is 100,000 yen. The traveler is interested in cultural experiences and nature walks, and wants to relax. Please automatically generate a travel plan based on these conditions."

[2200] Example prompt 2:

[2201] "Traveller B is lost in the crowds of Tokyo and feeling stressed. Based on their current location and emotional data, please suggest a nearby cafe or rest spot where they can relax."

[2202] In this way, by combining an emotion engine, the system of this invention aims to provide detailed responses in real time according to the emotional state of inbound tourists, making their stay more comfortable. In addition, by utilizing a generative AI model, it is possible to make quick and accurate suggestions based on user needs.

[2203] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2204] Account Registration

[2205] Step 1: Launch the app and select Create a new account

[2206] User:

[2207] Launch the app and select "Create a new account" on the initial screen. Input: None. Output: New account creation screen.

[2208] Step 2: Enter your personal information

[2209] User:

[2210] Enter personal information such as name, nationality, emergency contact, email address, etc. Input: User information. Output: Entered personal information.

[2211] Step 3: Submit your personal information

[2212] Device:

[2213] The entered personal information is sent to the server. Input: Personal information provided by the user. Output: Data sent to the server.

[2214] Step 4: Save to database and send confirmation email

[2215] server:

[2216] The received personal information is stored in a database and a confirmation email is sent to the user. Input: Personal information from the terminal. Output: Data stored in the database and the confirmation email sent.

[2217] Step 5: Activate your account

[2218] User:

[2219] Click the link in the confirmation email to activate your account. Input: Link in confirmation email. Output: Account activation confirmation message.

[2220] Creating a travel plan

[2221] Step 1: Enter your travel information

[2222] User:

[2223] Enter your desired travel dates, destinations, budget, and activities you're interested in into the app. Input: Trip information. Output: Entered trip information.

[2224] Step 2: Analyze emotional state

[2225] Emotion Engine:

[2226] Recognizes emotions from facial expressions, voice, and text input, and determines emotional state. Input: User's facial expressions, voice, and text data. Output: Analyzed emotional information.

[2227] Step 3: Sending information to the server

[2228] Device:

[2229] Send travel information and emotion information to the server. Input: Travel information, emotion information. Output: Data sent to the server.

[2230] Step 4: Collate the data and generate a plan

[2231] server:

[2232] The database is collated based on travel information and emotional information to select appropriate tourist spots and activities. The suggestions are adjusted based on the analysis results of the emotional engine, and a travel plan is automatically generated. Input: Travel information, emotional information. Output: Automatically generated travel plan.

[2233] Step 5: Present and confirm your plan

[2234] Device:

[2235] Display the itinerary to the user and prompt for confirmation and adjustment. Input: Generated itinerary. Output: Itinerary presented to the user.

[2236] Real-time navigation and information

[2237] Step 1: Enter your destination

[2238] User:

[2239] Enter your destination into the app and launch the navigation function. Input: Destination information. Output: Navigation start screen.

[2240] Step 2: Get your current location

[2241] Device:

[2242] Uses GPS to obtain current location and sends it to the server. Input: Current location information from GPS. Output: Data sent to the server.

[2243] Step 3: Calculating the travel route

[2244] server:

[2245] Calculates the optimal travel route based on the current location and destination, and generates detailed route guidance including transportation information. Input: Current location, destination information. Output: Calculated travel route.

[2246] Step 4: Provide directions

[2247] Device:

[2248] It displays route directions on a map and provides real-time navigation. Input: Calculated travel route. Output: Route directions displayed on a map.

[2249] Payment and cashless payment

[2250] Step 1: Launching the payment function

[2251] User:

[2252] When using a store or service, activate the app's payment function and present the QR code to the store. Input: None. Output: QR code display screen.

[2253] Step 2: Submit your payment information

[2254] Device:

[2255] Sends payment information to the server and starts the payment process. Input: QR code information. Output: Data sent to the server.

[2256] Step 3: Confirm payment

[2257] server:

[2258] Confirm the payment information and execute the payment. Once the payment is completed, the result is sent to the terminal. Input: Payment information. Output: Payment result data.

[2259] Step 4: Payment completion screen displayed

[2260] Device:

[2261] Displays a payment completion screen to the user, and optionally provides a receipt and payment history. Input: Payment result. Output: Payment completion screen and receipt.

[2262] Emergency support

[2263] Step 1: Activating the Emergency Support Feature

[2264] User:

[2265] When you encounter an emergency, activate the app's emergency support function and enter the details of the emergency. Input: Details of the emergency. Output: Emergency support screen.

[2266] Step 2: Send emergency information

[2267] Device:

[2268] Send emergency information to the server. Input: Emergency information. Output: Data sent to the server.

[2269] Step 3: Find and provide emergency services

[2270] server:

[2271] Search for information on emergency services and hospitals closest to the current location and generate data to provide to the user. Input: Emergency information, current location information. Output: Emergency service information.

[2272] Step 4: Displaying emergency information and translation assistance

[2273] Device:

[2274] It displays emergency information and provides translation functionality where necessary to help users communicate with local medical staff. Input: Emergency service information. Output: Displayed emergency information, translated results.

[2275] Recognizing and responding to user emotions

[2276] Step 1: Change your emotions

[2277] User:

[2278] The emotional state changes depending on the situation during the trip. Input: The situation of the user during the trip. Output: The changed emotional state.

[2279] Step 2: Determine your emotional state

[2280] Emotion Engine:

[2281] The system analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine their emotional state. If an emotion is recognized, the data is sent to the server. Input: Facial expressions, tone of voice, text data. Output: Analyzed emotional state, data sent to the server.

[2282] Step 3: Emotional information analysis

[2283] server:

[2284] Based on emotion data, the system understands the user's current emotional state and suggests activities that will help them relax. Input: Emotion data. Output: Activity suggestions.

[2285] Step 4: View the proposal

[2286] Device:

[2287] The server displays suggestions to the user, providing real-time assistance. Input: Suggested activity. Output: Displayed suggestions.

[2288] (Application example 2)

[2289] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2290] Today's inbound travelers are increasingly seeking a comfortable travel experience through the use of hardware and software. However, very few systems offer personalized recommendations based on their emotional state. This can lead to stress during travel and difficulty in selecting the right product. Furthermore, the lack of real-time navigation and personalized information can lead to travelers becoming lost or not receiving appropriate assistance.

[2291] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input personal information, means for saving the input personal information in a database, means for proposing individual services based on the saved data, means for recognizing the user's emotional state using an emotion engine, means for suggesting items and rest spots that will help the user relax based on the recognized emotional state, and means for transmitting the suggestions to the user terminal. This makes it possible to provide individual services that are in line with the traveler's emotional state, thereby achieving a comfortable travel experience.

[2292] "Personal information" is information that can identify a specific person, such as a user's name, nationality, contact information, and email address.

[2293] A "database" is a system that systematically stores information such as input personal information and proposal details, allowing for rapid search and use.

[2294] An "emotion engine" is software or an algorithm that analyzes and determines a user's emotional state based on facial expressions, tone of voice, text input, etc.

[2295] "Suggestions" are presentations of individual services such as tourist spots, activities, relaxation items, and rest spots that are generated based on the user's input information and emotional state.

[2296] A "user terminal" is a portable computing device such as a smartphone or tablet used by a traveler.

[2297] The "GPS function" is a global positioning system that allows precise acquisition of location information for any point on Earth.

[2298] A "generative AI model" is an artificial intelligence model that learns from large datasets, extracts certain patterns and features, and generates new data.

[2299] A "prompt sentence" is a sentence that serves as data or instructions to be input into a generative AI model.

[2300]

[2301] The present invention is an integrated system that uses an emotion engine to improve the travel experience of inbound tourists, and functions in cooperation with users, terminals, and servers. Specific embodiments for implementing the present invention will be described below.

[2302] Account Registration

[2303] A user launches the app and enters their personal information. The device sends the information to the server, which stores it in a database and sends a confirmation email. The user clicks on a link in the email to activate their account.

[2304] Creating a travel plan

[2305] Users input their travel itinerary, destinations, budget, and activities of interest into the app. The emotion engine analyzes the user's emotional state from their facial expressions, tone of voice, and text input, and sends this information to the server. The server then selects matching tourist spots and activities based on the database, automatically generates a travel plan, and sends it to the user's device.

[2306] Suggestions and Emotion Recognition

[2307] The emotion engine assesses the user's real-time emotional state and suggests items and resting spots to help them relax if they are feeling stressed. The server sends these suggestions to the user's device, where the user can confirm the information presented.

[2308] Navigating travel routes

[2309] When a user travels to a tourist spot or other destination, the current location is acquired using the GPS function and sent to the server. The server then calculates the optimal route and displays detailed route guidance on the user's device. Route guidance can also be generated in real time using a generative AI model by entering a prompt sentence.

[2310] Payment and cashless payment

[2311] When a user finds a product in the virtual shopping area, they launch the app's payment function and scan the QR code. The device sends the payment information to the server, which then executes the payment process and sends the results back to the device. A payment completion screen is displayed on the user's device, along with a receipt and payment history.

[2312] Emergency Support

[2313] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency. The device then sends the emergency information to a server, which then provides information on the nearest emergency services and medical institutions. The app also provides a translation function to help users communicate with local medical staff.

[2314] Hardware or software used

[2315] The following hardware and software are used to implement this system.

[2316] Smartphone (iOS or Android)

[2317] Sentiment engine: Google Cloud's Natural Language API and Microsoft Azure's Text Analytics API

[2318] GPS module: Built-in GPS in smartphone

[2319] Database: Firebase Firestore or Amazon RDS

[2320] Payment system: Stripe or PayPal API

[2321] Specific examples

[2322] Tourist C is exploring a virtual shopping area in Japan. The emotion engine recognizes from his tone of voice that he is tired, and the app suggests items and resting spots to help him relax. The user can use the app's help function to find more information about these items and how to spend their time.

[2323] Prompt Sentence Examples

[2324] A user launches the "Virtual Tourist Shopping Assistant" application and walks around a virtual shopping area. The system detects fatigue from the user's facial expressions and tone of voice. Please suggest items and resting spots that will help them relax.

[2325] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2326] Step 1:

[2327] A user launches the app and enters their personal information (such as name, nationality, contact details, and email address). The device sends the information to the server, which stores it in a database and sends a confirmation email. The user clicks on a link in the confirmation email to activate their account.

[2328] Step 2:

[2329] The user enters travel information such as travel itinerary, destinations, budget, and activities of interest into the app. The device sends this travel information to the server. The emotion engine analyzes the user's emotional state from their facial expressions, tone of voice, and text input, and sends the results to the server. The server compares the received data with a database to select the most suitable tourist spots and activities.

[2330] Step 3:

[2331] The server automatically generates a travel plan based on the selected tourist spots and activities and sends it to the user's device. The user can then review the proposed plan on their device and make adjustments as necessary. The final travel plan is then finalized.

[2332] Step 4:

[2333] When a user travels to a tourist spot or other destination, they activate the navigation function using the app. The device's GPS function acquires the user's current location and sends that information to the server. The server calculates the optimal route based on the current location and destination, and generates the route in real time based on a generation AI model. The generated route guidance is sent to the user's device and displayed on a map.

[2334] Step 5:

[2335] When a user finds a product in the virtual shopping area, they launch the app's payment function and scan the QR code. The terminal sends the payment information to the server, which then executes the payment process and sends the result to the terminal. The user sees a payment completion screen, and is provided with a receipt and payment history.

[2336] Step 6:

[2337] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency. The device then sends this emergency information to a server. The server then searches for information on the nearest emergency service or medical institution based on the user's current location and sends that information to the user's device. The translation function also helps users communicate with local medical staff.

[2338] Step 7:

[2339] If a user experiences emotional changes while traveling or participating in an activity, the emotion engine analyzes them in real time and sends the results to the server. Based on the user's emotional state, the server creates suggestions for relaxing items and rest spots using a generative AI model and sends the suggestions to the user's device. The user can then review the suggestions and receive any support they need.

[2340] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2341] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2342] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2343] [Fourth embodiment]

[2344] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2345] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2346] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2347] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2348] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2349] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2350] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2351] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2352] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2353] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2354] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2355] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2356] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2357] The present invention is an integrated system for enabling inbound tourists to enjoy their stay in Japan smoothly, and functions in cooperation with users, terminals, and servers. Specific embodiments for carrying out the present invention will be described below.

[2358] Account Registration

[2359] User:

[2360] The first step a traveler takes after launching the app is to create an account. Users follow the app's instructions and enter personal information such as their name, nationality, emergency contact details, and email address.

[2361] Device:

[2362] The terminal transmits the personal information input by the user to the server.

[2363] server:

[2364] The server stores the received personal information in a database, and once the storage is complete, a confirmation email is sent to the user's email address, requesting that they activate their account.

[2365] Creating a travel plan

[2366] User:

[2367] Users use the app to input their desired travel dates, destinations, budget, and activities they are interested in.

[2368] Device:

[2369] The terminal transmits the input information to the server.

[2370] server:

[2371] The server compares the travel information with the database, selects appropriate tourist spots and activities, and automatically generates and proposes a travel plan to the user.

[2372] Device:

[2373] The device displays the proposed travel plan to the user, who can review the plan, make adjustments as needed, and finalize the plan.

[2374] Real-time navigation and information

[2375] User:

[2376] When a user travels to a tourist spot or destination, they enter the destination into the app to activate the navigation function.

[2377] Device:

[2378] The device uses GPS to obtain its current location and transmits it to the server.

[2379] server:

[2380] The server calculates the optimal route based on the current location and destination, and generates detailed route guidance including transportation information.

[2381] Device:

[2382] The device displays the route guidance received from the server on a map and provides real-time navigation.

[2383] Payment and cashless payment

[2384] User:

[2385] When using a store or service, users activate the app's payment function and present the QR code to the store.

[2386] Device:

[2387] The terminal sends payment information to the server to initiate the payment process.

[2388] server:

[2389] The server checks the payment information, executes the payment, and sends the result to the terminal once the payment is complete.

[2390] Device:

[2391] The terminal will display a payment completion screen to the user and provide receipts and payment history as needed.

[2392] Emergency support

[2393] User:

[2394] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency.

[2395] Device:

[2396] The terminal transmits emergency information to the server.

[2397] server:

[2398] The server searches for emergency services and hospital information closest to the current location and generates data to provide to the user.

[2399] Device:

[2400] The device displays emergency information received from the server and provides translation functions as needed to help users communicate with local medical staff.

[2401] Specific examples

[2402] Example 1: Creating a travel plan

[2403] User:

[2404] Traveler A starts the app and plans a trip. The trip is for 7 days, with the main tourist destinations being Tokyo and Kyoto. The budget is 100,000 yen.

[2405] Device:

[2406] The user's input information is sent to the server.

[2407] server:

[2408] Based on the information provided by Traveler A, the plan suggests tourist spots such as Sensoji Temple, Skytree, Fushimi Inari Taisha Shrine, and Kiyomizu-dera Temple. In addition, activities such as a maiko experience and sushi-making experience are also included in the plan.

[2409] Device:

[2410] The created travel plan is displayed to Traveler A, prompting them to confirm and make adjustments.

[2411] Example 2: Real-time navigation

[2412] User:

[2413] Traveler B needs navigation to travel from Tokyo to Osaka. He enters "Tokyo to Osaka" into the app.

[2414] Device:

[2415] Use the GPS function to obtain the current location (Tokyo Station) and send it to the server.

[2416] server:

[2417] The system calculates the optimal travel route from Tokyo Station to Osaka Station, including using the Shinkansen, and generates detailed navigation information.

[2418] Device:

[2419] It displays the travel route received from the server and provides real-time information on the next train and transfers.

[2420] In this way, the present invention provides a comprehensive support system to help inbound travelers enjoy their stay in Japan to the fullest.

[2421] The processing flow will be explained below.

[2422] Account Registration

[2423] Step 1:

[2424] The user launches the app and selects "Create a new account" on the initial screen.

[2425] Step 2:

[2426] The user enters their name, nationality, emergency contact, and email address.

[2427] Step 3:

[2428] The terminal transmits the input information to the server.

[2429] Step 4:

[2430] The server stores the received information in a database.

[2431] Step 5:

[2432] The server sends a confirmation email to the user's email address.

[2433] Step 6:

[2434] The user activates their account by clicking the link in the confirmation email.

[2435] Step 7:

[2436] The device will display a screen indicating that the account registration was successful.

[2437] Creating a travel plan

[2438] Step 1:

[2439] The user selects "Create a trip" in the app.

[2440] Step 2:

[2441] The user enters desired travel dates, destinations, budget, and activities of interest.

[2442] Step 3:

[2443] The terminal transmits the input travel information to the server.

[2444] Step 4:

[2445] The server compares the travel information with a database and selects appropriate attractions and activities.

[2446] Step 5:

[2447] The server automatically generates a travel plan based on the selected tourist spots and activities.

[2448] Step 6:

[2449] The server transmits the generated travel plan to the terminal.

[2450] Step 7:

[2451] The device displays the travel plan to the user and prompts them to confirm and adjust it.

[2452] Step 8:

[2453] The user reviews and confirms the plan.

[2454] Real-time navigation

[2455] Step 1:

[2456] The user enters a destination in the app and launches the navigation function.

[2457] Step 2:

[2458] The device uses GPS to obtain its current location and sends it to the server.

[2459] Step 3:

[2460] The server calculates the optimal route based on the current location and destination.

[2461] Step 4:

[2462] The server sends route guidance information to the terminal.

[2463] Step 5:

[2464] The device displays route guidance on a map and provides real-time navigation to the user.

[2465] Payment and cashless payment

[2466] Step 1:

[2467] When the user visits a store or uses a service, they activate the app's payment function.

[2468] Step 2:

[2469] The user confirms the payment amount and scans the QR code.

[2470] Step 3:

[2471] The terminal sends the payment information to the server.

[2472] Step 4:

[2473] The server verifies the payment information and executes the payment.

[2474] Step 5:

[2475] The server sends the payment result to the terminal.

[2476] Step 6:

[2477] The terminal displays a payment completion screen to the user.

[2478] Emergency support

[2479] Step 1:

[2480] The user launches the emergency support function and enters the details of the emergency.

[2481] Step 2:

[2482] The terminal transmits emergency information to the server.

[2483] Step 3:

[2484] The server searches for information about emergency services and hospitals closest to your location.

[2485] Step 4:

[2486] The server sends the search results to the terminal.

[2487] Step 5:

[2488] The terminal displays the emergency information to the user.

[2489] Step 6:

[2490] The device provides translation functionality as needed to help users communicate with local medical staff.

[2491] Example 1

[2492] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2493] Today's travelers are expected to plan their own trips, navigate their way around, make payments, and even respond to emergencies all in one place, but existing systems struggle to provide integrated support for all of these. It's particularly difficult to provide a single platform for planning sightseeing trips, real-time navigation, cashless payments, and emergency response. As a result, travelers are forced to use multiple applications and services, which creates hassle and confusion that can degrade the quality of their trip.

[2494] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2495] In this invention, the server includes a means for a user to input personal information, a means for saving the input personal information in a database, and a means for sending a confirmation email of account activation to the user's email address after saving is complete, thereby enabling a user to manage all travel-related information on one platform and enjoy personalized services.

[2496] "User" refers to a traveler who uses this system.

[2497] "Terminal" refers to an electronic device such as a smartphone or tablet used by a user.

[2498] "Server" refers to a computer system that processes, stores, and manages data for the entire system.

[2499] "Personal information" refers to information entered by the user, such as name, nationality, emergency contact information, and email address.

[2500] "Database" refers to a system for storing and managing information such as personal information and travel plans.

[2501] "Verification Email" means the email sent to a User's email address to activate their account.

[2502] "Travel itinerary" refers to the planned duration of the trip entered by the user.

[2503] "Destinations" refer to places or cities that a user plans to visit during their trip.

[2504] "Budget" refers to the amount of money a user plans to spend on a trip.

[2505] "Activities" refer to experiential tourist events and activities suggested based on the user's interests.

[2506] "GPS Function" refers to the location technology used by a Device to obtain its current location.

[2507] "Travel route" refers to the route from the current location to the destination.

[2508] "Payment functionality" refers to the functionality of the app that allows users to make cashless payments.

[2509] "Payment System" means an online payment platform for processing payment information.

[2510] "Emergency Support Feature" refers to the feature of the App that enables a User to receive necessary support in an emergency.

[2511] "Emergency information" refers to the user's report of an emergency situation.

[2512] "Translation Feature" refers to the automatic translation feature used by Users to communicate in their local language.

[2513] This invention is an integrated system that enables inbound tourists to enjoy their stay in Japan smoothly. The user, terminal, and server work together, allowing the user to manage their entire trip on a single platform.

[2514] First, the user launches the app using a device such as a smartphone or tablet. When the app is launched for the first time, the user enters their personal information, which is then sent from the device to the server. The server stores the received information in a database and sends a confirmation email to the user's email address to activate their account. The account is activated when the user approves this confirmation email.

[2515] Next, the user enters their travel itinerary, destinations, budget, and activities of interest into the app. The device sends this travel information to the server, which compares it with a database and uses a generative AI model (e.g., GPT-4) to suggest suitable attractions and activities. The proposed plan is automatically generated, saved in JSON format, and then sent to the user's device. The device displays the received plan to the user and makes adjustments if necessary.

[2516] Furthermore, when a user travels to a tourist spot or other destination, the device's GPS function is used to send the user's current location to the server. The server then calculates the optimal route from the user's current location to the destination. Detailed navigation information is generated and sent to the device. The device then displays this information on a map and provides real-time navigation.

[2517] When a user visits a store or uses a service, they activate the app's payment function and present the QR code to the store. This payment information is sent from the device to the server, and the server executes the payment using a payment system (e.g., Stripe, PayPal). The payment completion result is sent to the terminal, which displays it to the user.

[2518] In an emergency, the user enters emergency information using the app's emergency support function. This information is sent from the device to a server, which searches for the most appropriate emergency service or hospital information. The search results are sent to the device and displayed there. Furthermore, a translation function is provided if necessary, allowing the user to communicate with local staff.

[2519] Specific examples

[2520] Example 1: Creating a travel plan

[2521] User A launches the app and creates a seven-day travel plan. The destinations entered are Tokyo and Kyoto, with a budget of 100,000 yen. The entered information is sent to the server, which then suggests tourist spots such as Sensoji Temple, Skytree, Fushimi Inari Taisha Shrine, and Kiyomizu-dera Temple. Activities such as a maiko experience and a sushi-making experience are also incorporated into the plan. The generated plan is displayed on the device to User A, who then reviews the contents and finalizes the plan.

[2522] Example 2: Real-time navigation

[2523] User B uses the app's navigation function when traveling from Tokyo to Osaka. The current location (Tokyo Station) is obtained using the GPS function and sent to the server. The server calculates the optimal route from Tokyo Station to Osaka Station, generates detailed navigation including Shinkansen information, and sends it to the device. The device displays this on a map and provides real-time information on the next train and transfers.

[2524] Prompt Sentence Examples

[2525] "Generate a 7-day travel plan with a budget of 100,000 yen, focusing on Tokyo and Kyoto."

[2526] As described above, the present invention allows users to manage all travel-related information in one system, providing a smooth travel experience.

[2527] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2528] Step 1:

[2529] User information entry and submission

[2530] A user launches the app and navigates to the account creation screen, where they enter personal information such as their name, nationality, emergency contact, and email address. After entering the information, the device encrypts it using the HTTPS protocol and sends it to the server. The input contains personal information, and the output is encrypted data.

[2531] Step 2:

[2532] Server saves information and sends confirmation email

[2533] The server stores the received personal information in a database (e.g., MySQL). If the storage is successful, the server sends a confirmation email of account activation to the user's email address via the SMTP server. The input is the received encrypted personal information, and the output is the storage result in the database and the confirmation email.

[2534] Step 3:

[2535] User's travel information input and submission

[2536] The user inputs their travel itinerary, destinations, budget, and activities of interest into the app. The device sends this information to the server. The input is information about the trip, and the output is data sent to the server.

[2537] Step 4:

[2538] Travel plan generation by server

[2539] The server compares the received travel information with a database (e.g., MongoDB), selects appropriate tourist spots and activities using a generative AI model (e.g., GPT-4), and automatically generates a travel plan. The generated plan is saved in JSON format and sent to the device. The input is travel information, and the output is the generated travel plan.

[2540] Step 5:

[2541] View and adjust travel plans on your device

[2542] The device displays the travel plan received from the server to the user. The user reviews the plan and makes adjustments as necessary. If adjustments are necessary, the device sends the information back to the server. The input is the generated travel plan, and the output is the adjusted plan approval.

[2543] Step 6:

[2544] Launching the navigation function

[2545] When a user travels to a tourist spot or other destination, they input their destination into the app and activate the navigation function. The device uses GPS to obtain current location information and sends it to the server. The input is destination information and current location information, and the output is the data sent to the server.

[2546] Step 7:

[2547] Route calculation by the server

[2548] The server calculates the optimal route based on the received current location and destination information using the Google Maps API and route calculation algorithms. Detailed route guidance is generated in JSON format and sent to the device. The input is the current location information and destination information, and the output is the route guidance.

[2549] Step 8:

[2550] Displaying travel route on device

[2551] The device displays the route guidance received from the server on a map and provides real-time navigation. The user moves around while checking the app screen. The input is the route guidance, and the output is the display of navigation information.

[2552] Step 9:

[2553] Activating payment functions and sending information

[2554] When using a store or service, the user activates the app's payment function, selects a payment method, and presents a QR code. The terminal then sends the payment information to the server. The input is the payment information, and the output is the data sent to the server.

[2555] Step 10:

[2556] Server-based payment execution

[2557] The server processes the received payment information via a payment system (e.g., Stripe, PayPal). After the payment is completed, the result is sent to the terminal. The input is the payment information, and the output is the payment result.

[2558] Step 11:

[2559] Terminal indicates payment is complete

[2560] The terminal receives the payment result from the server, displays it to the user, and provides receipts and payment history as needed. The input is the payment result, and the output is a payment completion screen.

[2561] Step 12:

[2562] Enter and submit emergency information

[2563] When a user encounters an emergency, they activate the app's emergency support function and input the details of the emergency. The device then sends the emergency information to the server. The input is the emergency information, and the output is the data sent to the server.

[2564] Step 13:

[2565] Provision of emergency response information by server

[2566] The server searches for emergency services and hospital information nearest to the user's current location based on the received emergency information. The search results are sent to the device. The input is the emergency information, and the output is emergency response information.

[2567] Step 14:

[2568] Emergency information display and translation support on devices

[2569] The terminal displays emergency response information received from the server to the user and provides translation function as needed, allowing the user to communicate with local medical staff. The input is emergency response information, and the output is display of emergency information and translation support.

[2570] (Application example 1)

[2571] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2572] Support systems to help inbound tourists enjoy their stay in Japan include travel plan creation, navigation, cashless payment, and emergency assistance, but these services are often provided separately. Such systems require travelers to use multiple applications and devices, which is inconvenient. These functions are rarely integrated into autonomous transportation, particularly in these modes of transportation, so there is a need for centralized, integrated support.

[2573] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2574] In this invention, the server includes means for a user to input personal information, means for saving the input personal information in a database, means for proposing individual services based on the saved data, means for transmitting the proposals to a user terminal, means for managing the user's itinerary in the autonomous transportation means, means for providing navigation based on the user's location information, means for sending payment information to the server and executing payment, and means for providing information on nearby medical institutions in the event of an emergency. This allows travelers to use a single system to receive a variety of integrated services.

[2575] "User" refers to travelers and end users who use this system.

[2576] "Personal Information" refers to information about you, including your name, nationality, emergency contact details, email address, etc.

[2577] "Database" refers to an information system for storing and managing users' personal information, travel information, payment information, etc.

[2578] A "server" is a computer that manages the entire system and processes and stores data.

[2579] "Autonomous mobility" refers to devices that move without user intervention, such as self-driving vehicles and robots.

[2580] "Location information" refers to a user's current location and past location data obtained using technologies such as GPS.

[2581] "Navigation" refers to the function of calculating the optimal route from the current location to the destination and guiding the user along the route.

[2582] "Payment" refers to the process by which a user pays for goods or services cashlessly.

[2583] An "emergency" refers to a situation in which the user encounters a sudden situation such as an accident or sudden illness.

[2584] "Medical institution information" refers to information about medical facilities such as hospitals and clinics.

[2585] This invention is an integrated system that allows inbound tourists to enjoy their stay in Japan smoothly. The user, terminal, and server work together to provide personal information management, travel plan creation, real-time navigation, cashless payment, and emergency support.

[2586] System Program

[2587] The system includes the following elements:

[2588] 1. Registration of personal information

[2589] The user uses a terminal (for example, a touch panel inside the car) to input personal information (name, nationality, emergency contact, email address).

[2590] The terminal sends the entered personal information to the server, which stores it in a database.

[2591] Once saved, the server will send a confirmation email to the user's email address, asking them to activate their account.

[2592] 2. Create a travel plan

[2593] The user uses the terminal to input the desired travel dates, destinations, budget, and activities of interest.

[2594] The device sends the entered information to a server, which then compares the travel information with a database to select appropriate tourist spots and activities.

[2595] The server automatically generates a travel plan and sends it to the terminal, which displays it to the user for confirmation and adjustment.

[2596] 3. Real-time navigation

[2597] When a user travels to a tourist spot or other destination, the user inputs the destination into the terminal.

[2598] The device uses GPS to obtain its current location and transmits it to the server.

[2599] The server calculates the optimal route based on the current location and destination and generates detailed route guidance.

[2600] The device displays the route guidance received from the server on a map and provides real-time navigation.

[2601] 4. Payment and Cashless Payment

[2602] The user activates the payment function using the terminal and presents the QR code to the store.

[2603] The terminal sends the payment information to the server, which then executes the payment. Once the process is complete, the server sends the result to the terminal, which then displays a payment completion screen to the user.

[2604] 5. Emergency Support

[2605] When a user encounters an emergency, the user activates an emergency support function on the terminal and inputs the details of the emergency.

[2606] The device sends emergency information to the server, which then searches for information on emergency services and medical institutions closest to the current location and generates the data.

[2607] The device displays emergency information received from the server and provides translation functions as needed to help users communicate with local medical staff.

[2608] Specific examples

[2609] 1. Personal information registration:

[2610] Travelers enter personal information such as their name and nationality on a touch panel inside the vehicle and submit the information.

[2611] The server stores the data and sends a confirmation email.

[2612] 2. Create your travel plan:

[2613] Travelers enter their dates of stay and destination and choose from suggested attractions and activities.

[2614] The server automatically generates a travel plan and displays it on the in-car display.

[2615] 3. Real-time navigation:

[2616] The route from the current location to the next tourist spot is calculated, and the autonomous vehicle operates according to that route.

[2617] 4. Payment and Cashless Payment:

[2618] When making a purchase at a store, scan the QR code to complete the payment.

[2619] 5. Emergency Support:

[2620] When someone suddenly falls ill, the system searches for and navigates to the nearest hospital.

[2621] Prompt Sentence Examples

[2622] Design an autonomous mobility application with an integrated tourism support system. The app will have the following features:

[2623] 1. Registering user personal information and sending a confirmation email.

[2624] 2. Creating a travel plan and suggesting suitable tourist spots.

[2625] 3. Real-time navigation.

[2626] 4. Cashless payments within autonomous vehicles.

[2627] 5. Navigation to the nearest medical facility in case of an emergency.

[2628] Please also describe the data flow and the hardware and software used to realize these functions.

[2629] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2630] Step 1:

[2631] The user uses the touch panel in the car to enter personal information (name, nationality, emergency contact, email address), after which the device sends this information to the server and requests a confirmation email.

[2632] Input: Personal information entered by the user on the touch panel

[2633] Output: Personal information sent to the server, confirmation email request

[2634] Step 2:

[2635] The server stores the received personal information in a database, and once the storage is complete, the server sends a confirmation email to the user's email address, requesting that the account be activated.

[2636] Input: Personal information sent from the device, confirmation email request

[2637] Output: Personal information stored in database, confirmation email sent

[2638] Step 3:

[2639] The user uses the device to input their travel itinerary, destination, budget, and activities they are interested in. The device then sends this information to the server.

[2640] Input: Travel itinerary, destination, budget, and activities of interest entered by the user on the touch panel

[2641] Output: Trip information sent to the server

[2642] Step 4:

[2643] The server compares the received travel information with its database and selects appropriate tourist spots and activities, then automatically generates a travel plan and sends it to the device.

[2644] Input: Travel information

[2645] Output: Automatically generated itinerary, itinerary sent to the device

[2646] Step 5:

[2647] The user checks the travel plan and makes adjustments as necessary. The device sends the user's adjustment request to the server. The server then sends the final travel plan reflecting the adjustments to the device.

[2648] Input: User adjustment request

[2649] Output: Final itinerary with adjustments

[2650] Step 6:

[2651] When a user travels to a tourist spot or other destination, they input their destination into the device and activate the navigation function. The device then uses GPS to obtain their current location and sends it to the server.

[2652] Input: Destination or current location entered by the user on the touch panel

[2653] Output: Current location and destination sent to the server

[2654] Step 7:

[2655] The server calculates the optimal route based on the current location and destination, generates detailed route guidance including transportation information, and sends the generated route guidance to the device.

[2656] Input: Current location and destination

[2657] Output: Optimized travel route, directions sent to the device

[2658] Step 8:

[2659] The device displays the route guidance received from the server on a map and provides real-time navigation.

[2660] Input: Directions sent from the server

[2661] Output: Route directions displayed on a map

[2662] Step 9:

[2663] When using a store or service, the user activates the payment function on the device and presents the QR code to the store. The device then sends the payment information to the server and begins the payment process.

[2664] Input: Payment information entered by the user on the touch panel

[2665] Output: Payment information sent to the server

[2666] Step 10:

[2667] The server executes the payment based on the received payment information. Once the payment is complete, it sends the result to the terminal. The terminal then displays a payment completion screen to the user.

[2668] Input: Payment Information

[2669] Output: Payment result, payment result sent to the terminal

[2670] Step 11:

[2671] When a user encounters an emergency, the user activates the emergency support function using the terminal and inputs the details of the emergency, which then transmits the emergency information to the server.

[2672] Input: Emergency details entered by the user on the touch panel

[2673] Output: Urgent information sent to the server

[2674] Step 12:

[2675] The server searches for information on emergency services and medical institutions closest to the user's current location based on the emergency information, and sends it to the device. The device then displays the emergency information received from the server and provides translation functionality as needed.

[2676] Input: Emergency Information

[2677] Output: Information about nearby medical institutions, emergency information sent to the device

[2678] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2679] The present invention is an integrated system for helping inbound tourists have a comfortable stay in Japan, in which users, terminals, a server, and an emotion engine function in cooperation with each other. Specific embodiments for carrying out the present invention will be described below.

[2680] Account Registration

[2681] User:

[2682] A traveler launches the app and selects "Create a new account" on the initial screen. The user follows the app's instructions and enters personal information such as name, nationality, emergency contact, and email address.

[2683] Device:

[2684] The terminal transmits the personal information input by the user to the server.

[2685] server:

[2686] The server stores the received personal information in a database and sends the user a confirmation email, which the user must click to activate their account.

[2687] Creating a travel plan

[2688] User:

[2689] Users use the app to input their desired travel dates, destinations, budget, and activities they are interested in.

[2690] Emotion Engine:

[2691] The emotion engine recognizes emotions from the user's facial expressions, voice, text input, etc. and determines their emotional state.

[2692] Device:

[2693] The terminal transmits the input travel information and emotion information to the server.

[2694] server:

[2695] The server compares the travel information and emotion information with a database to select appropriate tourist spots and activities. The server adjusts the recommendations according to the emotional state recognized by the emotion engine. It then automatically generates a travel plan and presents it to the user.

[2696] Device:

[2697] The device displays the proposed travel plan to the user, prompting them to confirm and adjust it. The user then confirms the plan and finalizes it.

[2698] Real-time navigation and information

[2699] User:

[2700] When a user travels to a tourist spot or destination, they enter the destination into the app to activate the navigation function.

[2701] Device:

[2702] The device uses GPS to obtain its current location and transmits it to the server.

[2703] server:

[2704] The server calculates the optimal route based on the current location and destination, and generates detailed route guidance including transportation information.

[2705] Device:

[2706] The device displays the route guidance received from the server on a map and provides real-time navigation.

[2707] Payment and cashless payment

[2708] User:

[2709] When using a store or service, users activate the app's payment function and present the QR code to the store.

[2710] Device:

[2711] The terminal sends payment information to the server to initiate the payment process.

[2712] server:

[2713] The server checks the payment information, executes the payment, and sends the result to the terminal once the payment is complete.

[2714] Device:

[2715] The terminal will display a payment completion screen to the user and provide receipts and payment history as needed.

[2716] Emergency support

[2717] User:

[2718] When a user encounters an emergency, they activate the app's emergency support function and enter the details of the emergency.

[2719] Device:

[2720] The terminal transmits emergency information to the server.

[2721] server:

[2722] The server searches for information on emergency services and hospitals closest to the user's current location and generates data to provide to the user.

[2723] Device:

[2724] The device displays emergency information received from the server and provides translation functions as needed to help users communicate with local medical staff.

[2725] Recognizing and responding to user emotions

[2726] User:

[2727] A user's emotional state may change in various situations during travel, such as while moving around a tourist spot or participating in an activity.

[2728] Emotion Engine:

[2729] The emotion engine analyzes the user's facial expressions, tone of voice, text input, etc. in real time to determine the user's emotional state. If an emotion is recognized, the corresponding data is sent to the server.

[2730] server:

[2731] The server uses the emotion data to understand the user's current emotional state. For example, if the user is feeling stressed, it can suggest relaxing activities.

[2732] Device:

[2733] The terminal displays the suggestions from the server to the user and provides assistance in real time.

[2734] Specific examples

[2735] Example 1: Travel planning and emotional response

[2736] User:

[2737] Traveler A starts the app and plans a trip. The trip is for 7 days, with the main tourist destinations being Tokyo and Kyoto. The budget is 100,000 yen.

[2738] Traveler A selects cultural experiences and nature walks as activities that interest him.

[2739] Emotion Engine:

[2740] The emotion engine recognizes from Traveler A's voice and text input that he wants to relax.

[2741] Device:

[2742] The user's input information and emotion information are sent to the server.

[2743] server:

[2744] Based on traveler A's information and emotional data, the system suggests relaxing tourist spots such as Sensoji Temple, Fushimi Inari Taisha Shrine, and the bamboo forests of Kamakura. It also automatically generates a travel plan that includes a relaxing hot spring experience.

[2745] Device:

[2746] The travel plan is displayed to Traveler A, prompting them to confirm and adjust. Traveler A confirms and confirms the plan.

[2747] Example 2: Real-time emotion recognition and response

[2748] User:

[2749] Tourist B gets lost in the crowds of Tokyo and becomes stressed.

[2750] Emotion Engine:

[2751] The emotion engine recognizes stress from traveler B's facial expressions and tone of voice.

[2752] Device:

[2753] Emotion data is sent to the server.

[2754] server:

[2755] Based on current location data and emotional data, the app sug...

Claims

1. a means for a user to input personal information; A means for storing the entered personal information in a database; A means of proposing individual services based on the stored data; means for transmitting the proposal to a user terminal; A system including:

2. A means for the user to input desired travel dates, destinations, and budget; A way to suggest suitable tourist spots and activities based on the travel information entered, A means for automatically generating a travel plan including suggested tourist spots and activities and transmitting the plan to a user terminal; The system of claim 1 , comprising:

3. A means for acquiring a current location using a GPS function of a user terminal; A way to calculate the optimal travel route based on your current location and destination, means for displaying the calculated travel route on a user terminal; The system of claim 1 , comprising:

Images