System

Abstract

A system is provided.SOLUTION: In a smartphone application that supports travel within Japan for travelers from abroad, a system comprising: means for creating a traveler's account; means for suggesting a travel plan; means for providing translations in multiple languages; means for calculating a travel route; means for explaining local rules; and means for interfacing with other national applications.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Foreign tourists visiting Japan are often confused by differences in Japanese culture and local rules, making it difficult to fully enjoy their trip. Furthermore, many overseas smartphones and applications do not work properly in Japan. As a result, travelers often get lost, lack the means to obtain appropriate information, and experience inconvenience. There is a need to solve these problems and provide an environment in which foreign tourists can enjoy a more comfortable and fulfilling trip to Japan. [Means for solving the problem]

[0005] The present invention provides a smartphone application for assisting foreign travelers in traveling within Japan, the application including the following means:

[0006] 1. Provide a means for travellers to create accounts and provide the ability to input personal information about the traveller and store it in a database.

[0007] 2. It has a means of proposing travel plans, and generates optimal travel plans using a generation algorithm based on the traveler's input of purpose, duration, and areas of interest.

[0008] 3. Provide a means to provide translation in multiple languages ​​and translate traveler input text in real time.

[0009] 4. It has a means for calculating travel routes and calculates the optimal means of travel and route from the traveler's current location to the destination.

[0010] 5. Have a means to explain local rules and provide information such as points to consider when behaving in Japan and emergency response measures.

[0011] 6. Provide a means to link with other domestic applications, allowing users to smoothly use various services such as meal information and payment methods.

[0012] By taking the above measures, it is possible to solve the various issues that foreign tourists face in Japan in a unified manner and provide them with a comfortable travel experience.

[0013] "Foreign tourists" are tourists visiting Japan who travel internationally.

[0014] A "smartphone application" is a software program that runs on a smartphone and assists users in their travels.

[0015] The "account creation means" is a function that prompts the user to enter personal information and creates a user-specific account based on that information.

[0016] The "travel plan suggestion means" is a function that creates and suggests the most suitable travel plan based on the user's travel purpose and interests.

[0017] "Translation means" is a function that translates into multiple languages ​​in real time and supports user communication.

[0018] The "travel route calculation means" is a function that calculates the optimal travel route based on the current location and destination and provides guidance to the user.

[0019] "Local rule explanation means" is a function that provides users with information such as cultural rules in Japan and emergency response measures.

[0020] "Means of linking with other domestic applications" refers to a function that links with domestic services that provide meal information, payment methods, etc., enabling seamless use. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] System Overview

[0043] The present invention provides a smartphone application that helps foreign travelers enjoy a comfortable trip within Japan. This application is a system that includes functions such as account creation for travelers, travel plan suggestions, translation functions, route calculation, explanation of local rules, and linkage with other domestic applications. Details of each function and specific operating procedures are explained below.

[0044] Account creation method

[0045] 1. Initial Setup

[0046] Device: Turn on the power and display the initial setup screen.

[0047] User: Select a language and enter personal information.

[0048] Server: Receives the entered information, creates a traveler account, and stores it in a database.

[0049] Travel plan proposal method

[0050] 1. Travel plan creation

[0051] Device: Display the travel plan creation screen.

[0052] User: Enter the purpose of the trip, duration, and genres of interest.

[0053] Server: Based on this input information, an algorithm is used to generate the optimal travel plan.

[0054] Server: Sends the generated plan to the device and allows the user to view and edit the displayed plan.

[0055] Translation tools

[0056] 1. Real-time translation

[0057] Terminal: Display the translation function screen.

[0058] User: Enter the text you want to translate.

[0059] Server: Translates the input text and sends the results to the device.

[0060] Device: The translation results are displayed on the screen and played aloud if necessary.

[0061] Travel route calculation method

[0062] 1. Route calculation

[0063] Device: Display the destination input screen.

[0064] User: Enter destination.

[0065] Server: Calculates the optimal travel route based on current location information and destination information, and sends the results to the device.

[0066] Terminal: Displays travel route and navigation information on the screen.

[0067] Local Rules Explanation Method

[0068] 1. Provision of local rules

[0069] Terminal: Display the local rules explanation screen.

[0070] Terminal: Provides emergency support when users are in trouble.

[0071] Server: Sends appropriate advice and support information to the terminal in response to an emergency support request.

[0072] Linking with other domestic applications

[0073] 1. App Integration

[0074] Device: Display the domestic application integration settings screen.

[0075] User: Select the application you want to link and set up the link.

[0076] Server: Uses the application integration API to share user information and establish integration.

[0077] Terminal: Notifies the user that the connection is complete and makes the related application available.

[0078] Specific examples

[0079] Specific examples of travel plan proposals

[0080] 1. User operations

[0081] User: You are planning a two-week trip for sightseeing and you are interested in historical sites.

[0082] 2. Server Processing

[0083] Server: Based on this information, it generates a travel plan that includes recommended spots in Tokyo, Kyoto, Osaka, etc.

[0084] Example plans:

[0085] Day 1: Tokyo Tower, Sensoji Temple

[0086] Day 2: Kinkakuji Temple and Fushimi Inari Shrine in Kyoto

[0087] 3. Displaying the terminal

[0088] Terminal: Presents the generated plan to the user and allows the user to customize the plan.

[0089] Examples of translation functions

[0090] 1. User operations

[0091] User: Type "Where is the nearest train station?"

[0092] 2. Server Processing

[0093] Server: Translate this text into Japanese and make it "Where is the nearest station?"

[0094] 3. Displaying the terminal

[0095] Terminal: Helps users ask for directions by displaying translation results and playing them aloud.

[0096] As described above, the system of the present invention provides a multifunctional platform that meets the various needs of travelers and makes their travels more comfortable.

[0097] The processing flow will be explained below.

[0098] Account creation method

[0099] Step 1:

[0100] Terminal: When the application is launched for the first time, the initial setup screen is displayed.

[0101] Step 2:

[0102] User: Select the desired language from the language selection menu that appears.

[0103] Step 3:

[0104] Device: Displays the account creation screen based on the selected language.

[0105] Step 4:

[0106] User: Enter personal information such as name, email address, and password and press the "Register" button.

[0107] Step 5:

[0108] Terminal: Sends the entered information to the server.

[0109] Step 6:

[0110] Server: Creates a user account in the database based on the received information and sends a registration completion status to the terminal.

[0111] Step 7:

[0112] Terminal: Display a registration completion message to the user.

[0113] Travel plan proposal method

[0114] Step 1:

[0115] Device: Display the travel plan creation screen.

[0116] Step 2:

[0117] User: Enter the purpose of the trip (e.g. sightseeing), duration (e.g. 2 weeks), and genre of interest (e.g. historical buildings).

[0118] Step 3:

[0119] Terminal: Sends the entered information to the server.

[0120] Step 4:

[0121] Server: Runs the travel plan suggestion algorithm and generates the optimal travel plan based on the user's input information.

[0122] Step 5:

[0123] Server: Sends the generated travel plan to the terminal.

[0124] Step 6:

[0125] Terminal: Displays the generated travel plan to the user, allowing the user to review and edit the plan.

[0126] Translation tools

[0127] Step 1:

[0128] Terminal: Display the translation function screen.

[0129] Step 2:

[0130] User: Enter the text you want to translate.

[0131] Step 3:

[0132] Terminal: Sends entered text to the server.

[0133] Step 4:

[0134] Server: Uses the translation API to translate the input text into the specified language.

[0135] Step 5:

[0136] Server: Sends the translation results to the device.

[0137] Step 6:

[0138] Device: The translation results are displayed on the screen and played aloud if necessary.

[0139] Travel route calculation method

[0140] Step 1:

[0141] Device: Display the destination input screen.

[0142] Step 2:

[0143] User: Enter your destination (e.g. Kyoto Station).

[0144] Step 3:

[0145] Terminal: Sends current location information and destination information to the server.

[0146] Step 4:

[0147] Server: Executes the travel route calculation algorithm and calculates the optimal travel route.

[0148] Step 5:

[0149] Server: Sends calculated travel route data and navigation information to the terminal.

[0150] Step 6:

[0151] Terminal: Displays travel route and navigation information on the screen.

[0152] Local Rules Explanation Method

[0153] Step 1:

[0154] Terminal: Display the local rules explanation screen.

[0155] Step 2:

[0156] User: Check the local rules, or press the emergency support button if you need emergency support.

[0157] Step 3:

[0158] Terminal: When the emergency support button is pressed, the information is sent to the server.

[0159] Step 4:

[0160] Server: Provides specific advice and support information tailored to the user's situation and sends it to the terminal.

[0161] Step 5:

[0162] Terminal: Displays advice and assistance information received from the server to the user.

[0163] Linking with other domestic applications

[0164] Step 1:

[0165] Device: Display the domestic application integration settings screen.

[0166] Step 2:

[0167] User: Select the application you want to integrate.

[0168] Step 3:

[0169] Terminal: Sends the link setting information of the selected application to the server.

[0170] Step 4:

[0171] Server: Uses the application integration API to share user information and establish integration.

[0172] Step 5:

[0173] Server: Notifies the terminal that the connection is complete.

[0174] Step 6:

[0175] Terminal: Notifies the user that the linking is complete and makes the linked application available.

[0176] The above are the specific processing steps of the present invention.

[0177] Example 1

[0178] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0179] In order for foreign travelers to enjoy a comfortable trip within Japan, they need a wide range of information and support, but they often have difficulty gathering information and communicating due to differences in language and culture.The purpose of this invention is to alleviate difficulties during travel and ensure a comfortable travel experience by providing the information and support travelers need in a centralized manner.

[0180] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0181] In this invention, the server includes a means for creating a traveler's account, a means for inputting the traveler's purpose, period, and areas of interest and proposing an optimal travel plan using a generative AI model, and a means for translating the input text into multiple languages ​​and displaying and playing back the results by voice, thereby enabling travelers to easily complete complex procedures and obtain the necessary information in a timely manner.

[0182] The "means of creating a traveler's account" is the process by which a traveler inputs personal information, and the server uses that information to create and store a new account in the database.

[0183] "A means of having travelers input their purpose, duration, and areas of interest, and then using a generative AI model to suggest the optimal travel plan" refers to the process of having travelers input their purpose, duration, and areas of interest, and then using a generative AI model to suggest the optimal travel plan based on that data.

[0184] "Means for translating input text into multiple languages ​​and displaying and playing the results aloud" refers to a function that translates text input by a traveler into different languages ​​in real time, and not only displays the results on the device, but also plays them aloud if necessary.

[0185] "Means for calculating the optimal travel route based on the traveler's current location information and destination information and displaying the results" refers to the process in which, when a traveler inputs their current location and destination, the server calculates the optimal travel route and displays the results on the terminal.

[0186] "Means to explain local rules of the destination and provide emergency support" is a function that provides travelers with advance information about specific local rules of the destination they will be visiting, and provides appropriate support information and advice in real time in the event of an emergency.

[0187] "Means of linking with other domestic and international software and sharing user information" refers to the process of linking with other domestic and international applications via APIs, etc., and safely sharing user information.

[0188] This invention is software for mobile information terminals that helps foreign travelers to have a comfortable trip within Japan. This software has functions such as creating a travel account for travelers, proposing travel plans, translating, calculating travel routes, explaining local rules, and linking with other domestic and international software.

[0189] Account creation method

[0190] 1. Initial Setup

[0191] Terminal: When a traveler uses the application for the first time, an initial setup screen will be displayed.

[0192] User: Select your language and enter your personal information such as name, email address and password.

[0193] Server: Receives the personal information sent by the user and stores the new user account in the database. The server generates a unique user ID based on this information and returns it to the user.

[0194] Travel plan proposal method

[0195] 1. Create a travel plan

[0196] Terminal: Provides the user with a travel plan creation screen, prompting them to input the purpose of the trip, the duration, and the genres of interest (e.g., history, nature, shopping, etc.).

[0197] User: Enter the purpose of the trip, duration, and genres of interest, and press the send button.

[0198] Server: Receives this information and uses a generative AI model (e.g., GPT-4) to generate an optimal travel plan based on the user's preferences. Specifically, it extracts and combines relevant tourist spot information from the database to generate the optimal plan.

[0199] Server: Sends the generated travel plan to the terminal and displays it to the user.

[0200] Translation tools

[0201] 1. Real-time translation

[0202] Device: Display the translation function screen, and set up a text input field and a display area for the translation results.

[0203] User: Enter the text you want to translate and press the translate button.

[0204] Server: Receives the text sent by the user and translates the text using, for example, the Google Translate API.

[0205] Server: Sends the translation results to the device.

[0206] Terminal: The translation results are displayed on the screen and can also be played aloud if necessary.

[0207] Travel route calculation method

[0208] 1. Route calculation

[0209] Terminal: Provides a screen for entering a destination.

[0210] User: Enter the destination and press the route calculation button.

[0211] Server: Obtains the user's current location and destination information, and calculates the optimal route using, for example, the Google Maps API.

[0212] Server: Sends the calculation results to the terminal.

[0213] Device: Displays calculated route and navigation information on the screen.

[0214] Local Rules Explanation Method

[0215] 1. Provision of local rules

[0216] Terminal: Display the local rules explanation screen.

[0217] Users: Learn about the local rules of the places they visit.

[0218] Terminal: Also displays a screen that provides emergency support when the user is in trouble.

[0219] User: Press the emergency support button when in trouble.

[0220] Server: Receives emergency support requests, generates appropriate advice and assistance information, and sends it.

[0221] Terminal: Displays the support information sent from the server.

[0222] Linking with other domestic and international software

[0223] 1. App Integration

[0224] Device: Display the domestic and international software integration settings screen.

[0225] User: Select the software you want to link and set up the link.

[0226] Server: Uses the app integration API to securely share user information with selected software.

[0227] Terminal: Notifies the user that the connection is complete and displays the related software.

[0228] Specific examples

[0229] Specific examples of travel plan proposals

[0230] 1. User operations

[0231] User: You are planning a two-week trip for sightseeing and you are interested in historical sites.

[0232] 2. Server Processing

[0233] Server: Based on this information, a travel plan is generated that includes recommended spots in Tokyo, Kyoto, Osaka, etc.

[0234] Example plans:

[0235] Day 1: Tokyo Tower, Sensoji Temple

[0236] Day 2: Kinkakuji Temple and Fushimi Inari Shrine in Kyoto

[0237] 3. Displaying the terminal

[0238] Terminal: Presents the generated plan to the user and allows the user to customize the plan.

[0239] Examples of translation functions

[0240] 1. User operations

[0241] User: Type "Where is the nearest train station?"

[0242] 2. Server Processing

[0243] Server: Translate this text into Japanese and make it "Where is the nearest station?"

[0244] 3. Displaying the terminal

[0245] Terminal: Helps users ask for directions by displaying translation results and playing them aloud.

[0246] Prompt Sentence Examples

[0247] "I want to create software that suggests travel plans within Japan. Please explain the steps to generate the optimal travel plan when the user inputs the purpose of the trip, duration, and genres of interest."

[0248] The present invention allows travelers to easily complete complicated procedures and obtain necessary information in real time, making travel more comfortable and fulfilling.

[0249] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0250] Step 1:

[0251] Displaying the initial settings screen

[0252] Terminal: When a traveler launches the application, an initial setup screen appears. The terminal offers the user language selection, name, email address, and password entry fields.

[0253] Input: Personal information such as language, name, email address, and password

[0254] Output: Information entered on the initial setup screen

[0255] Step 2:

[0256] Entering and submitting personal information

[0257] User: The traveler selects a language, enters their name, email address, and password, and presses the send button.

[0258] Input: Personal information such as language, name, email address, and password

[0259] Output: Personal information is sent from the device to the server

[0260] Step 3:

[0261] Creating an account

[0262] Server: Saves a new user account in the database based on the received personal information. The server generates a unique user ID based on this information and returns it to the device.

[0263] Input: Personal information (language, name, email address, password)

[0264] Data processing: User ID generation, information storage in database

[0265] Output: Generated user ID

[0266] Step 4:

[0267] Display of travel plan creation screen

[0268] Terminal: Provides the user with a travel plan creation screen, displaying fields for inputting the purpose of the trip, duration, and genres of interest.

[0269] Input: None (screen input only)

[0270] Output: Travel plan creation screen

[0271] Step 5:

[0272] Enter and submit your travel purpose, duration, and genre

[0273] User: Enter the purpose of the trip, duration, and genres of interest, and press the send button.

[0274] Input: purpose of trip, duration, genre of interest

[0275] Output: Input information is sent from the device to the server

[0276] Step 6:

[0277] Generate a travel plan

[0278] Server: Based on the received travel information, a generative AI model (e.g., GPT-4) is used to generate an optimal travel plan based on the user's preferences. Specifically, it extracts and combines relevant tourist spot information from a database to create the optimal plan.

[0279] Input: purpose of trip, duration, areas of interest

[0280] Data Computing: Using generative AI models to generate travel plans

[0281] Output: Optimal travel plan

[0282] Step 7:

[0283] View and edit your travel plans

[0284] Terminal: Presents the generated travel plan to the user and provides an interface that allows the user to customize the plan.

[0285] Input: Optimal travel plan

[0286] Output: Customizable travel plan screen

[0287] Step 8:

[0288] Display of translation function screen

[0289] Device: Display the translation function screen, and set up a text input field and a display area for the translation results.

[0290] Input: None (screen input only)

[0291] Output: Translation function screen

[0292] Step 9:

[0293] Entering text and submitting translation requests

[0294] User: Enter the text you want to translate and press the send button.

[0295] Input: Text to translate

[0296] Output: The entered text is sent from the terminal to the server.

[0297] Step 10:

[0298] Text translation

[0299] Server: Receives the text sent by the user and translates the text using, for example, the Google Translate API.

[0300] Input: Text to translate

[0301] Data processing: Translate text using the translation API

[0302] Output: The translated text

[0303] Step 11:

[0304] Displaying translation results and playing audio

[0305] Terminal: The translation results are displayed on the screen and can also be played aloud if necessary.

[0306] Input: Translated text

[0307] Output: Display of translation results and audio playback

[0308] Step 12:

[0309] Display destination entry screen

[0310] Device: Displays a screen for entering your destination.

[0311] Input: None (screen input only)

[0312] Output: Destination input screen

[0313] Step 13:

[0314] Entering a destination and sending a route calculation request

[0315] User: Enter the destination and press the route calculation button.

[0316] Input: Destination information

[0317] Output: The entered destination information is sent from the device to the server.

[0318] Step 14:

[0319] Calculating the best route

[0320] Server: Obtains the user's current location and destination information, and calculates the optimal route using, for example, the Google Maps API.

[0321] Input: current location information, destination information

[0322] Data calculation: Calculate the optimal route using Google Maps API

[0323] Output: Optimal travel route

[0324] Step 15:

[0325] View route information

[0326] Device: Displays calculated route and navigation information on the screen.

[0327] Input: Optimal travel route

[0328] Output: Route information display screen

[0329] Step 16:

[0330] Display of local rules explanation screen

[0331] Terminal: Provides a screen explaining the local rules.

[0332] Input: None (screen input only)

[0333] Output: Local rules explanation screen

[0334] Step 17:

[0335] Display of emergency support screen

[0336] Terminal: Displays a screen that provides emergency support functions.

[0337] Input: None (screen input only)

[0338] Output: Emergency support screen

[0339] Step 18:

[0340] Submitting an Emergency Support Request

[0341] User: Press the emergency support button when in trouble.

[0342] Input: Emergency Support Request

[0343] Output: The request is sent to the server

[0344] Step 19:

[0345] Generate and submit support information

[0346] Server: Receives emergency support requests, generates appropriate advice and assistance information, and sends it to the device.

[0347] Input: Emergency Support Request

[0348] Data calculation: Generation of advice and assistance information according to the situation

[0349] Output: Support information

[0350] Step 20:

[0351] View support information

[0352] Terminal: Displays the support information sent from the server.

[0353] Input: Support Information

[0354] Output: Support information display screen

[0355] Step 21:

[0356] Display the app link setting screen

[0357] Device: Display the domestic and international software integration settings screen.

[0358] Input: None (screen input only)

[0359] Output: App link setting screen

[0360] Step 22:

[0361] Selecting and setting up linked apps

[0362] User: Select the software you want to link and set up the link.

[0363] Input: Selected linked application information

[0364] Output: Sending information from the device to the server

[0365] Step 23:

[0366] Sharing of User Information

[0367] Server: Uses the app integration API to securely share user information with selected software.

[0368] Input: Selected linked application information, user information

[0369] Data processing: Share user information using the collaboration API

[0370] Output: Connection completion information

[0371] Step 24:

[0372] Viewing integration notifications and related software

[0373] Terminal: Notifies the user that the connection is complete and displays the related software.

[0374] Input: Connection completion information

[0375] Output: Link notification and related software list

[0376] The above are the specific processing steps of the system. At each step, the user, terminal, and server cooperate to provide services that meet the needs of travelers.

[0377] (Application example 1)

[0378] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0379] Today's travelers need support not only in selecting tourist destinations and planning routes, but also in choosing food and learning dining etiquette based on local culture. Foreign travelers, in particular, are prone to experiencing inconveniences due to language barriers and cultural differences, and often have difficulty obtaining information about and choosing food. This can limit their travel experience and cause significant stress for travelers. Therefore, an integrated solution is needed to help travelers enjoy their meals while understanding local food culture, enabling them to have a comfortable trip.

[0380] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0381] In this invention, the server includes means for creating an account for a traveler, means for proposing a travel plan, means for providing translations in multiple languages, means for calculating a travel route, means for explaining local rules, means for linking with other domestic applications, means for proposing a meal plan based on the traveler's food preferences, means for calculating a travel route to a local restaurant, means for translating restaurant menus and dish descriptions, and means for explaining local rules regarding meals. This enables travelers to enjoy optimal travel plans and dining experiences without experiencing language or cultural barriers, and to have a comfortable and fulfilling trip.

[0382] "Means for creating a traveller's account" refers to a system whereby a traveller enters personal information and food preferences and stores them in a database.

[0383] The "means for suggesting travel plans" is a system that includes an algorithm that generates optimal travel plans based on the traveler's purpose, duration, and areas of interest.

[0384] A "means for providing translation in multiple languages" is a system that translates input text or audio information into multiple languages ​​and displays or plays the results.

[0385] The "means for calculating a travel route" is a system that calculates the optimal route based on the current location and destination and provides navigation information.

[0386] "A means of explaining local rules" is a system that provides travelers with the ability to explain local rules, manners, and precautions.

[0387] "Means for linking with other domestic applications" refers to a system that includes APIs and interfaces for sharing data with other domestic applications and establishing linking functions.

[0388] The "means for proposing meal plans based on a traveler's food preferences" is a system that includes an algorithm that proposes optimal meal plans based on a traveler's food preferences, current location, and regional information about the travel destination.

[0389] The "means for calculating a travel route to a local restaurant" is a system that calculates the optimal route from a traveler's current location to a local restaurant and provides that information.

[0390] The "means for translating restaurant menus and food descriptions" is a system that translates restaurant menus and food descriptions in real time and displays or plays back the results.

[0391] "Means for explaining local rules regarding meals" is a system that includes a function that explains local etiquette and rules that travelers should observe when eating.

[0392] The present invention is a system that provides a smartphone application that provides a series of support required for foreign travelers traveling within Japan. This system includes functions for creating a traveler's account, proposing travel plans, providing translation functions, calculating travel routes, explaining local rules, linking with other domestic applications, proposing meal plans, and providing information about meals.

[0393] System Overview

[0394] Account creation method

[0395] What it is: A way for travelers to enter their personal information and food preferences and store them in a database.

[0396] Example: When a traveler launches the app for the first time, they are prompted to select their language and enter their personal information. Here, they enter their name, email address, and food preferences (e.g., vegetarian, Italian, etc.) and send them to the server, which stores them in a database.

[0397] Travel plan proposal method

[0398] What it is: A way to generate optimal travel plans based on a traveler's purpose, duration, and areas of interest.

[0399] Example: If a traveler plans a two-week sightseeing trip and inputs that they are interested in historical buildings, the server will use this information to generate a travel plan including recommended spots in Tokyo, Kyoto, Osaka, etc. and send it to the device.

[0400] A means of providing translations in multiple languages

[0401] What it is: A means of translating input text or audio information into multiple languages ​​and displaying or playing the results.

[0402] Example: When a traveler types "Where is the nearest train station?", the server translates this into Japanese and displays "Where is the nearest station?" on the device, playing it aloud if necessary.

[0403] Travel route calculation method

[0404] What it is: A means of calculating the optimal route based on your current location and destination, and providing that navigation information.

[0405] Example: If a traveler wants to go to Kinkakuji Temple in Kyoto, the device will prompt the traveler to input their current location and destination, and the server will calculate the optimal travel route and send the results to the device. The traveler will then navigate using the route information displayed on the device.

[0406] Local Rules Explanation Method

[0407] Content: A means of explaining local rules, etiquette, and precautions to travelers.

[0408] Example: If a traveler encounters a problem on a train and uses the emergency support function, the device will display local rules (e.g., refrain from making calls on the train) and provide support information.

[0409] A means of linking with other domestic applications

[0410] What it is: A means to share data and establish federation capabilities with other domestic applications.

[0411] Example: If a traveler wants to link a food delivery app, the device will display a link setting screen, and when the traveler selects linking, the server will share data and establish the link.

[0412] Application example: Meal plan suggestions

[0413] Meal plan proposal method

[0414] What it is: A way to suggest optimal meal plans based on traveler's food preferences, current location, and local information about the destination.

[0415] Example: If a traveler inputs that they prefer "Italian food" and "vegetarian," the server searches for Italian vegetarian restaurants, generates the optimal meal plan, and sends it to the device.

[0416] A way to calculate travel routes to local restaurants

[0417] What it is: A way to calculate the best route from a traveler's current location to a local restaurant and display the results.

[0418] Example: A traveler calculates a route from their current location to the nearest Italian vegetarian restaurant, and the server sends the route information to the device. The traveler then uses the displayed route information to navigate to the restaurant.

[0419] A way to translate restaurant menus and food descriptions

[0420] What it is: A means of translating restaurant menus and food descriptions in real time and displaying or playing them aloud on a device.

[0421] Example: A traveler takes a photo of a restaurant menu and enters it into the app. The server translates the menu and displays "vegetarian pizza" and plays it aloud if needed.

[0422] A means of explaining local dietary rules

[0423] What it is: A way to explain to travelers the local etiquette and rules they should follow when dining out.

[0424] Example: If a traveler gets into trouble while eating, the app will display local rules such as "don't talk loudly while eating" and provide emergency support information.

[0425] Prompt Sentence Examples

[0426] User's name is John Doe, prefers Italian cuisine and is a vegetarian. Recommend the best Italian vegetarian restaurants nearby.

[0427] As a result, the system of the present invention allows travelers to enjoy optimal travel plans and dining experiences, and to have a comfortable and fulfilling trip without having to worry about language or cultural barriers.

[0428] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0429] Step 1:

[0430] The user launches the application for the first time and selects language preference and personal information, including name, email address, and food preferences (e.g., Italian, vegetarian).

[0431] Input: Name, email address, food preferences

[0432] Output: Personal information and food preferences are stored in a database

[0433] Specific operation: The terminal receives input data from the user and sends it to the server, which then stores the received information in a database.

[0434] Step 2:

[0435] The user moves to the travel plan proposal screen and inputs the purpose of the trip, the duration, and areas of interest.

[0436] Input: purpose of trip, duration, areas of interest

[0437] Output: Travel plan proposal data

[0438] How it works: The device receives user input and sends it to the server. The server uses the generative AI model to generate an optimal travel plan and sends it to the device.

[0439] Step 3:

[0440] The user uses the translation feature within the application to translate the text they entered.

[0441] Input:Text

[0442] Output: Translation result

[0443] Specific operation: The device sends the text entered by the user to the server. The server uses a translation algorithm to translate the entered text into the target language and sends the translation result to the device. The device displays the result and plays it aloud if necessary.

[0444] Step 4:

[0445] The user inputs a destination within the application and calculates a route.

[0446] Input:Destination

[0447] Output: Optimal travel route

[0448] Specific operation: The device sends current location information and destination information entered by the user to the server. The server uses a geographic information system (GIS) to calculate the optimal travel route and sends the results to the device. The device then displays the route information.

[0449] Step 5:

[0450] The user is taken to a meal plan suggestion screen where suggestions are given based on their food preferences and current location.

[0451] Input: Food preferences, current location

[0452] Output: Optimal meal plan

[0453] How it works: The device sends the user's dining preferences and current location information to the server. The server then searches a database of restaurants and generates an optimal meal plan, which it then sends to the device. The device then displays the results.

[0454] Step 6:

[0455] A user translates a restaurant menu and checks the dish descriptions.

[0456] Input: Menu text or image

[0457] Output: Translated menu and dish descriptions

[0458] Specific operation: The device sends the menu text or image data from the user to the server. The server uses OCR (optical character recognition) and a translation algorithm to translate the menu and send the result to the device. The device then displays or plays the translated menu aloud.

[0459] Step 7:

[0460] Users can check local rules while eating and get any assistance they need.

[0461] Input: Request for local rules

[0462] Output: Local rule explanation and supporting information

[0463] Specific operation: The terminal sends a request for a local rule from the user to the server. The server searches the database for the relevant local rule information and sends the result to the terminal. The terminal displays an explanation of the local rule and provides the user with the necessary information.

[0464] In this way, the system provides multifunctional support to help travelers enjoy their trips comfortably.

[0465] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0466] System Overview

[0467] This invention provides a smartphone application that helps foreign travelers enjoy a comfortable trip within Japan. This application is a system that combines account creation for travelers, travel plan suggestions, translation functions, route calculations, explanations of local rules, and integration with other domestic applications, as well as an emotion engine that recognizes the user's emotions. Details of each function and specific operating procedures are explained below.

[0468] Account creation method

[0469] 1. Initial Setup

[0470] Device: Turn on the power and display the initial setup screen.

[0471] User: Select a language and enter personal information.

[0472] Server: Creates a traveler account based on the entered information and stores it in a database.

[0473] Travel plan proposal method

[0474] 1. Travel plan creation

[0475] Device: Display the travel plan creation screen.

[0476] User: Enter the purpose of the trip, duration, and genres of interest.

[0477] Server: Based on this input information, the generation algorithm is executed to generate the optimal travel plan.

[0478] Server: Sends the generated plan to the device and allows the user to view and edit the displayed plan.

[0479] Translation tools

[0480] 1. Real-time translation

[0481] Terminal: Display the translation function screen.

[0482] User: Enter the text you want to translate.

[0483] Terminal: Sends entered text to the server.

[0484] Server: Uses the translation API to translate the input text into the specified language.

[0485] Server: Sends the translation results to the device.

[0486] Device: The translation results are displayed on the screen and played aloud if necessary.

[0487] Travel route calculation method

[0488] 1. Route calculation

[0489] Device: Display the destination input screen.

[0490] User: Enter destination.

[0491] Terminal: Sends current location information and destination information to the server.

[0492] Server: Executes the travel route calculation algorithm and calculates the optimal travel route.

[0493] Server: Sends calculated travel route data and navigation information to the terminal.

[0494] Terminal: Displays travel route and navigation information on the screen.

[0495] Local Rules Explanation Method

[0496] 1. Provision of local rules

[0497] Terminal: Display the local rules explanation screen.

[0498] Terminal: Provides emergency support when users are in trouble.

[0499] Server: Sends appropriate advice and support information to the terminal in response to an emergency support request.

[0500] Linking with other domestic applications

[0501] 1. App Integration

[0502] Device: Display the domestic application integration settings screen.

[0503] User: Select the application you want to integrate.

[0504] Terminal: Sends the link setting information of the selected application to the server.

[0505] Server: Uses the application integration API to share user information and establish integration.

[0506] Terminal: Notifies the user that the connection is complete and makes the related application available.

[0507] Emotion engine integration

[0508] The present invention is combined with an emotion engine that recognizes the user's emotions and analyzes user input and sensor data to determine the user's emotional state.

[0509] Using an emotion engine to suggest travel plans

[0510] The emotion engine takes into account the user's emotional state when suggesting travel plans, resulting in more personalized suggestions.

[0511] for example:

[0512] 1. User: When entering a travel plan, if the emotion engine determines that the user is "tired," it will suggest a plan that includes many relaxing spots.

[0513] 2. User: If the emotion engine determines that the user is "excited" when entering travel plans, it will suggest plans that include many active activities.

[0514] Using emotion engine in translation function

[0515] The emotion engine is also used in the translation function, providing translations in an appropriate tone according to the user's emotional state.

[0516] for example:

[0517] 1. User: If the emotion engine determines that the user is "lost," provide a translation in a reassuring tone.

[0518] 2. User: If the emotion engine determines that the user is grateful, it will provide a translation in a polite tone.

[0519] Using the emotion engine to explain local rules

[0520] By using the emotion engine in explaining local rules, it becomes possible to provide explanations that correspond to the user's emotional state.

[0521] for example:

[0522] 1. User: If the emotion engine determines that the user is "nervous," it will provide a more detailed and polite explanation.

[0523] 2. User: If the emotion engine determines that the user is "relaxed," provide a brief, to-the-point explanation.

[0524] Specific examples

[0525] A concrete example of travel plan suggestions using an emotion engine

[0526] 1. User operations

[0527] User: You are planning a two-week trip for sightseeing and you are interested in historical sites.

[0528] Emotion engine: Determines that the user is tired.

[0529] 2. Server Processing

[0530] Server: Based on the judgment of the emotion engine, generate a travel plan that includes many relaxing spots.

[0531] Example plans:

[0532] Day 1: Tokyo Tower, Sensoji Temple

[0533] Day 2: Kyoto's Kinkakuji Temple, a leisurely stroll in Arashiyama, and more

[0534] 3. Displaying the terminal

[0535] Terminal: Presents the generated plan to the user and allows the user to customize the plan.

[0536] A concrete example of a translation function using an emotion engine

[0537] 1. User operations

[0538] User: Type "I'm lost."

[0539] Emotion engine: Determines if the user is feeling anxious.

[0540] 2. Server Processing

[0541] Server: Uses the translation API to generate a translation in a tone that calms anxiety.

[0542] Translation result: "Don't worry. The nearest station is very close."

[0543] 3. Displaying the terminal

[0544] On the device: Displays the translation results and plays them aloud to give the user peace of mind.

[0545] The above is an embodiment of the invention that combines specific processing steps of the present invention and an emotion engine.

[0546] The processing flow will be explained below.

[0547] Specific processing steps for travel plan proposals using an emotion engine

[0548] Step 1:

[0549] Device: Display the travel plan creation screen.

[0550] Step 2:

[0551] User: Enter the purpose of the trip (e.g. sightseeing), duration (e.g. 2 weeks), and genre of interest (e.g. historical buildings).

[0552] Step 3:

[0553] Terminal: Sends the user's facial expressions and voice data to the emotion engine along with the user's input.

[0554] Step 4:

[0555] Emotion engine: Analyzes the user's facial expressions and voice data to determine the user's emotional state (e.g., tired, excited, etc.).

[0556] Step 5:

[0557] Emotion engine: Sends the judgment result of the emotional state to the server.

[0558] Step 6:

[0559] Server: Based on the user's input information and the judgment results of their emotional state, the server runs a travel plan suggestion algorithm and generates an optimal travel plan.

[0560] Example: If the user is determined to be tired, generate a plan that includes many relaxation spots.

[0561] Step 7:

[0562] Server: Sends the generated travel plan to the terminal.

[0563] Step 8:

[0564] Terminal: Displays the generated travel plan to the user, allowing the user to review and edit the plan.

[0565] Specific processing steps for translation function using emotion engine

[0566] Step 1:

[0567] Terminal: Display the translation function screen.

[0568] Step 2:

[0569] User: Enters the text they want to translate (e.g., "Where is the nearest train station?").

[0570] Step 3:

[0571] Terminal: Sends the user's facial expressions and voice data to the emotion engine along with the user's input.

[0572] Step 4:

[0573] Emotion engine: Analyzes the user's facial expressions and voice data to determine the user's emotional state (e.g., anxiety, joy, etc.).

[0574] Step 5:

[0575] Emotion engine: Sends the judgment result of the emotional state to the server.

[0576] Step 6:

[0577] Server: Based on the user's input text and the judged emotional state, the server translates in an appropriate tone.

[0578] Step 7:

[0579] Server: Translates the results (e.g., "Don't worry. The nearest station is nearby.") to the device.

[0580] Step 8:

[0581] Device: The translation results are displayed on the screen and played aloud if necessary.

[0582] Specific processing steps for explaining local rules using the emotion engine

[0583] Step 1:

[0584] Terminal: Display the local rules explanation screen.

[0585] Step 2:

[0586] User: Enter your problem or question, or press the emergency support button.

[0587] Step 3:

[0588] Terminal: Sends the user's facial expressions and voice data to the emotion engine along with the user's input.

[0589] Step 4:

[0590] Emotion engine: Analyzes the user's facial expressions and voice data to determine the user's emotional state (e.g., tension, relief, etc.).

[0591] Step 5:

[0592] Emotion engine: Sends the judgment result of the emotional state to the server.

[0593] Step 6:

[0594] Server: Based on the user's situation and emotional state, the server generates appropriate explanations of local rules and emergency support advice.

[0595] For example, if the user is perceived as nervous, provide more detailed and polite explanations.

[0596] Step 7:

[0597] Server: Sends the generated advice and support information to the terminal.

[0598] Step 8:

[0599] Terminal: Displays advice and support information received from the server to the user, giving them a sense of security.

[0600] The above are the specific processing steps of the present invention.

[0601] Example 2

[0602] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0603] For foreign tourists to travel comfortably within Japan, there are a wide range of issues to overcome, including language barriers, transportation methods, local rules, etc. In addition, it is also necessary to provide appropriate plans and support based on the emotional state of each individual traveler, which presents many challenges that cannot be addressed by existing travel support systems.

[0604] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0605] In this invention, the server includes means for creating a traveler's account, means for proposing a travel plan, means for providing translations in multiple languages, means for calculating travel routes, means for explaining local rules, means for linking with other domestic applications, and means for integrating an emotion engine that analyzes the user's emotional state. This makes it possible to provide optimal plans and support according to the emotions of travelers so that foreign travelers can travel comfortably within Japan.

[0606] "Means for creating a traveler's account" is a general term for devices and software that allow a traveler to enter personal information and store that information in a database.

[0607] "Means for proposing travel plans" is a general term for devices and software that use a generation algorithm to generate optimal travel plans based on inputs from travelers regarding their purpose, duration, and areas of interest, and then display them so that users can customize them.

[0608] "Means for providing translation in multiple languages" is a general term for devices and software that have the function of translating text entered by a user into a specified language and displaying the translation result on a screen or playing it aloud.

[0609] "Means for calculating a travel route" is a general term for devices and software that calculate the optimal travel route based on the destination and current location information entered by the user and provide navigation information.

[0610] "Means for explaining local rules" is a general term for devices and software that provide users with information about local culture, customs, and manners.

[0611] "Means for linking with other domestic applications" is a general term for devices and software for sharing information and establishing linkages with other domestic applications used by travelers.

[0612] An "emotion engine" is a general term for devices and software that analyze user input and sensor data to determine an emotional state and then respond or make appropriate suggestions accordingly.

[0613] MODE FOR CARRYING OUT THE INVENTION

[0614] The present invention provides an information processing device that helps foreign travelers to have a comfortable trip within Japan. This device is a system that includes functions such as creating a travel account for the traveler, proposing travel plans, translating into multiple languages, calculating travel routes, explaining local rules, linking with other domestic applications, and integrating an emotion engine.

[0615] Account creation method

[0616] Device: Turn on the power, launch the application, and display the initial setup screen. The user selects a language and enters personal information such as name, email address, and password. The device temporarily stores this information and sends it to the server.

[0617] Server: Stores the received information in a database and generates a confirmation message for the account creation. The device displays a notification to the user that the account was created successfully and provides the option to proceed to the next step.

[0618] Travel plan proposal method

[0619] Terminal: Displays a travel plan creation screen and provides input fields for the purpose of the trip, duration, and genres of interest. The user enters this information, and the terminal sends it to the server.

[0620] Server: Based on the received information, the server uses a generative AI model to generate an optimal travel plan. The generated plan is sent to the device, where the user can view and edit it.

[0621] Translation tools

[0622] Terminal: Displays the translation function screen and provides the user with a text input field. The user enters the text to be translated and selects the language to specify. The terminal sends the entered text and the selected language to the server.

[0623] Server: Calls the translation API to translate the input text into the specified language. The translation result is sent to the device, which displays it on the screen and, if necessary, plays it aloud.

[0624] Travel route calculation method

[0625] Device: Displays a destination entry screen and gives the user the option to enter their current location and destination. The user can enter their current location and destination, or use the GPS function to automatically obtain their current location. The device then sends this information to the server.

[0626] Server: Executes the travel route calculation algorithm to calculate the optimal travel route. Sends the calculated route and navigation information to the device, which displays it.

[0627] Local Rules Explanation Method

[0628] Terminal: Displays a local rules explanation screen and provides information on local culture, customs, and manners. If a user is in trouble, they can press the emergency support button to request assistance. The terminal then sends the request to the server.

[0629] Server: Generates appropriate advice and support information based on the request and sends it to the terminal, which then displays it to the user.

[0630] Linking with other domestic applications

[0631] Terminal: The domestic application linkage setting screen is displayed, along with a list of applications to be linked. The user selects the application they want to link, and the terminal sends the setting information to the server.

[0632] Server: Calls the application integration API, shares user information with related applications, and establishes integration. The server sends a notification of integration completion to the device, and the device notifies the user.

[0633] Emotion engine integration

[0634] The system incorporates an emotion engine that analyzes user input and sensor data to determine the user's emotional state, and based on this, it makes suggestions and offers the following:

[0635] Using an emotion engine to suggest travel plans

[0636] When a user inputs a travel plan, the emotion engine analyzes the user's state. For example, if it determines that the user is tired, it will generate a plan that includes many relaxing spots. On the other hand, if it determines that the user is excited, it will suggest a plan that includes many active activities.

[0637] Example prompt sentence:

[0638] "I want to relax, so please tell me about a travel plan that includes places where I can relax."

[0639] Using emotion engine in translation function

[0640] The emotion engine also analyzes when a user uses the translation function. For example, if the user is feeling anxious, the system will provide a translation in a reassuring tone.

[0641] Example prompt sentence:

[0642] "I'm lost. Please tell me what to do."

[0643] Using the emotion engine to explain local rules

[0644] When a user requests emergency support, the emotion engine analyzes the user's state and, if it determines that the user is nervous, for example, provides a detailed and thorough explanation.

[0645] This will enable us to provide a comprehensive support system to help foreign travelers enjoy a comfortable trip within Japan.

[0646] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0647] Program processing flow

[0648] Account creation method

[0649] Step 1:

[0650] When the device is turned on, it launches the application and displays the initial setup screen. When the user selects a language, the device switches the interface based on that selection: input (language selected by the user), output (interface display in the selected language).

[0651] Step 2:

[0652] The user enters personal information such as name, email address, password, etc. Input (personal information entered by the user), output (personal information once input is complete).

[0653] Step 3:

[0654] The terminal temporarily stores the input information and sends it to a server via the Internet. Input (user's personal information), Output (sending information to the server).

[0655] Step 4:

[0656] The server stores the received information in a database and generates a confirmation message for creating an account. Data processing (storing personal information), data calculation (generating a confirmation message), output (confirmation message).

[0657] Step 5:

[0658] The terminal displays a notification to the user that the account was created successfully and provides the option to proceed to the next step. Input (confirmation message received from the server), Output (notification displayed to the user).

[0659] Travel plan proposal method

[0660] Step 1:

[0661] The terminal displays a travel plan creation screen and provides input fields for the purpose of the trip, duration, and genres of interest. Input (request for screen display), output (display of input fields).

[0662] Step 2:

[0663] The user inputs the purpose of the trip, the duration, and the genres they are interested in. Input (user's travel information), output (entered travel information).

[0664] Step 3:

[0665] The terminal collects this information and sends it to the server. Input (travel information), Output (sending information to the server).

[0666] Step 4:

[0667] The server uses a generative AI model based on the received information to generate an optimal travel plan, including data processing (analysis of travel information), data calculation (generation of optimal plan), and output (generated travel plan).

[0668] Step 5:

[0669] The server sends the generated itinerary to the terminal. Input (generated itinerary), Output (transmission to terminal).

[0670] Step 6:

[0671] The terminal displays the received travel plan to the user and allows editing and saving. Input (travel plan received from server), Output (providing display and editing options to the user).

[0672] Translation tools

[0673] Step 1:

[0674] The terminal displays a translation function screen and provides the user with a text input field. Input (request for screen display), output (display of input field).

[0675] Step 2:

[0676] The user enters the text to be translated and selects the language to be translated. Input (translation text and language selection), Output (entered text and selected language).

[0677] Step 3:

[0678] The terminal sends the entered text and the selected language to the server. Input (text and language), Output (send to server).

[0679] Step 4:

[0680] The server calls the translation API and translates the input text into the specified language. Data processing (text analysis), data calculation (translation processing), and output (translation results) are performed.

[0681] Step 5:

[0682] The server sends the translation results to the terminal. Input (translation results), output (transmission to terminal).

[0683] Step 6:

[0684] The device displays the translation results on the screen and plays them aloud if necessary. Input (translation results), output (screen display and audio playback).

[0685] Travel route calculation method

[0686] Step 1:

[0687] The device displays a destination entry screen and gives the user the option to enter their current location and destination. Input (request for screen display), Output (display of input field).

[0688] Step 2:

[0689] The user can input their current location and destination, or use the GPS function to automatically obtain their current location. Input (current location and destination, or GPS data), output (input location data).

[0690] Step 3:

[0691] The device sends the acquired current location information and destination information to the server. Input (current location and destination information), output (transmission to server).

[0692] Step 4:

[0693] The server executes the travel route calculation algorithm to calculate the optimal travel route, including data processing (analysis of location information), data calculation (route calculation), and output (calculated route data).

[0694] Step 5:

[0695] The server sends the calculated route and navigation information to the device.,Input (calculated route data), Output (transmission to device).

[0696] Step 6:

[0697] The device displays navigation information and maps on the screen, allowing the user to check the route. Input (route data), output (display of map and information).

[0698] Local Rules Explanation Method

[0699] Step 1:

[0700] The terminal displays a local rules explanation screen and provides information on the culture, customs, and manners of each region. Input (request for screen display), output (display of rules explanation).

[0701] Step 2:

[0702] When a user is in trouble, he / she presses the emergency support button to request assistance. Input (pressing the support request button), Output (support request data).

[0703] Step 3:

[0704] The terminal sends an emergency support request to the server. Input (support request data), Output (transmission to server).

[0705] Step 4:

[0706] The server generates appropriate advice and support information based on the request content and sends it to the terminal. Data processing (analysis of the request content), data calculation (generation of advice), output (generated support information).

[0707] Step 5:

[0708] The terminal displays the received advice and support information to the user. Input (received support information), output (display to user).

[0709] Linking with other domestic applications

[0710] Step 1:

[0711] The terminal displays the domestic application linkage setting screen and displays a list of applications to be linked. Input (request for screen display), output (display of setting screen).

[0712] Step 2:

[0713] The user selects the application they want to link. Input (selected application), output (selected information).

[0714] Step 3:

[0715] The terminal sends the link setting information of the selected application to the server. Input (link setting information), output (transmission to server).

[0716] Step 4:

[0717] The server calls the application integration API, shares user information with related applications, and establishes integration. Data processing (sharing user information), data calculation (establishing integration), and output (notifying that integration is complete) are performed.

[0718] Step 5:

[0719] The server sends a notification of completion of collaboration to the terminal. Input (notification of completion of collaboration), output (send to terminal).

[0720] Step 6:

[0721] The device notifies the user that the connection is complete, and synchronization with related applications becomes possible. Input (connection completion notification), Output (notification to the user).

[0722] Emotion engine integration

[0723] Step 1:

[0724] The server analyzes the user's input and sensor data to determine the emotional state. Input (user's input and sensor data), Output (determined emotional state).

[0725] Step 2:

[0726] When a user inputs their travel plan, the emotion engine analyzes the user's state: input (user's travel information), output (determining their emotional state).

[0727] Step 3:

[0728] The server generates a travel plan based on the user's emotional state. For example, if the server determines that the user is tired, it generates a plan that includes many relaxation spots. Data processing (analysis of emotional state), data calculation (plan generation), output (generated plan).

[0729] Step 4:

[0730] The server uses an emotion engine to translate in a reassuring tone if the user is feeling anxious. Data processing (analysis of emotional state), data calculation (translation processing), output (translation results).

[0731] Step 5:

[0732] The server analyzes the user's emotional state during emergency support and provides detailed and thorough explanations. Data processing (analysis of emotional state), data calculation (generation of support information), output (support information).

[0733] (Application example 2)

[0734] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0735] Previous travel support applications lacked comprehensive support for foreign travelers to have a comfortable stay in Japan, particularly due to language barriers and a lack of understanding of local rules. Furthermore, the guidance and product explanations within the virtual stores travelers visited were limited to providing general information and lacked personalized responses tailored to the individual emotional state of the travelers. This resulted in a decrease in traveler satisfaction.

[0736] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0737] In this invention, the server includes means for creating an account for a traveler, means for proposing a travel plan, means for providing translations in multiple languages, means for calculating travel routes, means for explaining local rules, means for linking with other domestic applications, means for recognizing a user's emotions and providing information in an appropriate tone accordingly, and means for proposing product guides and recommended products in a virtual store. This makes it possible to provide personalized guides and product explanations tailored to the emotional state of a traveler while overcoming language barriers, thereby improving traveler satisfaction.

[0738] "Means for creating a traveler's account" refers to the process by which a traveler inputs personal information and manages that information on the server.

[0739] The "means for proposing a travel plan" is a process for generating and proposing an optimal travel plan based on the traveler's purpose, period, and areas of interest.

[0740] A "means for providing translation in multiple languages" is a process for translating user-entered text into multiple languages ​​and providing the results.

[0741] The "means for calculating a travel route" is a process for calculating and providing guidance on the optimal travel route based on the user's current location and destination.

[0742] "Means of explaining local rules" refers to the process of explaining the rules and manners of each region in Japan to foreign tourists.

[0743] "Means of linking with other domestic applications" refers to the process by which travelers link with other domestic applications when using those applications.

[0744] "Means for recognizing the user's emotions and providing information in an appropriate tone" is a process for analyzing and determining the user's emotional state and providing information in the most appropriate tone based on that.

[0745] "Means for providing product information and suggesting recommended products within a virtual store" refers to the process of providing product information to users within a virtual store and suggesting recommended products based on the user's interests and emotions.

[0746] MODE FOR CARRYING OUT THE INVENTION

[0747] This invention provides a mobile terminal application system to support foreign travelers in enjoying a comfortable trip within Japan. This system can create a traveler's account, suggest travel plans, provide translations in multiple languages, calculate travel routes, explain local rules, link with other domestic applications, and even provide information in an appropriate tone according to the user's emotional state.

[0748] Hardware and software used

[0749] Hardware: Smartphones, smart glasses, head-mounted displays

[0750] Software: Python, TensorFlow (emotion engine), Google Translate API (translation), Google Maps API (route calculation)

[0751] Embodiments of each means of the system

[0752] 1. How to create a traveller account

[0753] The user enters personal information and sends it to the server, which stores it in a database and creates an account for the traveler.

[0754] 2. A way to suggest travel plans

[0755] The user inputs the purpose, duration, and areas of interest of the trip. The server runs a generation algorithm based on this input information to generate an optimal travel plan. In doing so, it uses an emotion engine to analyze the user's emotional state. For example, if the server determines that the user is "tired," it will suggest a plan that includes many relaxation spots.

[0756] 3. A way to provide translations in multiple languages

[0757] The user enters the text they want to translate. The device sends this text to the server. The server uses the Google Translate API to translate it and sends the results back to the device. The translation is delivered in an appropriate tone based on the emotion engine.

[0758] 4. Means of calculating travel routes

[0759] The user inputs a destination. The device sends the current location and destination information to the server. The server uses the Google Maps API to calculate the optimal route and provide navigation information.

[0760] 5. Means of Explaining Local Rules

[0761] By storing local rule information on the device, the information can be displayed when the user is in trouble. In addition, the server provides appropriate advice and support information in response to emergency support requests from the device.

[0762] 6. Means of linking with other domestic applications

[0763] On the device, select the application you want to link and send the link setting information to the server. The server then uses the link API to share information between the applications and establish the link.

[0764] 7. A way to recognize the user's emotions and deliver information in an appropriate tone.

[0765] The camera feed and audio data are fed into a TensorFlow model to analyze the user's emotional state. For example, if the system detects that the user is feeling anxious, it will provide information in a tone that will calm the user's anxiety.

[0766] 8. A means of product guidance and recommendations within a virtual store

[0767] When a user views product information in a virtual store, the system translates and displays guidance information. It recognizes the user's emotional state and provides recommendations and guidance accordingly.

[0768] Specific examples

[0769] Example 1: Travel plan suggestions

[0770] 1. A user plans a two-week sightseeing trip and indicates that they are interested in historical sites.

[0771] 2. The emotion engine determines that the user is tired.

[0772] 3. The server generates an itinerary that includes many relaxing spots.

[0773] 4. Display the generated plan on the device and allow the user to customize it.

[0774] Example 2: Applying the translation function

[0775] 1. The user types "I'm lost."

[0776] 2. The emotion engine determines that the user is feeling anxious.

[0777] 3. The server generates a translation in a calming tone.

[0778] 4. The device will display the message "Don't worry, the nearest station is right nearby" and play it aloud.

[0779] Example 3: Guidance in a virtual store

[0780] 1. A user views products in a virtual store.

[0781] 2. The emotion engine determines the user's emotional state and recognizes that they are feeling anxious.

[0782] 3. The server provides product descriptions in a reassuring translation tone.

[0783] 4. The device will display and play a message saying, "This product is a high-quality item that you can use with confidence."

[0784] Prompt Sentence Examples

[0785] If users are feeling unsure, provide a reassuring tone in your product description. Translate the product information below to create a reassuring message:

[0786] Product Name: High-quality bags

[0787] Description: This bag is durable and can be used for a long time. It is made of high-quality materials and is lightweight and easy to carry.

[0788] Example output: "This bag is of very high quality and will last a long time. You can use it with confidence."

[0789] The above is an embodiment of the present invention.

[0790] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0791] Step 1:

[0792] The user launches the smartphone application, selects a language, and enters their personal information. The device sends this information to the server, which then creates a traveler account based on the information entered and stores it in a database.

[0793] Input: Language selection, personal information

[0794] Output: Traveler account created and saved to database

[0795] Data processing: Convert input information into database format and save

[0796] Step 2:

[0797] The user opens the travel plan suggestion screen within the application and inputs the purpose of the trip, duration, and genres of interest. The device sends this input information to the server, which then runs a generation algorithm to generate the optimal travel plan. In doing so, the server uses an emotion engine to determine the user's emotional state and personalize the plan accordingly.

[0798] Input: purpose of trip, duration, genre of interest

[0799] Output: A personalized itinerary

[0800] Data processing: Input information into a generative algorithm, and an emotion engine determines the emotional state and optimizes the plan.

[0801] Step 3:

[0802] The user uses the translation function and inputs the text they want to translate. The device sends this text to the server, which translates it using the Google Translate API. Once the translation result is generated, the server adjusts the tone using an emotion engine and sends it to the device for display.

[0803] Input: The text you want to translate

[0804] Output: Translation result with adjusted tone

[0805] Data processing: Text translation, tone adjustment with emotion engine

[0806] Step 4:

[0807] The user inputs the destination they wish to visit. The device sends the current location and destination information to the server, which then uses the Google Maps API to calculate the optimal route. The resulting route and navigation information are then sent to the device and displayed.

[0808] Input: Destination information, current location information

[0809] Output: Travel route and navigation information

[0810] Data processing: Route calculation, navigation information generation

[0811] Step 5:

[0812] When a user requests an explanation of local rules, the terminal displays the information on the local rules. Furthermore, if the user encounters an emergency, the terminal can use the emergency support function to request assistance from the server. The server provides appropriate advice and assistance information, which is then sent to the terminal for display.

[0813] Input: Request for local rules clarification, emergency support request

[0814] Output: Local rules information, emergency support information

[0815] Data processing: information search and display, generation of supporting information

[0816] Step 6:

[0817] The user opens the link setting screen to link with other domestic applications. The user selects the application they want to link, and the device sends the link setting information to the server. The server uses the link API to share information and establish the link.

[0818] Input: Select the application you want to link

[0819] Output: Established integration settings

[0820] Data processing: Execution of linked API, information sharing settings

[0821] Step 7:

[0822] When a user wants to check product information in a virtual store, the device sends the product information to the server. The server translates the product information using a translation API and recognizes the user's emotional state using an emotion engine. It then provides a product description and recommended products in an appropriate tone and sends them to the device. The device can then display the information and play it back aloud.

[0823] Input: Product information request

[0824] Output: Translated product information, description and suggestions based on emotional state

[0825] Data processing: translating product information, analyzing emotional states, adjusting tone and generating recommendations

[0826] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0827] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0828] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0829] [Second embodiment]

[0830] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0831] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0832] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0833] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0834] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0835] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0836] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0837] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0838] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0839] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0840] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0841] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0842] System Overview

[0843] The present invention provides a smartphone application that helps foreign travelers enjoy a comfortable trip within Japan. This application is a system that includes functions such as account creation for travelers, travel plan suggestions, translation functions, route calculation, explanation of local rules, and linkage with other domestic applications. Details of each function and specific operating procedures are explained below.

[0844] Account creation method

[0845] 1. Initial Setup

[0846] Device: Turn on the power and display the initial setup screen.

[0847] User: Select a language and enter personal information.

[0848] Server: Receives the entered information, creates a traveler account, and stores it in a database.

[0849] Travel plan proposal method

[0850] 1. Travel plan creation

[0851] Device: Display the travel plan creation screen.

[0852] User: Enter the purpose of the trip, duration, and genres of interest.

[0853] Server: Based on this input information, an algorithm is used to generate the optimal travel plan.

[0854] Server: Sends the generated plan to the device and allows the user to view and edit the displayed plan.

[0855] Translation tools

[0856] 1. Real-time translation

[0857] Terminal: Display the translation function screen.

[0858] User: Enter the text you want to translate.

[0859] Server: Translates the input text and sends the results to the device.

[0860] Device: The translation results are displayed on the screen and played aloud if necessary.

[0861] Travel route calculation method

[0862] 1. Route calculation

[0863] Device: Display the destination input screen.

[0864] User: Enter destination.

[0865] Server: Calculates the optimal travel route based on current location information and destination information, and sends the results to the device.

[0866] Terminal: Displays travel route and navigation information on the screen.

[0867] Local Rules Explanation Method

[0868] 1. Provision of local rules

[0869] Terminal: Display the local rules explanation screen.

[0870] Terminal: Provides emergency support when users are in trouble.

[0871] Server: Sends appropriate advice and support information to the terminal in response to an emergency support request.

[0872] Linking with other domestic applications

[0873] 1. App Integration

[0874] Device: Display the domestic application integration settings screen.

[0875] User: Select the application you want to link and set up the link.

[0876] Server: Uses the application integration API to share user information and establish integration.

[0877] Terminal: Notifies the user that the connection is complete and makes the related application available.

[0878] Specific examples

[0879] Specific examples of travel plan proposals

[0880] 1. User operations

[0881] User: You are planning a two-week trip for sightseeing and you are interested in historical sites.

[0882] 2. Server Processing

[0883] Server: Based on this information, it generates a travel plan that includes recommended spots in Tokyo, Kyoto, Osaka, etc.

[0884] Example plans:

[0885] Day 1: Tokyo Tower, Sensoji Temple

[0886] Day 2: Kinkakuji Temple and Fushimi Inari Shrine in Kyoto

[0887] 3. Displaying the terminal

[0888] Terminal: Presents the generated plan to the user and allows the user to customize the plan.

[0889] Examples of translation functions

[0890] 1. User operations

[0891] User: Type "Where is the nearest train station?"

[0892] 2. Server Processing

[0893] Server: Translate this text into Japanese and make it "Where is the nearest station?"

[0894] 3. Displaying the terminal

[0895] Terminal: Helps users ask for directions by displaying translation results and playing them aloud.

[0896] As described above, the system of the present invention provides a multifunctional platform that meets the various needs of travelers and makes their travels more comfortable.

[0897] The processing flow will be explained below.

[0898] Account creation method

[0899] Step 1:

[0900] Terminal: When the application is launched for the first time, the initial setup screen is displayed.

[0901] Step 2:

[0902] User: Select the desired language from the language selection menu that appears.

[0903] Step 3:

[0904] Device: Displays the account creation screen based on the selected language.

[0905] Step 4:

[0906] User: Enter personal information such as name, email address, and password and press the "Register" button.

[0907] Step 5:

[0908] Terminal: Sends the entered information to the server.

[0909] Step 6:

[0910] Server: Creates a user account in the database based on the received information and sends a registration completion status to the terminal.

[0911] Step 7:

[0912] Terminal: Display a registration completion message to the user.

[0913] Travel plan proposal method

[0914] Step 1:

[0915] Device: Display the travel plan creation screen.

[0916] Step 2:

[0917] User: Enter the purpose of the trip (e.g. sightseeing), duration (e.g. 2 weeks), and genre of interest (e.g. historical buildings).

[0918] Step 3:

[0919] Terminal: Sends the entered information to the server.

[0920] Step 4:

[0921] Server: Runs the travel plan suggestion algorithm and generates the optimal travel plan based on the user's input information.

[0922] Step 5:

[0923] Server: Sends the generated travel plan to the terminal.

[0924] Step 6:

[0925] Terminal: Displays the generated travel plan to the user, allowing the user to review and edit the plan.

[0926] Translation tools

[0927] Step 1:

[0928] Terminal: Display the translation function screen.

[0929] Step 2:

[0930] User: Enter the text you want to translate.

[0931] Step 3:

[0932] Terminal: Sends entered text to the server.

[0933] Step 4:

[0934] Server: Uses the translation API to translate the input text into the specified language.

[0935] Step 5:

[0936] Server: Sends the translation results to the device.

[0937] Step 6:

[0938] Device: The translation results are displayed on the screen and played aloud if necessary.

[0939] Travel route calculation method

[0940] Step 1:

[0941] Device: Display the destination input screen.

[0942] Step 2:

[0943] User: Enter your destination (e.g. Kyoto Station).

[0944] Step 3:

[0945] Terminal: Sends current location information and destination information to the server.

[0946] Step 4:

[0947] Server: Executes the travel route calculation algorithm and calculates the optimal travel route.

[0948] Step 5:

[0949] Server: Sends calculated travel route data and navigation information to the terminal.

[0950] Step 6:

[0951] Terminal: Displays travel route and navigation information on the screen.

[0952] Local Rules Explanation Method

[0953] Step 1:

[0954] Terminal: Display the local rules explanation screen.

[0955] Step 2:

[0956] User: Check the local rules, or press the emergency support button if you need emergency support.

[0957] Step 3:

[0958] Terminal: When the emergency support button is pressed, the information is sent to the server.

[0959] Step 4:

[0960] Server: Provides specific advice and support information tailored to the user's situation and sends it to the terminal.

[0961] Step 5:

[0962] Terminal: Displays advice and assistance information received from the server to the user.

[0963] Linking with other domestic applications

[0964] Step 1:

[0965] Device: Display the domestic application integration settings screen.

[0966] Step 2:

[0967] User: Select the application you want to integrate.

[0968] Step 3:

[0969] Terminal: Sends the link setting information of the selected application to the server.

[0970] Step 4:

[0971] Server: Uses the application integration API to share user information and establish integration.

[0972] Step 5:

[0973] Server: Notifies the terminal that the connection is complete.

[0974] Step 6:

[0975] Terminal: Notifies the user that the linking is complete and makes the linked application available.

[0976] The above are the specific processing steps of the present invention.

[0977] Example 1

[0978] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0979] In order for foreign travelers to enjoy a comfortable trip within Japan, they need a wide range of information and support, but they often have difficulty gathering information and communicating due to differences in language and culture.The purpose of this invention is to alleviate difficulties during travel and ensure a comfortable travel experience by providing the information and support travelers need in a centralized manner.

[0980] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0981] In this invention, the server includes a means for creating a traveler's account, a means for inputting the traveler's purpose, period, and areas of interest and proposing an optimal travel plan using a generative AI model, and a means for translating the input text into multiple languages ​​and displaying and playing back the results by voice, thereby enabling travelers to easily complete complex procedures and obtain the necessary information in a timely manner.

[0982] The "means of creating a traveler's account" is the process by which a traveler inputs personal information, and the server uses that information to create and store a new account in the database.

[0983] "A means of having travelers input their purpose, duration, and areas of interest, and then using a generative AI model to suggest the optimal travel plan" refers to the process of having travelers input their purpose, duration, and areas of interest, and then using a generative AI model to suggest the optimal travel plan based on that data.

[0984] "Means for translating input text into multiple languages ​​and displaying and playing the results aloud" refers to a function that translates text input by a traveler into different languages ​​in real time, and not only displays the results on the device, but also plays them aloud if necessary.

[0985] "Means for calculating the optimal travel route based on the traveler's current location information and destination information and displaying the results" refers to the process in which, when a traveler inputs their current location and destination, the server calculates the optimal travel route and displays the results on the terminal.

[0986] "Means to explain local rules of the destination and provide emergency support" is a function that provides travelers with advance information about specific local rules of the destination they will be visiting, and provides appropriate support information and advice in real time in the event of an emergency.

[0987] "Means of linking with other domestic and international software and sharing user information" refers to the process of linking with other domestic and international applications via APIs, etc., and safely sharing user information.

[0988] This invention is software for mobile information terminals that helps foreign travelers to have a comfortable trip within Japan. This software has functions such as creating a travel account for travelers, proposing travel plans, translating, calculating travel routes, explaining local rules, and linking with other domestic and international software.

[0989] Account creation method

[0990] 1. Initial Setup

[0991] Terminal: When a traveler uses the application for the first time, an initial setup screen will be displayed.

[0992] User: Select your language and enter your personal information such as name, email address and password.

[0993] Server: Receives the personal information sent by the user and stores the new user account in the database. The server generates a unique user ID based on this information and returns it to the user.

[0994] Travel plan proposal method

[0995] 1. Create a travel plan

[0996] Terminal: Provides the user with a travel plan creation screen, prompting them to input the purpose of the trip, the duration, and the genres of interest (e.g., history, nature, shopping, etc.).

[0997] User: Enter the purpose of the trip, duration, and genres of interest, and press the send button.

[0998] Server: Receives this information and uses a generative AI model (e.g., GPT-4) to generate an optimal travel plan based on the user's preferences. Specifically, it extracts and combines relevant tourist spot information from the database to generate the optimal plan.

[0999] Server: Sends the generated travel plan to the terminal and displays it to the user.

[1000] Translation tools

[1001] 1. Real-time translation

[1002] Device: Display the translation function screen, and set up a text input field and a display area for the translation results.

[1003] User: Enter the text you want to translate and press the translate button.

[1004] Server: Receives the text sent by the user and translates the text using, for example, the Google Translate API.

[1005] Server: Sends the translation results to the device.

[1006] Terminal: The translation results are displayed on the screen and can also be played aloud if necessary.

[1007] Travel route calculation method

[1008] 1. Route calculation

[1009] Terminal: Provides a screen for entering a destination.

[1010] User: Enter the destination and press the route calculation button.

[1011] Server: Obtains the user's current location and destination information, and calculates the optimal route using, for example, the Google Maps API.

[1012] Server: Sends the calculation results to the terminal.

[1013] Device: Displays calculated route and navigation information on the screen.

[1014] Local Rules Explanation Method

[1015] 1. Provision of local rules

[1016] Terminal: Display the local rules explanation screen.

[1017] Users: Learn about the local rules of the places they visit.

[1018] Terminal: Also displays a screen that provides emergency support when the user is in trouble.

[1019] User: Press the emergency support button when in trouble.

[1020] Server: Receives emergency support requests, generates appropriate advice and assistance information, and sends it.

[1021] Terminal: Displays the support information sent from the server.

[1022] Linking with other domestic and international software

[1023] 1. App Integration

[1024] Device: Display the domestic and international software integration settings screen.

[1025] User: Select the software you want to link and set up the link.

[1026] Server: Uses the app integration API to securely share user information with selected software.

[1027] Terminal: Notifies the user that the connection is complete and displays the related software.

[1028] Specific examples

[1029] Specific examples of travel plan proposals

[1030] 1. User operations

[1031] User: You are planning a two-week trip for sightseeing and you are interested in historical sites.

[1032] 2. Server Processing

[1033] Server: Based on this information, a travel plan is generated that includes recommended spots in Tokyo, Kyoto, Osaka, etc.

[1034] Example plans:

[1035] Day 1: Tokyo Tower, Sensoji Temple

[1036] Day 2: Kinkakuji Temple and Fushimi Inari Shrine in Kyoto

[1037] 3. Displaying the terminal

[1038] Terminal: Presents the generated plan to the user and allows the user to customize the plan.

[1039] Examples of translation functions

[1040] 1. User operations

[1041] User: Type "Where is the nearest train station?"

[1042] 2. Server Processing

[1043] Server: Translate this text into Japanese and make it "Where is the nearest station?"

[1044] 3. Displaying the terminal

[1045] Terminal: Helps users ask for directions by displaying translation results and playing them aloud.

[1046] Prompt Sentence Examples

[1047] "I want to create software that suggests travel plans within Japan. Please explain the steps to generate the optimal travel plan when the user inputs the purpose of the trip, duration, and genres of interest."

[1048] The present invention allows travelers to easily complete complicated procedures and obtain necessary information in real time, making travel more comfortable and fulfilling.

[1049] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1050] Step 1:

[1051] Displaying the initial settings screen

[1052] Terminal: When a traveler launches the application, an initial setup screen appears. The terminal offers the user language selection, name, email address, and password entry fields.

[1053] Input: Personal information such as language, name, email address, and password

[1054] Output: Information entered on the initial setup screen

[1055] Step 2:

[1056] Entering and submitting personal information

[1057] User: The traveler selects a language, enters their name, email address, and password, and presses the send button.

[1058] Input: Personal information such as language, name, email address, and password

[1059] Output: Personal information is sent from the device to the server

[1060] Step 3:

[1061] Creating an account

[1062] Server: Saves a new user account in the database based on the received personal information. The server generates a unique user ID based on this information and returns it to the device.

[1063] Input: Personal information (language, name, email address, password)

[1064] Data processing: User ID generation, information storage in database

[1065] Output: Generated user ID

[1066] Step 4:

[1067] Display of travel plan creation screen

[1068] Terminal: Provides the user with a travel plan creation screen, displaying fields for inputting the purpose of the trip, duration, and genres of interest.

[1069] Input: None (screen input only)

[1070] Output: Travel plan creation screen

[1071] Step 5:

[1072] Enter and submit your travel purpose, duration, and genre

[1073] User: Enter the purpose of the trip, duration, and genres of interest, and press the send button.

[1074] Input: purpose of trip, duration, genre of interest

[1075] Output: Input information is sent from the device to the server

[1076] Step 6:

[1077] Generate a travel plan

[1078] Server: Based on the received travel information, a generative AI model (e.g., GPT-4) is used to generate an optimal travel plan based on the user's preferences. Specifically, it extracts and combines relevant tourist spot information from a database to create the optimal plan.

[1079] Input: purpose of trip, duration, areas of interest

[1080] Data Computing: Using generative AI models to generate travel plans

[1081] Output: Optimal travel plan

[1082] Step 7:

[1083] View and edit your travel plans

[1084] Terminal: Presents the generated travel plan to the user and provides an interface that allows the user to customize the plan.

[1085] Input: Optimal travel plan

[1086] Output: Customizable travel plan screen

[1087] Step 8:

[1088] Display of translation function screen

[1089] Device: Display the translation function screen, and set up a text input field and a display area for the translation results.

[1090] Input: None (screen input only)

[1091] Output: Translation function screen

[1092] Step 9:

[1093] Entering text and submitting translation requests

[1094] User: Enter the text you want to translate and press the send button.

[1095] Input: Text to translate

[1096] Output: The entered text is sent from the terminal to the server.

[1097] Step 10:

[1098] Text translation

[1099] Server: Receives the text sent by the user and translates the text using, for example, the Google Translate API.

[1100] Input: Text to translate

[1101] Data processing: Translate text using the translation API

[1102] Output: The translated text

[1103] Step 11:

[1104] Displaying translation results and playing audio

[1105] Terminal: The translation results are displayed on the screen and can also be played aloud if necessary.

[1106] Input: Translated text

[1107] Output: Display of translation results and audio playback

[1108] Step 12:

[1109] Display destination entry screen

[1110] Device: Displays a screen for entering your destination.

[1111] Input: None (screen input only)

[1112] Output: Destination input screen

[1113] Step 13:

[1114] Entering a destination and sending a route calculation request

[1115] User: Enter the destination and press the route calculation button.

[1116] Input: Destination information

[1117] Output: The entered destination information is sent from the device to the server.

[1118] Step 14:

[1119] Calculating the best route

[1120] Server: Obtains the user's current location and destination information, and calculates the optimal route using, for example, the Google Maps API.

[1121] Input: current location information, destination information

[1122] Data calculation: Calculate the optimal route using Google Maps API

[1123] Output: Optimal travel route

[1124] Step 15:

[1125] View route information

[1126] Device: Displays calculated route and navigation information on the screen.

[1127] Input: Optimal travel route

[1128] Output: Route information display screen

[1129] Step 16:

[1130] Display of local rules explanation screen

[1131] Terminal: Provides a screen explaining the local rules.

[1132] Input: None (screen input only)

[1133] Output: Local rules explanation screen

[1134] Step 17:

[1135] Display of emergency support screen

[1136] Terminal: Displays a screen that provides emergency support functions.

[1137] Input: None (screen input only)

[1138] Output: Emergency support screen

[1139] Step 18:

[1140] Submitting an Emergency Support Request

[1141] User: Press the emergency support button when in trouble.

[1142] Input: Emergency Support Request

[1143] Output: The request is sent to the server

[1144] Step 19:

[1145] Generate and submit support information

[1146] Server: Receives emergency support requests, generates appropriate advice and assistance information, and sends it to the device.

[1147] Input: Emergency Support Request

[1148] Data calculation: Generation of advice and assistance information according to the situation

[1149] Output: Support information

[1150] Step 20:

[1151] View support information

[1152] Terminal: Displays the support information sent from the server.

[1153] Input: Support Information

[1154] Output: Support information display screen

[1155] Step 21:

[1156] Display the app link setting screen

[1157] Device: Display the domestic and international software integration settings screen.

[1158] Input: None (screen input only)

[1159] Output: App link setting screen

[1160] Step 22:

[1161] Selecting and setting up linked apps

[1162] User: Select the software you want to link and set up the link.

[1163] Input: Selected linked application information

[1164] Output: Sending information from the device to the server

[1165] Step 23:

[1166] Sharing of User Information

[1167] Server: Uses the app integration API to securely share user information with selected software.

[1168] Input: Selected linked application information, user information

[1169] Data processing: Share user information using the collaboration API

[1170] Output: Connection completion information

[1171] Step 24:

[1172] Viewing integration notifications and related software

[1173] Terminal: Notifies the user that the connection is complete and displays the related software.

[1174] Input: Connection completion information

[1175] Output: Link notification and related software list

[1176] The above are the specific processing steps of the system. At each step, the user, terminal, and server cooperate to provide services that meet the needs of travelers.

[1177] (Application example 1)

[1178] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1179] Today's travelers need support not only in selecting tourist destinations and planning routes, but also in choosing food and learning dining etiquette based on local culture. Foreign travelers, in particular, are prone to experiencing inconveniences due to language barriers and cultural differences, and often have difficulty obtaining information about and choosing food. This can limit their travel experience and cause significant stress for travelers. Therefore, an integrated solution is needed to help travelers enjoy their meals while understanding local food culture, enabling them to have a comfortable trip.

[1180] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1181] In this invention, the server includes means for creating an account for a traveler, means for proposing a travel plan, means for providing translations in multiple languages, means for calculating a travel route, means for explaining local rules, means for linking with other domestic applications, means for proposing a meal plan based on the traveler's food preferences, means for calculating a travel route to a local restaurant, means for translating restaurant menus and dish descriptions, and means for explaining local rules regarding meals. This enables travelers to enjoy optimal travel plans and dining experiences without experiencing language or cultural barriers, and to have a comfortable and fulfilling trip.

[1182] "Means for creating a traveller's account" refers to a system whereby a traveller enters personal information and food preferences and stores them in a database.

[1183] The "means for suggesting travel plans" is a system that includes an algorithm that generates optimal travel plans based on the traveler's purpose, duration, and areas of interest.

[1184] A "means for providing translation in multiple languages" is a system that translates input text or audio information into multiple languages ​​and displays or plays the results.

[1185] The "means for calculating a travel route" is a system that calculates the optimal route based on the current location and destination and provides navigation information.

[1186] "A means of explaining local rules" is a system that provides travelers with the ability to explain local rules, manners, and precautions.

[1187] "Means for linking with other domestic applications" refers to a system that includes APIs and interfaces for sharing data with other domestic applications and establishing linking functions.

[1188] The "means for proposing meal plans based on a traveler's food preferences" is a system that includes an algorithm that proposes optimal meal plans based on a traveler's food preferences, current location, and regional information about the travel destination.

[1189] The "means for calculating a travel route to a local restaurant" is a system that calculates the optimal route from a traveler's current location to a local restaurant and provides that information.

[1190] The "means for translating restaurant menus and food descriptions" is a system that translates restaurant menus and food descriptions in real time and displays or plays back the results.

[1191] "Means for explaining local rules regarding meals" is a system that includes a function that explains local etiquette and rules that travelers should observe when eating.

[1192] The present invention is a system that provides a smartphone application that provides a series of support required for foreign travelers traveling within Japan. This system includes functions for creating a traveler's account, proposing travel plans, providing translation functions, calculating travel routes, explaining local rules, linking with other domestic applications, proposing meal plans, and providing information about meals.

[1193] System Overview

[1194] Account creation method

[1195] What it is: A way for travelers to enter their personal information and food preferences and store them in a database.

[1196] Example: When a traveler launches the app for the first time, they are prompted to select their language and enter their personal information. Here, they enter their name, email address, and food preferences (e.g., vegetarian, Italian, etc.) and send them to the server, which stores them in a database.

[1197] Travel plan proposal method

[1198] What it is: A way to generate optimal travel plans based on a traveler's purpose, duration, and areas of interest.

[1199] Example: If a traveler plans a two-week sightseeing trip and inputs that they are interested in historical buildings, the server will use this information to generate a travel plan including recommended spots in Tokyo, Kyoto, Osaka, etc. and send it to the device.

[1200] A means of providing translations in multiple languages

[1201] What it is: A means of translating input text or audio information into multiple languages ​​and displaying or playing the results.

[1202] Example: When a traveler types "Where is the nearest train station?", the server translates this into Japanese and displays "Where is the nearest station?" on the device, playing it aloud if necessary.

[1203] Travel route calculation method

[1204] What it is: A means of calculating the optimal route based on your current location and destination, and providing that navigation information.

[1205] Example: If a traveler wants to go to Kinkakuji Temple in Kyoto, the device will prompt the traveler to input their current location and destination, and the server will calculate the optimal travel route and send the results to the device. The traveler will then navigate using the route information displayed on the device.

[1206] Local Rules Explanation Method

[1207] Content: A means of explaining local rules, etiquette, and precautions to travelers.

[1208] Example: If a traveler encounters a problem on a train and uses the emergency support function, the device will display local rules (e.g., refrain from making calls on the train) and provide support information.

[1209] A means of linking with other domestic applications

[1210] What it is: A means to share data and establish federation capabilities with other domestic applications.

[1211] Example: If a traveler wants to link a food delivery app, the device will display a link setting screen, and when the traveler selects linking, the server will share data and establish the link.

[1212] Application example: Meal plan suggestions

[1213] Meal plan proposal method

[1214] What it is: A way to suggest optimal meal plans based on traveler's food preferences, current location, and local information about the destination.

[1215] Example: If a traveler inputs that they prefer "Italian food" and "vegetarian," the server searches for Italian vegetarian restaurants, generates the optimal meal plan, and sends it to the device.

[1216] A way to calculate travel routes to local restaurants

[1217] What it is: A way to calculate the best route from a traveler's current location to a local restaurant and display the results.

[1218] Example: A traveler calculates a route from their current location to the nearest Italian vegetarian restaurant, and the server sends the route information to the device. The traveler then uses the displayed route information to navigate to the restaurant.

[1219] A way to translate restaurant menus and food descriptions

[1220] What it is: A means of translating restaurant menus and food descriptions in real time and displaying or playing them aloud on a device.

[1221] Example: A traveler takes a photo of a restaurant menu and enters it into the app. The server translates the menu and displays "vegetarian pizza" and plays it aloud if needed.

[1222] A means of explaining local dietary rules

[1223] What it is: A way to explain to travelers the local etiquette and rules they should follow when dining out.

[1224] Example: If a traveler gets into trouble while eating, the app will display local rules such as "don't talk loudly while eating" and provide emergency support information.

[1225] Prompt Sentence Examples

[1226] User's name is John Doe, prefers Italian cuisine and is a vegetarian. Recommend the best Italian vegetarian restaurants nearby.

[1227] As a result, the system of the present invention allows travelers to enjoy optimal travel plans and dining experiences, and to have a comfortable and fulfilling trip without having to worry about language or cultural barriers.

[1228] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1229] Step 1:

[1230] The user launches the application for the first time and selects language preference and personal information, including name, email address, and food preferences (e.g., Italian, vegetarian).

[1231] Input: Name, email address, food preferences

[1232] Output: Personal information and food preferences are stored in a database

[1233] Specific operation: The terminal receives input data from the user and sends it to the server, which then stores the received information in a database.

[1234] Step 2:

[1235] The user moves to the travel plan proposal screen and inputs the purpose of the trip, the duration, and areas of interest.

[1236] Input: purpose of trip, duration, areas of interest

[1237] Output: Travel plan proposal data

[1238] How it works: The device receives user input and sends it to the server. The server uses the generative AI model to generate an optimal travel plan and sends it to the device.

[1239] Step 3:

[1240] The user uses the translation feature within the application to translate the text they entered.

[1241] Input:Text

[1242] Output: Translation result

[1243] Specific operation: The device sends the text entered by the user to the server. The server uses a translation algorithm to translate the entered text into the target language and sends the translation result to the device. The device displays the result and plays it aloud if necessary.

[1244] Step 4:

[1245] The user inputs a destination within the application and calculates a route.

[1246] Input:Destination

[1247] Output: Optimal travel route

[1248] Specific operation: The device sends current location information and destination information entered by the user to the server. The server uses a geographic information system (GIS) to calculate the optimal travel route and sends the results to the device. The device then displays the route information.

[1249] Step 5:

[1250] The user is taken to a meal plan suggestion screen where suggestions are given based on their food preferences and current location.

[1251] Input: Food preferences, current location

[1252] Output: Optimal meal plan

[1253] How it works: The device sends the user's dining preferences and current location information to the server. The server then searches a database of restaurants and generates an optimal meal plan, which it then sends to the device. The device then displays the results.

[1254] Step 6:

[1255] A user translates a restaurant menu and checks the dish descriptions.

[1256] Input: Menu text or image

[1257] Output: Translated menu and dish descriptions

[1258] Specific operation: The device sends the menu text or image data from the user to the server. The server uses OCR (optical character recognition) and a translation algorithm to translate the menu and send the result to the device. The device then displays or plays the translated menu aloud.

[1259] Step 7:

[1260] Users can check local rules while eating and get any assistance they need.

[1261] Input: Request for local rules

[1262] Output: Local rule explanation and supporting information

[1263] Specific operation: The terminal sends a request for a local rule from the user to the server. The server searches the database for the relevant local rule information and sends the result to the terminal. The terminal displays an explanation of the local rule and provides the user with the necessary information.

[1264] In this way, the system provides multifunctional support to help travelers enjoy their trips comfortably.

[1265] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1266] System Overview

[1267] This invention provides a smartphone application that helps foreign travelers enjoy a comfortable trip within Japan. This application is a system that combines account creation for travelers, travel plan suggestions, translation functions, route calculations, explanations of local rules, and integration with other domestic applications, as well as an emotion engine that recognizes the user's emotions. Details of each function and specific operating procedures are explained below.

[1268] Account creation method

[1269] 1. Initial Setup

[1270] Device: Turn on the power and display the initial setup screen.

[1271] User: Select a language and enter personal information.

[1272] Server: Creates a traveler account based on the entered information and stores it in a database.

[1273] Travel plan proposal method

[1274] 1. Travel plan creation

[1275] Device: Display the travel plan creation screen.

[1276] User: Enter the purpose of the trip, duration, and genres of interest.

[1277] Server: Based on this input information, the generation algorithm is executed to generate the optimal travel plan.

[1278] Server: Sends the generated plan to the device and allows the user to view and edit the displayed plan.

[1279] Translation tools

[1280] 1. Real-time translation

[1281] Terminal: Display the translation function screen.

[1282] User: Enter the text you want to translate.

[1283] Terminal: Sends entered text to the server.

[1284] Server: Uses the translation API to translate the input text into the specified language.

[1285] Server: Sends the translation results to the device.

[1286] Device: The translation results are displayed on the screen and played aloud if necessary.

[1287] Travel route calculation method

[1288] 1. Route calculation

[1289] Device: Display the destination input screen.

[1290] User: Enter destination.

[1291] Terminal: Sends current location information and destination information to the server.

[1292] Server: Executes the travel route calculation algorithm and calculates the optimal travel route.

[1293] Server: Sends calculated travel route data and navigation information to the terminal.

[1294] Terminal: Displays travel route and navigation information on the screen.

[1295] Local Rules Explanation Method

[1296] 1. Provision of local rules

[1297] Terminal: Display the local rules explanation screen.

[1298] Terminal: Provides emergency support when users are in trouble.

[1299] Server: Sends appropriate advice and support information to the terminal in response to an emergency support request.

[1300] Linking with other domestic applications

[1301] 1. App Integration

[1302] Device: Display the domestic application integration settings screen.

[1303] User: Select the application you want to integrate.

[1304] Terminal: Sends the link setting information of the selected application to the server.

[1305] Server: Uses the application integration API to share user information and establish integration.

[1306] Terminal: Notifies the user that the connection is complete and makes the related application available.

[1307] Emotion engine integration

[1308] The present invention is combined with an emotion engine that recognizes the user's emotions and analyzes user input and sensor data to determine the user's emotional state.

[1309] Using an emotion engine to suggest travel plans

[1310] The emotion engine takes into account the user's emotional state when suggesting travel plans, resulting in more personalized suggestions.

[1311] for example:

[1312] 1. User: When entering a travel plan, if the emotion engine determines that the user is "tired," it will suggest a plan that includes many relaxing spots.

[1313] 2. User: If the emotion engine determines that the user is "excited" when entering travel plans, it will suggest plans that include many active activities.

[1314] Using emotion engine in translation function

[1315] The emotion engine is also used in the translation function, providing translations in an appropriate tone according to the user's emotional state.

[1316] for example:

[1317] 1. User: If the emotion engine determines that the user is "lost," provide a translation in a reassuring tone.

[1318] 2. User: If the emotion engine determines that the user is grateful, it will provide a translation in a polite tone.

[1319] Using the emotion engine to explain local rules

[1320] By using the emotion engine in explaining local rules, it becomes possible to provide explanations that correspond to the user's emotional state.

[1321] for example:

[1322] 1. User: If the emotion engine determines that the user is "nervous," it will provide a more detailed and polite explanation.

[1323] 2. User: If the emotion engine determines that the user is "relaxed," provide a brief, to-the-point explanation.

[1324] Specific examples

[1325] A concrete example of travel plan suggestions using an emotion engine

[1326] 1. User operations

[1327] User: You are planning a two-week trip for sightseeing and you are interested in historical sites.

[1328] Emotion engine: Determines that the user is tired.

[1329] 2. Server Processing

[1330] Server: Based on the judgment of the emotion engine, generate a travel plan that includes many relaxing spots.

[1331] Example plans:

[1332] Day 1: Tokyo Tower, Sensoji Temple

[1333] Day 2: Kyoto's Kinkakuji Temple, a leisurely stroll in Arashiyama, and more

[1334] 3. Displaying the terminal

[1335] Terminal: Presents the generated plan to the user and allows the user to customize the plan.

[1336] A concrete example of a translation function using an emotion engine

[1337] 1. User operations

[1338] User: Type "I'm lost."

[1339] Emotion engine: Determines if the user is feeling anxious.

[1340] 2. Server Processing

[1341] Server: Uses the translation API to generate a translation in a tone that calms anxiety.

[1342] Translation result: "Don't worry. The nearest station is very close."

[1343] 3. Displaying the terminal

[1344] On the device: Displays the translation results and plays them aloud to give the user peace of mind.

[1345] The above is an embodiment of the invention that combines specific processing steps of the present invention and an emotion engine.

[1346] The processing flow will be explained below.

[1347] Specific processing steps for travel plan proposals using an emotion engine

[1348] Step 1:

[1349] Device: Display the travel plan creation screen.

[1350] Step 2:

[1351] User: Enter the purpose of the trip (e.g. sightseeing), duration (e.g. 2 weeks), and genre of interest (e.g. historical buildings).

[1352] Step 3:

[1353] Terminal: Sends the user's facial expressions and voice data to the emotion engine along with the user's input.

[1354] Step 4:

[1355] Emotion engine: Analyzes the user's facial expressions and voice data to determine the user's emotional state (e.g., tired, excited, etc.).

[1356] Step 5:

[1357] Emotion engine: Sends the judgment result of the emotional state to the server.

[1358] Step 6:

[1359] Server: Based on the user's input information and the judgment results of their emotional state, the server runs a travel plan suggestion algorithm and generates an optimal travel plan.

[1360] Example: If the user is determined to be tired, generate a plan that includes many relaxation spots.

[1361] Step 7:

[1362] Server: Sends the generated travel plan to the terminal.

[1363] Step 8:

[1364] Terminal: Displays the generated travel plan to the user, allowing the user to review and edit the plan.

[1365] Specific processing steps for translation function using emotion engine

[1366] Step 1:

[1367] Terminal: Display the translation function screen.

[1368] Step 2:

[1369] User: Enters the text they want to translate (e.g., "Where is the nearest train station?").

[1370] Step 3:

[1371] Terminal: Sends the user's facial expressions and voice data to the emotion engine along with the user's input.

[1372] Step 4:

[1373] Emotion engine: Analyzes the user's facial expressions and voice data to determine the user's emotional state (e.g., anxiety, joy, etc.).

[1374] Step 5:

[1375] Emotion engine: Sends the judgment result of the emotional state to the server.

[1376] Step 6:

[1377] Server: Based on the user's input text and the judged emotional state, the server translates in an appropriate tone.

[1378] Step 7:

[1379] Server: Translates the results (e.g., "Don't worry. The nearest station is nearby.") to the device.

[1380] Step 8:

[1381] Device: The translation results are displayed on the screen and played aloud if necessary.

[1382] Specific processing steps for explaining local rules using the emotion engine

[1383] Step 1:

[1384] Terminal: Display the local rules explanation screen.

[1385] Step 2:

[1386] User: Enter your problem or question, or press the emergency support button.

[1387] Step 3:

[1388] Terminal: Sends the user's facial expressions and voice data to the emotion engine along with the user's input.

[1389] Step 4:

[1390] Emotion engine: Analyzes the user's facial expressions and voice data to determine the user's emotional state (e.g., tension, relief, etc.).

[1391] Step 5:

[1392] Emotion engine: Sends the judgment result of the emotional state to the server.

[1393] Step 6:

[1394] Server: Based on the user's situation and emotional state, the server generates appropriate explanations of local rules and emergency support advice.

[1395] For example, if the user is perceived as nervous, provide more detailed and polite explanations.

[1396] Step 7:

[1397] Server: Sends the generated advice and support information to the terminal.

[1398] Step 8:

[1399] Terminal: Displays advice and support information received from the server to the user, giving them a sense of security.

[1400] The above are the specific processing steps of the present invention.

[1401] Example 2

[1402] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1403] For foreign tourists to travel comfortably within Japan, there are a wide range of issues to overcome, including language barriers, transportation methods, local rules, etc. In addition, it is also necessary to provide appropriate plans and support based on the emotional state of each individual traveler, which presents many challenges that cannot be addressed by existing travel support systems.

[1404] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1405] In this invention, the server includes means for creating a traveler's account, means for proposing a travel plan, means for providing translations in multiple languages, means for calculating travel routes, means for explaining local rules, means for linking with other domestic applications, and means for integrating an emotion engine that analyzes the user's emotional state. This makes it possible to provide optimal plans and support according to the emotions of travelers so that foreign travelers can travel comfortably within Japan.

[1406] "Means for creating a traveler's account" is a general term for devices and software that allow a traveler to enter personal information and store that information in a database.

[1407] "Means for proposing travel plans" is a general term for devices and software that use a generation algorithm to generate optimal travel plans based on inputs from travelers regarding their purpose, duration, and areas of interest, and then display them so that users can customize them.

[1408] "Means for providing translation in multiple languages" is a general term for devices and software that have the function of translating text entered by a user into a specified language and displaying the translation result on a screen or playing it aloud.

[1409] "Means for calculating a travel route" is a general term for devices and software that calculate the optimal travel route based on the destination and current location information entered by the user and provide navigation information.

[1410] "Means for explaining local rules" is a general term for devices and software that provide users with information about local culture, customs, and manners.

[1411] "Means for linking with other domestic applications" is a general term for devices and software for sharing information and establishing linkages with other domestic applications used by travelers.

[1412] An "emotion engine" is a general term for devices and software that analyze user input and sensor data to determine an emotional state and then respond or make appropriate suggestions accordingly.

[1413] MODE FOR CARRYING OUT THE INVENTION

[1414] The present invention provides an information processing device that helps foreign travelers to have a comfortable trip within Japan. This device is a system that includes functions such as creating a travel account for the traveler, proposing travel plans, translating into multiple languages, calculating travel routes, explaining local rules, linking with other domestic applications, and integrating an emotion engine.

[1415] Account creation method

[1416] Device: Turn on the power, launch the application, and display the initial setup screen. The user selects a language and enters personal information such as name, email address, and password. The device temporarily stores this information and sends it to the server.

[1417] Server: Stores the received information in a database and generates a confirmation message for the account creation. The device displays a notification to the user that the account was created successfully and provides the option to proceed to the next step.

[1418] Travel plan proposal method

[1419] Terminal: Displays a travel plan creation screen and provides input fields for the purpose of the trip, duration, and genres of interest. The user enters this information, and the terminal sends it to the server.

[1420] Server: Based on the received information, the server uses a generative AI model to generate an optimal travel plan. The generated plan is sent to the device, where the user can view and edit it.

[1421] Translation tools

[1422] Terminal: Displays the translation function screen and provides the user with a text input field. The user enters the text to be translated and selects the language to specify. The terminal sends the entered text and the selected language to the server.

[1423] Server: Calls the translation API to translate the input text into the specified language. The translation result is sent to the device, which displays it on the screen and, if necessary, plays it aloud.

[1424] Travel route calculation method

[1425] Device: Displays a destination entry screen and gives the user the option to enter their current location and destination. The user can enter their current location and destination, or use the GPS function to automatically obtain their current location. The device then sends this information to the server.

[1426] Server: Executes the travel route calculation algorithm to calculate the optimal travel route. Sends the calculated route and navigation information to the device, which displays it.

[1427] Local Rules Explanation Method

[1428] Terminal: Displays a local rules explanation screen and provides information on local culture, customs, and manners. If a user is in trouble, they can press the emergency support button to request assistance. The terminal then sends the request to the server.

[1429] Server: Generates appropriate advice and support information based on the request and sends it to the terminal, which then displays it to the user.

[1430] Linking with other domestic applications

[1431] Terminal: The domestic application linkage setting screen is displayed, along with a list of applications to be linked. The user selects the application they want to link, and the terminal sends the setting information to the server.

[1432] Server: Calls the application integration API, shares user information with related applications, and establishes integration. The server sends a notification of integration completion to the device, and the device notifies the user.

[1433] Emotion engine integration

[1434] The system incorporates an emotion engine that analyzes user input and sensor data to determine the user's emotional state, and based on this, it makes suggestions and offers the following:

[1435] Using an emotion engine to suggest travel plans

[1436] When a user inputs a travel plan, the emotion engine analyzes the user's state. For example, if it determines that the user is tired, it will generate a plan that includes many relaxing spots. On the other hand, if it determines that the user is excited, it will suggest a plan that includes many active activities.

[1437] Example prompt sentence:

[1438] "I want to relax, so please tell me about a travel plan that includes places where I can relax."

[1439] Using emotion engine in translation function

[1440] The emotion engine also analyzes when a user uses the translation function. For example, if the user is feeling anxious, the system will provide a translation in a reassuring tone.

[1441] Example prompt sentence:

[1442] "I'm lost. Please tell me what to do."

[1443] Using the emotion engine to explain local rules

[1444] When a user requests emergency support, the emotion engine analyzes the user's state and, if it determines that the user is nervous, for example, provides a detailed and thorough explanation.

[1445] This will enable us to provide a comprehensive support system to help foreign travelers enjoy a comfortable trip within Japan.

[1446] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1447] Program processing flow

[1448] Account creation method

[1449] Step 1:

[1450] When the device is turned on, it launches the application and displays the initial setup screen. When the user selects a language, the device switches the interface based on that selection: input (language selected by the user), output (interface display in the selected language).

[1451] Step 2:

[1452] The user enters personal information such as name, email address, password, etc. Input (personal information entered by the user), output (personal information once input is complete).

[1453] Step 3:

[1454] The terminal temporarily stores the input information and sends it to a server via the Internet. Input (user's personal information), Output (sending information to the server).

[1455] Step 4:

[1456] The server stores the received information in a database and generates a confirmation message for creating an account. Data processing (storing personal information), data calculation (generating a confirmation message), output (confirmation message).

[1457] Step 5:

[1458] The terminal displays a notification to the user that the account was created successfully and provides the option to proceed to the next step. Input (confirmation message received from the server), Output (notification displayed to the user).

[1459] Travel plan proposal method

[1460] Step 1:

[1461] The terminal displays a travel plan creation screen and provides input fields for the purpose of the trip, duration, and genres of interest. Input (request for screen display), output (display of input fields).

[1462] Step 2:

[1463] The user inputs the purpose of the trip, the duration, and the genres they are interested in. Input (user's travel information), output (entered travel information).

[1464] Step 3:

[1465] The terminal collects this information and sends it to the server. Input (travel information), Output (sending information to the server).

[1466] Step 4:

[1467] The server uses a generative AI model based on the received information to generate an optimal travel plan, including data processing (analysis of travel information), data calculation (generation of optimal plan), and output (generated travel plan).

[1468] Step 5:

[1469] The server sends the generated itinerary to the terminal. Input (generated itinerary), Output (transmission to terminal).

[1470] Step 6:

[1471] The terminal displays the received travel plan to the user and allows editing and saving. Input (travel plan received from server), Output (providing display and editing options to the user).

[1472] Translation tools

[1473] Step 1:

[1474] The terminal displays a translation function screen and provides the user with a text input field. Input (request for screen display), output (display of input field).

[1475] Step 2:

[1476] The user enters the text to be translated and selects the language to be translated. Input (translation text and language selection), Output (entered text and selected language).

[1477] Step 3:

[1478] The terminal sends the entered text and the selected language to the server. Input (text and language), Output (send to server).

[1479] Step 4:

[1480] The server calls the translation API and translates the input text into the specified language. Data processing (text analysis), data calculation (translation processing), and output (translation results) are performed.

[1481] Step 5:

[1482] The server sends the translation results to the terminal. Input (translation results), output (transmission to terminal).

[1483] Step 6:

[1484] The device displays the translation results on the screen and plays them aloud if necessary. Input (translation results), output (screen display and audio playback).

[1485] Travel route calculation method

[1486] Step 1:

[1487] The device displays a destination entry screen and gives the user the option to enter their current location and destination. Input (request for screen display), Output (display of input field).

[1488] Step 2:

[1489] The user can input their current location and destination, or use the GPS function to automatically obtain their current location. Input (current location and destination, or GPS data), output (input location data).

[1490] Step 3:

[1491] The device sends the acquired current location information and destination information to the server. Input (current location and destination information), output (transmission to server).

[1492] Step 4:

[1493] The server executes the travel route calculation algorithm to calculate the optimal travel route, including data processing (analysis of location information), data calculation (route calculation), and output (calculated route data).

[1494] Step 5:

[1495] The server sends the calculated route and navigation information to the device.,Input (calculated route data), Output (transmission to device).

[1496] Step 6:

[1497] The device displays navigation information and maps on the screen, allowing the user to check the route. Input (route data), output (display of map and information).

[1498] Local Rules Explanation Method

[1499] Step 1:

[1500] The terminal displays a local rules explanation screen and provides information on the culture, customs, and manners of each region. Input (request for screen display), output (display of rules explanation).

[1501] Step 2:

[1502] When a user is in trouble, he / she presses the emergency support button to request assistance. Input (pressing the support request button), Output (support request data).

[1503] Step 3:

[1504] The terminal sends an emergency support request to the server. Input (support request data), Output (transmission to server).

[1505] Step 4:

[1506] The server generates appropriate advice and support information based on the request content and sends it to the terminal. Data processing (analysis of the request content), data calculation (generation of advice), output (generated support information).

[1507] Step 5:

[1508] The terminal displays the received advice and support information to the user. Input (received support information), output (display to user).

[1509] Linking with other domestic applications

[1510] Step 1:

[1511] The terminal displays the domestic application linkage setting screen and displays a list of applications to be linked. Input (request for screen display), output (display of setting screen).

[1512] Step 2:

[1513] The user selects the application they want to link. Input (selected application), output (selected information).

[1514] Step 3:

[1515] The terminal sends the link setting information of the selected application to the server. Input (link setting information), output (transmission to server).

[1516] Step 4:

[1517] The server calls the application integration API, shares user information with related applications, and establishes integration. Data processing (sharing user information), data calculation (establishing integration), and output (notifying that integration is complete) are performed.

[1518] Step 5:

[1519] The server sends a notification of completion of collaboration to the terminal. Input (notification of completion of collaboration), output (send to terminal).

[1520] Step 6:

[1521] The device notifies the user that the connection is complete, and synchronization with related applications becomes possible. Input (connection completion notification), Output (notification to the user).

[1522] Emotion engine integration

[1523] Step 1:

[1524] The server analyzes the user's input and sensor data to determine the emotional state. Input (user's input and sensor data), Output (determined emotional state).

[1525] Step 2:

[1526] When a user inputs their travel plan, the emotion engine analyzes the user's state: input (user's travel information), output (determining their emotional state).

[1527] Step 3:

[1528] The server generates a travel plan based on the user's emotional state. For example, if the server determines that the user is tired, it generates a plan that includes many relaxation spots. Data processing (analysis of emotional state), data calculation (plan generation), output (generated plan).

[1529] Step 4:

[1530] The server uses an emotion engine to translate in a reassuring tone if the user is feeling anxious. Data processing (analysis of emotional state), data calculation (translation processing), output (translation results).

[1531] Step 5:

[1532] The server analyzes the user's emotional state during emergency support and provides detailed and thorough explanations. Data processing (analysis of emotional state), data calculation (generation of support information), output (support information).

[1533] (Application example 2)

[1534] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1535] Previous travel support applications lacked comprehensive support for foreign travelers to have a comfortable stay in Japan, particularly due to language barriers and a lack of understanding of local rules. Furthermore, the guidance and product explanations within the virtual stores travelers visited were limited to providing general information and lacked personalized responses tailored to the individual emotional state of the travelers. This resulted in a decrease in traveler satisfaction.

[1536] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1537] In this invention, the server includes means for creating an account for a traveler, means for proposing a travel plan, means for providing translations in multiple languages, means for calculating travel routes, means for explaining local rules, means for linking with other domestic applications, means for recognizing a user's emotions and providing information in an appropriate tone accordingly, and means for proposing product guides and recommended products in a virtual store. This makes it possible to provide personalized guides and product explanations tailored to the emotional state of a traveler while overcoming language barriers, thereby improving traveler satisfaction.

[1538] "Means for creating a traveler's account" refers to the process by which a traveler inputs personal information and manages that information on the server.

[1539] The "means for proposing a travel plan" is a process for generating and proposing an optimal travel plan based on the traveler's purpose, period, and areas of interest.

[1540] A "means for providing translation in multiple languages" is a process for translating user-entered text into multiple languages ​​and providing the results.

[1541] The "means for calculating a travel route" is a process for calculating and providing guidance on the optimal travel route based on the user's current location and destination.

[1542] "Means of explaining local rules" refers to the process of explaining the rules and manners of each region in Japan to foreign tourists.

[1543] "Means of linking with other domestic applications" refers to the process by which travelers link with other domestic applications when using those applications.

[1544] "Means for recognizing the user's emotions and providing information in an appropriate tone" is a process for analyzing and determining the user's emotional state and providing information in the most appropriate tone based on that.

[1545] "Means for providing product information and suggesting recommended products within a virtual store" refers to the process of providing product information to users within a virtual store and suggesting recommended products based on the user's interests and emotions.

[1546] MODE FOR CARRYING OUT THE INVENTION

[1547] This invention provides a mobile terminal application system to support foreign travelers in enjoying a comfortable trip within Japan. This system can create a traveler's account, suggest travel plans, provide translations in multiple languages, calculate travel routes, explain local rules, link with other domestic applications, and even provide information in an appropriate tone according to the user's emotional state.

[1548] Hardware and software used

[1549] Hardware: Smartphones, smart glasses, head-mounted displays

[1550] Software: Python, TensorFlow (emotion engine), Google Translate API (translation), Google Maps API (route calculation)

[1551] Embodiments of each means of the system

[1552] 1. How to create a traveller account

[1553] The user enters personal information and sends it to the server, which stores it in a database and creates an account for the traveler.

[1554] 2. A way to suggest travel plans

[1555] The user inputs the purpose, duration, and areas of interest of the trip. The server runs a generation algorithm based on this input information to generate an optimal travel plan. In doing so, it uses an emotion engine to analyze the user's emotional state. For example, if the server determines that the user is "tired," it will suggest a plan that includes many relaxation spots.

[1556] 3. A way to provide translations in multiple languages

[1557] The user enters the text they want to translate. The device sends this text to the server. The server uses the Google Translate API to translate it and sends the results back to the device. The translation is delivered in an appropriate tone based on the emotion engine.

[1558] 4. Means of calculating travel routes

[1559] The user inputs a destination. The device sends the current location and destination information to the server. The server uses the Google Maps API to calculate the optimal route and provide navigation information.

[1560] 5. Means of Explaining Local Rules

[1561] By storing local rule information on the device, the information can be displayed when the user is in trouble. In addition, the server provides appropriate advice and support information in response to emergency support requests from the device.

[1562] 6. Means of linking with other domestic applications

[1563] On the device, select the application you want to link and send the link setting information to the server. The server then uses the link API to share information between the applications and establish the link.

[1564] 7. A way to recognize the user's emotions and deliver information in an appropriate tone.

[1565] The camera feed and audio data are fed into a TensorFlow model to analyze the user's emotional state. For example, if the system detects that the user is feeling anxious, it will provide information in a tone that will calm the user's anxiety.

[1566] 8. A means of product guidance and recommendations within a virtual store

[1567] When a user views product information in a virtual store, the system translates and displays guidance information. It recognizes the user's emotional state and provides recommendations and guidance accordingly.

[1568] Specific examples

[1569] Example 1: Travel plan suggestions

[1570] 1. A user plans a two-week sightseeing trip and indicates that they are interested in historical sites.

[1571] 2. The emotion engine determines that the user is tired.

[1572] 3. The server generates an itinerary that includes many relaxing spots.

[1573] 4. Display the generated plan on the device and allow the user to customize it.

[1574] Example 2: Applying the translation function

[1575] 1. The user types "I'm lost."

[1576] 2. The emotion engine determines that the user is feeling anxious.

[1577] 3. The server generates a translation in a calming tone.

[1578] 4. The device will display the message "Don't worry, the nearest station is right nearby" and play it aloud.

[1579] Example 3: Guidance in a virtual store

[1580] 1. A user views products in a virtual store.

[1581] 2. The emotion engine determines the user's emotional state and recognizes that they are feeling anxious.

[1582] 3. The server provides product descriptions in a reassuring translation tone.

[1583] 4. The device will display and play a message saying, "This product is a high-quality item that you can use with confidence."

[1584] Prompt Sentence Examples

[1585] If users are feeling unsure, provide a reassuring tone in your product description. Translate the product information below to create a reassuring message:

[1586] Product Name: High-quality bags

[1587] Description: This bag is durable and can be used for a long time. It is made of high-quality materials and is lightweight and easy to carry.

[1588] Example output: "This bag is of very high quality and will last a long time. You can use it with confidence."

[1589] The above is an embodiment of the present invention.

[1590] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1591] Step 1:

[1592] The user launches the smartphone application, selects a language, and enters their personal information. The device sends this information to the server, which then creates a traveler account based on the information entered and stores it in a database.

[1593] Input: Language selection, personal information

[1594] Output: Traveler account created and saved to database

[1595] Data processing: Convert input information into database format and save

[1596] Step 2:

[1597] The user opens the travel plan suggestion screen within the application and inputs the purpose of the trip, duration, and genres of interest. The device sends this input information to the server, which then runs a generation algorithm to generate the optimal travel plan. In doing so, the server uses an emotion engine to determine the user's emotional state and personalize the plan accordingly.

[1598] Input: purpose of trip, duration, genre of interest

[1599] Output: A personalized itinerary

[1600] Data processing: Input information into a generative algorithm, and an emotion engine determines the emotional state and optimizes the plan.

[1601] Step 3:

[1602] The user uses the translation function and inputs the text they want to translate. The device sends this text to the server, which translates it using the Google Translate API. Once the translation result is generated, the server adjusts the tone using an emotion engine and sends it to the device for display.

[1603] Input: The text you want to translate

[1604] Output: Translation result with adjusted tone

[1605] Data processing: Text translation, tone adjustment with emotion engine

[1606] Step 4:

[1607] The user inputs the destination they wish to visit. The device sends the current location and destination information to the server, which then uses the Google Maps API to calculate the optimal route. The resulting route and navigation information are then sent to the device and displayed.

[1608] Input: Destination information, current location information

[1609] Output: Travel route and navigation information

[1610] Data processing: Route calculation, navigation information generation

[1611] Step 5:

[1612] When a user requests an explanation of local rules, the terminal displays the information on the local rules. Furthermore, if the user encounters an emergency, the terminal can use the emergency support function to request assistance from the server. The server provides appropriate advice and assistance information, which is then sent to the terminal for display.

[1613] Input: Request for local rules clarification, emergency support request

[1614] Output: Local rules information, emergency support information

[1615] Data processing: information search and display, generation of supporting information

[1616] Step 6:

[1617] The user opens the link setting screen to link with other domestic applications. The user selects the application they want to link, and the device sends the link setting information to the server. The server uses the link API to share information and establish the link.

[1618] Input: Select the application you want to link

[1619] Output: Established integration settings

[1620] Data processing: Execution of linked API, information sharing settings

[1621] Step 7:

[1622] When a user wants to check product information in a virtual store, the device sends the product information to the server. The server translates the product information using a translation API and recognizes the user's emotional state using an emotion engine. It then provides a product description and recommended products in an appropriate tone and sends them to the device. The device can then display the information and play it back aloud.

[1623] Input: Product information request

[1624] Output: Translated product information, description and suggestions based on emotional state

[1625] Data processing: translating product information, analyzing emotional states, adjusting tone and generating recommendations

[1626] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1627] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1628] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1629] [Third embodiment]

[1630] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1631] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1632] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1633] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1634] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1635] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1636] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1637] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1638] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1639] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1640] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1641] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1642] System Overview

[1643] The present invention provides a smartphone application that helps foreign travelers enjoy a comfortable trip within Japan. This application is a system that includes functions such as account creation for travelers, travel plan suggestions, translation functions, route calculation, explanation of local rules, and linkage with other domestic applications. Details of each function and specific operating procedures are explained below.

[1644] Account creation method

[1645] 1. Initial Setup

[1646] Device: Turn on the power and display the initial setup screen.

[1647] User: Select a language and enter personal information.

[1648] Server: Receives the entered information, creates a traveler account, and stores it in a database.

[1649] Travel plan proposal method

[1650] 1. Travel plan creation

[1651] Device: Display the travel plan creation screen.

[1652] User: Enter the purpose of the trip, duration, and genres of interest.

[1653] Server: Based on this input information, an algorithm is used to generate the optimal travel plan.

[1654] Server: Sends the generated plan to the device and allows the user to view and edit the displayed plan.

[1655] Translation tools

[1656] 1. Real-time translation

[1657] Terminal: Display the translation function screen.

[1658] User: Enter the text you want to translate.

[1659] Server: Translates the input text and sends the results to the device.

[1660] Device: The translation results are displayed on the screen and played aloud if necessary.

[1661] Travel route calculation method

[1662] 1. Route calculation

[1663] Device: Display the destination input screen.

[1664] User: Enter destination.

[1665] Server: Calculates the optimal travel route based on current location information and destination information, and sends the results to the device.

[1666] Terminal: Displays travel route and navigation information on the screen.

[1667] Local Rules Explanation Method

[1668] 1. Provision of local rules

[1669] Terminal: Display the local rules explanation screen.

[1670] Terminal: Provides emergency support when users are in trouble.

[1671] Server: Sends appropriate advice and support information to the terminal in response to an emergency support request.

[1672] Linking with other domestic applications

[1673] 1. App Integration

[1674] Device: Display the domestic application integration settings screen.

[1675] User: Select the application you want to link and set up the link.

[1676] Server: Uses the application integration API to share user information and establish integration.

[1677] Terminal: Notifies the user that the connection is complete and makes the related application available.

[1678] Specific examples

[1679] Specific examples of travel plan proposals

[1680] 1. User operations

[1681] User: You are planning a two-week trip for sightseeing and you are interested in historical sites.

[1682] 2. Server Processing

[1683] Server: Based on this information, it generates a travel plan that includes recommended spots in Tokyo, Kyoto, Osaka, etc.

[1684] Example plans:

[1685] Day 1: Tokyo Tower, Sensoji Temple

[1686] Day 2: Kinkakuji Temple and Fushimi Inari Shrine in Kyoto

[1687] 3. Displaying the terminal

[1688] Terminal: Presents the generated plan to the user and allows the user to customize the plan.

[1689] Examples of translation functions

[1690] 1. User operations

[1691] User: Type "Where is the nearest train station?"

[1692] 2. Server Processing

[1693] Server: Translate this text into Japanese and make it "Where is the nearest station?"

[1694] 3. Displaying the terminal

[1695] Terminal: Helps users ask for directions by displaying translation results and playing them aloud.

[1696] As described above, the system of the present invention provides a multifunctional platform that meets the various needs of travelers and makes their travels more comfortable.

[1697] The processing flow will be explained below.

[1698] Account creation method

[1699] Step 1:

[1700] Terminal: When the application is launched for the first time, the initial setup screen is displayed.

[1701] Step 2:

[1702] User: Select the desired language from the language selection menu that appears.

[1703] Step 3:

[1704] Device: Displays the account creation screen based on the selected language.

[1705] Step 4:

[1706] User: Enter personal information such as name, email address, and password and press the "Register" button.

[1707] Step 5:

[1708] Terminal: Sends the entered information to the server.

[1709] Step 6:

[1710] Server: Creates a user account in the database based on the received information and sends a registration completion status to the terminal.

[1711] Step 7:

[1712] Terminal: Display a registration completion message to the user.

[1713] Travel plan proposal method

[1714] Step 1:

[1715] Device: Display the travel plan creation screen.

[1716] Step 2:

[1717] User: Enter the purpose of the trip (e.g. sightseeing), duration (e.g. 2 weeks), and genre of interest (e.g. historical buildings).

[1718] Step 3:

[1719] Terminal: Sends the entered information to the server.

[1720] Step 4:

[1721] Server: Runs the travel plan suggestion algorithm and generates the optimal travel plan based on the user's input information.

[1722] Step 5:

[1723] Server: Sends the generated travel plan to the terminal.

[1724] Step 6:

[1725] Terminal: Displays the generated travel plan to the user, allowing the user to review and edit the plan.

[1726] Translation tools

[1727] Step 1:

[1728] Terminal: Display the translation function screen.

[1729] Step 2:

[1730] User: Enter the text you want to translate.

[1731] Step 3:

[1732] Terminal: Sends entered text to the server.

[1733] Step 4:

[1734] Server: Uses the translation API to translate the input text into the specified language.

[1735] Step 5:

[1736] Server: Sends the translation results to the device.

[1737] Step 6:

[1738] Device: The translation results are displayed on the screen and played aloud if necessary.

[1739] Travel route calculation method

[1740] Step 1:

[1741] Device: Display the destination input screen.

[1742] Step 2:

[1743] User: Enter your destination (e.g. Kyoto Station).

[1744] Step 3:

[1745] Terminal: Sends current location information and destination information to the server.

[1746] Step 4:

[1747] Server: Executes the travel route calculation algorithm and calculates the optimal travel route.

[1748] Step 5:

[1749] Server: Sends calculated travel route data and navigation information to the terminal.

[1750] Step 6:

[1751] Terminal: Displays travel route and navigation information on the screen.

[1752] Local Rules Explanation Method

[1753] Step 1:

[1754] Terminal: Display the local rules explanation screen.

[1755] Step 2:

[1756] User: Check the local rules, or press the emergency support button if you need emergency support.

[1757] Step 3:

[1758] Terminal: When the emergency support button is pressed, the information is sent to the server.

[1759] Step 4:

[1760] Server: Provides specific advice and support information tailored to the user's situation and sends it to the terminal.

[1761] Step 5:

[1762] Terminal: Displays advice and assistance information received from the server to the user.

[1763] Linking with other domestic applications

[1764] Step 1:

[1765] Device: Display the domestic application integration settings screen.

[1766] Step 2:

[1767] User: Select the application you want to integrate.

[1768] Step 3:

[1769] Terminal: Sends the link setting information of the selected application to the server.

[1770] Step 4:

[1771] Server: Uses the application integration API to share user information and establish integration.

[1772] Step 5:

[1773] Server: Notifies the terminal that the connection is complete.

[1774] Step 6:

[1775] Terminal: Notifies the user that the linking is complete and makes the linked application available.

[1776] The above are the specific processing steps of the present invention.

[1777] Example 1

[1778] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1779] In order for foreign travelers to enjoy a comfortable trip within Japan, they need a wide range of information and support, but they often have difficulty gathering information and communicating due to differences in language and culture.The purpose of this invention is to alleviate difficulties during travel and ensure a comfortable travel experience by providing the information and support travelers need in a centralized manner.

[1780] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1781] In this invention, the server includes a means for creating a traveler's account, a means for inputting the traveler's purpose, period, and areas of interest and proposing an optimal travel plan using a generative AI model, and a means for translating the input text into multiple languages ​​and displaying and playing back the results by voice, thereby enabling travelers to easily complete complex procedures and obtain the necessary information in a timely manner.

[1782] The "means of creating a traveler's account" is the process by which a traveler inputs personal information, and the server uses that information to create and store a new account in the database.

[1783] "A means of having travelers input their purpose, duration, and areas of interest, and then using a generative AI model to suggest the optimal travel plan" refers to the process of having travelers input their purpose, duration, and areas of interest, and then using a generative AI model to suggest the optimal travel plan based on that data.

[1784] "Means for translating input text into multiple languages ​​and displaying and playing the results aloud" refers to a function that translates text input by a traveler into different languages ​​in real time, and not only displays the results on the device, but also plays them aloud if necessary.

[1785] "Means for calculating the optimal travel route based on the traveler's current location information and destination information and displaying the results" refers to the process in which, when a traveler inputs their current location and destination, the server calculates the optimal travel route and displays the results on the terminal.

[1786] "Means to explain local rules of the destination and provide emergency support" is a function that provides travelers with advance information about specific local rules of the destination they will be visiting, and provides appropriate support information and advice in real time in the event of an emergency.

[1787] "Means of linking with other domestic and international software and sharing user information" refers to the process of linking with other domestic and international applications via APIs, etc., and safely sharing user information.

[1788] This invention is software for mobile information terminals that helps foreign travelers to have a comfortable trip within Japan. This software has functions such as creating a travel account for travelers, proposing travel plans, translating, calculating travel routes, explaining local rules, and linking with other domestic and international software.

[1789] Account creation method

[1790] 1. Initial Setup

[1791] Terminal: When a traveler uses the application for the first time, an initial setup screen will be displayed.

[1792] User: Select your language and enter your personal information such as name, email address and password.

[1793] Server: Receives the personal information sent by the user and stores the new user account in the database. The server generates a unique user ID based on this information and returns it to the user.

[1794] Travel plan proposal method

[1795] 1. Create a travel plan

[1796] Terminal: Provides the user with a travel plan creation screen, prompting them to input the purpose of the trip, the duration, and the genres of interest (e.g., history, nature, shopping, etc.).

[1797] User: Enter the purpose of the trip, duration, and genres of interest, and press the send button.

[1798] Server: Receives this information and uses a generative AI model (e.g., GPT-4) to generate an optimal travel plan based on the user's preferences. Specifically, it extracts and combines relevant tourist spot information from the database to generate the optimal plan.

[1799] Server: Sends the generated travel plan to the terminal and displays it to the user.

[1800] Translation tools

[1801] 1. Real-time translation

[1802] Device: Display the translation function screen, and set up a text input field and a display area for the translation results.

[1803] User: Enter the text you want to translate and press the translate button.

[1804] Server: Receives the text sent by the user and translates the text using, for example, the Google Translate API.

[1805] Server: Sends the translation results to the device.

[1806] Terminal: The translation results are displayed on the screen and can also be played aloud if necessary.

[1807] Travel route calculation method

[1808] 1. Route calculation

[1809] Terminal: Provides a screen for entering a destination.

[1810] User: Enter the destination and press the route calculation button.

[1811] Server: Obtains the user's current location and destination information, and calculates the optimal route using, for example, the Google Maps API.

[1812] Server: Sends the calculation results to the terminal.

[1813] Device: Displays calculated route and navigation information on the screen.

[1814] Local Rules Explanation Method

[1815] 1. Provision of local rules

[1816] Terminal: Display the local rules explanation screen.

[1817] Users: Learn about the local rules of the places they visit.

[1818] Terminal: Also displays a screen that provides emergency support when the user is in trouble.

[1819] User: Press the emergency support button when in trouble.

[1820] Server: Receives emergency support requests, generates appropriate advice and assistance information, and sends it.

[1821] Terminal: Displays the support information sent from the server.

[1822] Linking with other domestic and international software

[1823] 1. App Integration

[1824] Device: Display the domestic and international software integration settings screen.

[1825] User: Select the software you want to link and set up the link.

[1826] Server: Uses the app integration API to securely share user information with selected software.

[1827] Terminal: Notifies the user that the connection is complete and displays the related software.

[1828] Specific examples

[1829] Specific examples of travel plan proposals

[1830] 1. User operations

[1831] User: You are planning a two-week trip for sightseeing and you are interested in historical sites.

[1832] 2. Server Processing

[1833] Server: Based on this information, a travel plan is generated that includes recommended spots in Tokyo, Kyoto, Osaka, etc.

[1834] Example plans:

[1835] Day 1: Tokyo Tower, Sensoji Temple

[1836] Day 2: Kinkakuji Temple and Fushimi Inari Shrine in Kyoto

[1837] 3. Displaying the terminal

[1838] Terminal: Presents the generated plan to the user and allows the user to customize the plan.

[1839] Examples of translation functions

[1840] 1. User operations

[1841] User: Type "Where is the nearest train station?"

[1842] 2. Server Processing

[1843] Server: Translate this text into Japanese and make it "Where is the nearest station?"

[1844] 3. Displaying the terminal

[1845] Terminal: Helps users ask for directions by displaying translation results and playing them aloud.

[1846] Prompt Sentence Examples

[1847] "I want to create software that suggests travel plans within Japan. Please explain the steps to generate the optimal travel plan when the user inputs the purpose of the trip, duration, and genres of interest."

[1848] The present invention allows travelers to easily complete complicated procedures and obtain necessary information in real time, making travel more comfortable and fulfilling.

[1849] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1850] Step 1:

[1851] Displaying the initial settings screen

[1852] Terminal: When a traveler launches the application, an initial setup screen appears. The terminal offers the user language selection, name, email address, and password entry fields.

[1853] Input: Personal information such as language, name, email address, and password

[1854] Output: Information entered on the initial setup screen

[1855] Step 2:

[1856] Entering and submitting personal information

[1857] User: The traveler selects a language, enters their name, email address, and password, and presses the send button.

[1858] Input: Personal information such as language, name, email address, and password

[1859] Output: Personal information is sent from the device to the server

[1860] Step 3:

[1861] Creating an account

[1862] Server: Saves a new user account in the database based on the received personal information. The server generates a unique user ID based on this information and returns it to the device.

[1863] Input: Personal information (language, name, email address, password)

[1864] Data processing: User ID generation, information storage in database

[1865] Output: Generated user ID

[1866] Step 4:

[1867] Display of travel plan creation screen

[1868] Terminal: Provides the user with a travel plan creation screen, displaying fields for inputting the purpose of the trip, duration, and genres of interest.

[1869] Input: None (screen input only)

[1870] Output: Travel plan creation screen

[1871] Step 5:

[1872] Enter and submit your travel purpose, duration, and genre

[1873] User: Enter the purpose of the trip, duration, and genres of interest, and press the send button.

[1874] Input: purpose of trip, duration, genre of interest

[1875] Output: Input information is sent from the device to the server

[1876] Step 6:

[1877] Generate a travel plan

[1878] Server: Based on the received travel information, a generative AI model (e.g., GPT-4) is used to generate an optimal travel plan based on the user's preferences. Specifically, it extracts and combines relevant tourist spot information from a database to create the optimal plan.

[1879] Input: purpose of trip, duration, areas of interest

[1880] Data Computing: Using generative AI models to generate travel plans

[1881] Output: Optimal travel plan

[1882] Step 7:

[1883] View and edit your travel plans

[1884] Terminal: Presents the generated travel plan to the user and provides an interface that allows the user to customize the plan.

[1885] Input: Optimal travel plan

[1886] Output: Customizable travel plan screen

[1887] Step 8:

[1888] Display of translation function screen

[1889] Device: Display the translation function screen, and set up a text input field and a display area for the translation results.

[1890] Input: None (screen input only)

[1891] Output: Translation function screen

[1892] Step 9:

[1893] Entering text and submitting translation requests

[1894] User: Enter the text you want to translate and press the send button.

[1895] Input: Text to translate

[1896] Output: The entered text is sent from the terminal to the server.

[1897] Step 10:

[1898] Text translation

[1899] Server: Receives the text sent by the user and translates the text using, for example, the Google Translate API.

[1900] Input: Text to translate

[1901] Data processing: Translate text using the translation API

[1902] Output: The translated text

[1903] Step 11:

[1904] Displaying translation results and playing audio

[1905] Terminal: The translation results are displayed on the screen and can also be played aloud if necessary.

[1906] Input: Translated text

[1907] Output: Display of translation results and audio playback

[1908] Step 12:

[1909] Display destination entry screen

[1910] Device: Displays a screen for entering your destination.

[1911] Input: None (screen input only)

[1912] Output: Destination input screen

[1913] Step 13:

[1914] Entering a destination and sending a route calculation request

[1915] User: Enter the destination and press the route calculation button.

[1916] Input: Destination information

[1917] Output: The entered destination information is sent from the device to the server.

[1918] Step 14:

[1919] Calculating the best route

[1920] Server: Obtains the user's current location and destination information, and calculates the optimal route using, for example, the Google Maps API.

[1921] Input: current location information, destination information

[1922] Data calculation: Calculate the optimal route using Google Maps API

[1923] Output: Optimal travel route

[1924] Step 15:

[1925] View route information

[1926] Device: Displays calculated route and navigation information on the screen.

[1927] Input: Optimal travel route

[1928] Output: Route information display screen

[1929] Step 16:

[1930] Display of local rules explanation screen

[1931] Terminal: Provides a screen explaining the local rules.

[1932] Input: None (screen input only)

[1933] Output: Local rules explanation screen

[1934] Step 17:

[1935] Display of emergency support screen

[1936] Terminal: Displays a screen that provides emergency support functions.

[1937] Input: None (screen input only)

[1938] Output: Emergency support screen

[1939] Step 18:

[1940] Submitting an Emergency Support Request

[1941] User: Press the emergency support button when in trouble.

[1942] Input: Emergency Support Request

[1943] Output: The request is sent to the server

[1944] Step 19:

[1945] Generate and submit support information

[1946] Server: Receives emergency support requests, generates appropriate advice and assistance information, and sends it to the device.

[1947] Input: Emergency Support Request

[1948] Data calculation: Generation of advice and assistance information according to the situation

[1949] Output: Support information

[1950] Step 20:

[1951] View support information

[1952] Terminal: Displays the support information sent from the server.

[1953] Input: Support Information

[1954] Output: Support information display screen

[1955] Step 21:

[1956] Display the app link setting screen

[1957] Device: Display the domestic and international software integration settings screen.

[1958] Input: None (screen input only)

[1959] Output: App link setting screen

[1960] Step 22:

[1961] Selecting and setting up linked apps

[1962] User: Select the software you want to link and set up the link.

[1963] Input: Selected linked application information

[1964] Output: Sending information from the device to the server

[1965] Step 23:

[1966] Sharing of User Information

[1967] Server: Uses the app integration API to securely share user information with selected software.

[1968] Input: Selected linked application information, user information

[1969] Data processing: Share user information using the collaboration API

[1970] Output: Connection completion information

[1971] Step 24:

[1972] Viewing integration notifications and related software

[1973] Terminal: Notifies the user that the connection is complete and displays the related software.

[1974] Input: Connection completion information

[1975] Output: Link notification and related software list

[1976] The above are the specific processing steps of the system. At each step, the user, terminal, and server cooperate to provide services that meet the needs of travelers.

[1977] (Application example 1)

[1978] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1979] Today's travelers need support not only in selecting tourist destinations and planning routes, but also in choosing food and learning dining etiquette based on local culture. Foreign travelers, in particular, are prone to experiencing inconveniences due to language barriers and cultural differences, and often have difficulty obtaining information about and choosing food. This can limit their travel experience and cause significant stress for travelers. Therefore, an integrated solution is needed to help travelers enjoy their meals while understanding local food culture, enabling them to have a comfortable trip.

[1980] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1981] In this invention, the server includes means for creating an account for a traveler, means for proposing a travel plan, means for providing translations in multiple languages, means for calculating a travel route, means for explaining local rules, means for linking with other domestic applications, means for proposing a meal plan based on the traveler's food preferences, means for calculating a travel route to a local restaurant, means for translating restaurant menus and dish descriptions, and means for explaining local rules regarding meals. This enables travelers to enjoy optimal travel plans and dining experiences without experiencing language or cultural barriers, and to have a comfortable and fulfilling trip.

[1982] "Means for creating a traveller's account" refers to a system whereby a traveller enters personal information and food preferences and stores them in a database.

[1983] The "means for suggesting travel plans" is a system that includes an algorithm that generates optimal travel plans based on the traveler's purpose, duration, and areas of interest.

[1984] A "means for providing translation in multiple languages" is a system that translates input text or audio information into multiple languages ​​and displays or plays the results.

[1985] The "means for calculating a travel route" is a system that calculates the optimal route based on the current location and destination and provides navigation information.

[1986] "A means of explaining local rules" is a system that provides travelers with the ability to explain local rules, manners, and precautions.

[1987] "Means for linking with other domestic applications" refers to a system that includes APIs and interfaces for sharing data with other domestic applications and establishing linking functions.

[1988] The "means for proposing meal plans based on a traveler's food preferences" is a system that includes an algorithm that proposes optimal meal plans based on a traveler's food preferences, current location, and regional information about the travel destination.

[1989] The "means for calculating a travel route to a local restaurant" is a system that calculates the optimal route from a traveler's current location to a local restaurant and provides that information.

[1990] The "means for translating restaurant menus and food descriptions" is a system that translates restaurant menus and food descriptions in real time and displays or plays back the results.

[1991] "Means for explaining local rules regarding meals" is a system that includes a function that explains local etiquette and rules that travelers should observe when eating.

[1992] The present invention is a system that provides a smartphone application that provides a series of support required for foreign travelers traveling within Japan. This system includes functions for creating a traveler's account, proposing travel plans, providing translation functions, calculating travel routes, explaining local rules, linking with other domestic applications, proposing meal plans, and providing information about meals.

[1993] System Overview

[1994] Account creation method

[1995] What it is: A way for travelers to enter their personal information and food preferences and store them in a database.

[1996] Example: When a traveler launches the app for the first time, they are prompted to select their language and enter their personal information. Here, they enter their name, email address, and food preferences (e.g., vegetarian, Italian, etc.) and send them to the server, which stores them in a database.

[1997] Travel plan proposal method

[1998] What it is: A way to generate optimal travel plans based on a traveler's purpose, duration, and areas of interest.

[1999] Example: If a traveler plans a two-week sightseeing trip and inputs that they are interested in historical buildings, the server will use this information to generate a travel plan including recommended spots in Tokyo, Kyoto, Osaka, etc. and send it to the device.

[2000] A means of providing translations in multiple languages

[2001] What it is: A means of translating input text or audio information into multiple languages ​​and displaying or playing the results.

[2002] Example: When a traveler types "Where is the nearest train station?", the server translates this into Japanese and displays "Where is the nearest station?" on the device, playing it aloud if necessary.

[2003] Travel route calculation method

[2004] What it is: A means of calculating the optimal route based on your current location and destination, and providing that navigation information.

[2005] Example: If a traveler wants to go to Kinkakuji Temple in Kyoto, the device will prompt the traveler to input their current location and destination, and the server will calculate the optimal travel route and send the results to the device. The traveler will then navigate using the route information displayed on the device.

[2006] Local Rules Explanation Method

[2007] Content: A means of explaining local rules, etiquette, and precautions to travelers.

[2008] Example: If a traveler encounters a problem on a train and uses the emergency support function, the device will display local rules (e.g., refrain from making calls on the train) and provide support information.

[2009] A means of linking with other domestic applications

[2010] What it is: A means to share data and establish federation capabilities with other domestic applications.

[2011] Example: If a traveler wants to link a food delivery app, the device will display a link setting screen, and when the traveler selects linking, the server will share data and establish the link.

[2012] Application example: Meal plan suggestions

[2013] Meal plan proposal method

[2014] What it is: A way to suggest optimal meal plans based on traveler's food preferences, current location, and local information about the destination.

[2015] Example: If a traveler inputs that they prefer "Italian food" and "vegetarian," the server searches for Italian vegetarian restaurants, generates the optimal meal plan, and sends it to the device.

[2016] A way to calculate travel routes to local restaurants

[2017] What it is: A way to calculate the best route from a traveler's current location to a local restaurant and display the results.

[2018] Example: A traveler calculates a route from their current location to the nearest Italian vegetarian restaurant, and the server sends the route information to the device. The traveler then uses the displayed route information to navigate to the restaurant.

[2019] A way to translate restaurant menus and food descriptions

[2020] What it is: A means of translating restaurant menus and food descriptions in real time and displaying or playing them aloud on a device.

[2021] Example: A traveler takes a photo of a restaurant menu and enters it into the app. The server translates the menu and displays "vegetarian pizza" and plays it aloud if needed.

[2022] A means of explaining local dietary rules

[2023] What it is: A way to explain to travelers the local etiquette and rules they should follow when dining out.

[2024] Example: If a traveler gets into trouble while eating, the app will display local rules such as "don't talk loudly while eating" and provide emergency support information.

[2025] Prompt Sentence Examples

[2026] User's name is John Doe, prefers Italian cuisine and is a vegetarian. Recommend the best Italian vegetarian restaurants nearby.

[2027] As a result, the system of the present invention allows travelers to enjoy optimal travel plans and dining experiences, and to have a comfortable and fulfilling trip without having to worry about language or cultural barriers.

[2028] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2029] Step 1:

[2030] The user launches the application for the first time and selects language preference and personal information, including name, email address, and food preferences (e.g., Italian, vegetarian).

[2031] Input: Name, email address, food preferences

[2032] Output: Personal information and food preferences are stored in a database

[2033] Specific operation: The terminal receives input data from the user and sends it to the server, which then stores the received information in a database.

[2034] Step 2:

[2035] The user moves to the travel plan proposal screen and inputs the purpose of the trip, the duration, and areas of interest.

[2036] Input: purpose of trip, duration, areas of interest

[2037] Output: Travel plan proposal data

[2038] How it works: The device receives user input and sends it to the server. The server uses the generative AI model to generate an optimal travel plan and sends it to the device.

[2039] Step 3:

[2040] The user uses the translation feature within the application to translate the text they entered.

[2041] Input:Text

[2042] Output: Translation result

[2043] Specific operation: The device sends the text entered by the user to the server. The server uses a translation algorithm to translate the entered text into the target language and sends the translation result to the device. The device displays the result and plays it aloud if necessary.

[2044] Step 4:

[2045] The user inputs a destination within the application and calculates a route.

[2046] Input:Destination

[2047] Output: Optimal travel route

[2048] Specific operation: The device sends current location information and destination information entered by the user to the server. The server uses a geographic information system (GIS) to calculate the optimal travel route and sends the results to the device. The device then displays the route information.

[2049] Step 5:

[2050] The user is taken to a meal plan suggestion screen where suggestions are given based on their food preferences and current location.

[2051] Input: Food preferences, current location

[2052] Output: Optimal meal plan

[2053] How it works: The device sends the user's dining preferences and current location information to the server. The server then searches a database of restaurants and generates an optimal meal plan, which it then sends to the device. The device then displays the results.

[2054] Step 6:

[2055] A user translates a restaurant menu and checks the dish descriptions.

[2056] Input: Menu text or image

[2057] Output: Translated menu and dish descriptions

[2058] Specific operation: The device sends the menu text or image data from the user to the server. The server uses OCR (optical character recognition) and a translation algorithm to translate the menu and send the result to the device. The device then displays or plays the translated menu aloud.

[2059] Step 7:

[2060] Users can check local rules while eating and get any assistance they need.

[2061] Input: Request for local rules

[2062] Output: Local rule explanation and supporting information

[2063] Specific operation: The terminal sends a request for a local rule from the user to the server. The server searches the database for the relevant local rule information and sends the result to the terminal. The terminal displays an explanation of the local rule and provides the user with the necessary information.

[2064] In this way, the system provides multifunctional support to help travelers enjoy their trips comfortably.

[2065] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2066] System Overview

[2067] This invention provides a smartphone application that helps foreign travelers enjoy a comfortable trip within Japan. This application is a system that combines account creation for travelers, travel plan suggestions, translation functions, route calculations, explanations of local rules, and integration with other domestic applications, as well as an emotion engine that recognizes the user's emotions. Details of each function and specific operating procedures are explained below.

[2068] Account creation method

[2069] 1. Initial Setup

[2070] Device: Turn on the power and display the initial setup screen.

[2071] User: Select a language and enter personal information.

[2072] Server: Creates a traveler account based on the entered information and stores it in a database.

[2073] Travel plan proposal method

[2074] 1. Travel plan creation

[2075] Device: Display the travel plan creation screen.

[2076] User: Enter the purpose of the trip, duration, and genres of interest.

[2077] Server: Based on this input information, the generation algorithm is executed to generate the optimal travel plan.

[2078] Server: Sends the generated plan to the device and allows the user to view and edit the displayed plan.

[2079] Translation tools

[2080] 1. Real-time translation

[2081] Terminal: Display the translation function screen.

[2082] User: Enter the text you want to translate.

[2083] Terminal: Sends entered text to the server.

[2084] Server: Uses the translation API to translate the input text into the specified language.

[2085] Server: Sends the translation results to the device.

[2086] Device: The translation results are displayed on the screen and played aloud if necessary.

[2087] Travel route calculation method

[2088] 1. Route calculation

[2089] Device: Display the destination input screen.

[2090] User: Enter destination.

[2091] Terminal: Sends current location information and destination information to the server.

[2092] Server: Executes the travel route calculation algorithm and calculates the optimal travel route.

[2093] Server: Sends calculated travel route data and navigation information to the terminal.

[2094] Terminal: Displays travel route and navigation information on the screen.

[2095] Local Rules Explanation Method

[2096] 1. Provision of local rules

[2097] Terminal: Display the local rules explanation screen.

[2098] Terminal: Provides emergency support when users are in trouble.

[2099] Server: Sends appropriate advice and support information to the terminal in response to an emergency support request.

[2100] Linking with other domestic applications

[2101] 1. App Integration

[2102] Device: Display the domestic application integration settings screen.

[2103] User: Select the application you want to integrate.

[2104] Terminal: Sends the link setting information of the selected application to the server.

[2105] Server: Uses the application integration API to share user information and establish integration.

[2106] Terminal: Notifies the user that the connection is complete and makes the related application available.

[2107] Emotion engine integration

[2108] The present invention is combined with an emotion engine that recognizes the user's emotions and analyzes user input and sensor data to determine the user's emotional state.

[2109] Using an emotion engine to suggest travel plans

[2110] The emotion engine takes into account the user's emotional state when suggesting travel plans, resulting in more personalized suggestions.

[2111] for example:

[2112] 1. User: When entering a travel plan, if the emotion engine determines that the user is "tired," it will suggest a plan that includes many relaxing spots.

[2113] 2. User: If the emotion engine determines that the user is "excited" when entering travel plans, it will suggest plans that include many active activities.

[2114] Using emotion engine in translation function

[2115] The emotion engine is also used in the translation function, providing translations in an appropriate tone according to the user's emotional state.

[2116] for example:

[2117] 1. User: If the emotion engine determines that the user is "lost," provide a translation in a reassuring tone.

[2118] 2. User: If the emotion engine determines that the user is grateful, it will provide a translation in a polite tone.

[2119] Using the emotion engine to explain local rules

[2120] By using the emotion engine in explaining local rules, it becomes possible to provide explanations that correspond to the user's emotional state.

[2121] for example:

[2122] 1. User: If the emotion engine determines that the user is "nervous," it will provide a more detailed and polite explanation.

[2123] 2. User: If the emotion engine determines that the user is "relaxed," provide a brief, to-the-point explanation.

[2124] Specific examples

[2125] A concrete example of travel plan suggestions using an emotion engine

[2126] 1. User operations

[2127] User: You are planning a two-week trip for sightseeing and you are interested in historical sites.

[2128] Emotion engine: Determines that the user is tired.

[2129] 2. Server Processing

[2130] Server: Based on the judgment of the emotion engine, generate a travel plan that includes many relaxing spots.

[2131] Example plans:

[2132] Day 1: Tokyo Tower, Sensoji Temple

[2133] Day 2: Kyoto's Kinkakuji Temple, a leisurely stroll in Arashiyama, and more

[2134] 3. Displaying the terminal

[2135] Terminal: Presents the generated plan to the user and allows the user to customize the plan.

[2136] A concrete example of a translation function using an emotion engine

[2137] 1. User operations

[2138] User: Type "I'm lost."

[2139] Emotion engine: Determines if the user is feeling anxious.

[2140] 2. Server Processing

[2141] Server: Uses the translation API to generate a translation in a tone that calms anxiety.

[2142] Translation result: "Don't worry. The nearest station is very close."

[2143] 3. Displaying the terminal

[2144] On the device: Displays the translation results and plays them aloud to give the user peace of mind.

[2145] The above is an embodiment of the invention that combines specific processing steps of the present invention and an emotion engine.

[2146] The processing flow will be explained below.

[2147] Specific processing steps for travel plan proposals using an emotion engine

[2148] Step 1:

[2149] Device: Display the travel plan creation screen.

[2150] Step 2:

[2151] User: Enter the purpose of the trip (e.g. sightseeing), duration (e.g. 2 weeks), and genre of interest (e.g. historical buildings).

[2152] Step 3:

[2153] Terminal: Sends the user's facial expressions and voice data to the emotion engine along with the user's input.

[2154] Step 4:

[2155] Emotion engine: Analyzes the user's facial expressions and voice data to determine the user's emotional state (e.g., tired, excited, etc.).

[2156] Step 5:

[2157] Emotion engine: Sends the judgment result of the emotional state to the server.

[2158] Step 6:

[2159] Server: Based on the user's input information and the judgment results of their emotional state, the server runs a travel plan suggestion algorithm and generates an optimal travel plan.

[2160] Example: If the user is determined to be tired, generate a plan that includes many relaxation spots.

[2161] Step 7:

[2162] Server: Sends the generated travel plan to the terminal.

[2163] Step 8:

[2164] Terminal: Displays the generated travel plan to the user, allowing the user to review and edit the plan.

[2165] Specific processing steps for translation function using emotion engine

[2166] Step 1:

[2167] Terminal: Display the translation function screen.

[2168] Step 2:

[2169] User: Enters the text they want to translate (e.g., "Where is the nearest train station?").

[2170] Step 3:

[2171] Terminal: Sends the user's facial expressions and voice data to the emotion engine along with the user's input.

[2172] Step 4:

[2173] Emotion engine: Analyzes the user's facial expressions and voice data to determine the user's emotional state (e.g., anxiety, joy, etc.).

[2174] Step 5:

[2175] Emotion engine: Sends the judgment result of the emotional state to the server.

[2176] Step 6:

[2177] Server: Based on the user's input text and the judged emotional state, the server translates in an appropriate tone.

[2178] Step 7:

[2179] Server: Translates the results (e.g., "Don't worry. The nearest station is nearby.") to the device.

[2180] Step 8:

[2181] Device: The translation results are displayed on the screen and played aloud if necessary.

[2182] Specific processing steps for explaining local rules using the emotion engine

[2183] Step 1:

[2184] Terminal: Display the local rules explanation screen.

[2185] Step 2:

[2186] User: Enter your problem or question, or press the emergency support button.

[2187] Step 3:

[2188] Terminal: Sends the user's facial expressions and voice data to the emotion engine along with the user's input.

[2189] Step 4:

[2190] Emotion engine: Analyzes the user's facial expressions and voice data to determine the user's emotional state (e.g., tension, relief, etc.).

[2191] Step 5:

[2192] Emotion engine: Sends the judgment result of the emotional state to the server.

[2193] Step 6:

[2194] Server: Based on the user's situation and emotional state, the server generates appropriate explanations of local rules and emergency support advice.

[2195] For example, if the user is perceived as nervous, provide more detailed and polite explanations.

[2196] Step 7:

[2197] Server: Sends the generated advice and support information to the terminal.

[2198] Step 8:

[2199] Terminal: Displays advice and support information received from the server to the user, giving them a sense of security.

[2200] The above are the specific processing steps of the present invention.

[2201] Example 2

[2202] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2203] For foreign tourists to travel comfortably within Japan, there are a wide range of issues to overcome, including language barriers, transportation methods, local rules, etc. In addition, it is also necessary to provide appropriate plans and support based on the emotional state of each individual traveler, which presents many challenges that cannot be addressed by existing travel support systems.

[2204] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2205] In this invention, the server includes means for creating a traveler's account, means for proposing a travel plan, means for providing translations in multiple languages, means for calculating travel routes, means for explaining local rules, means for linking with other domestic applications, and means for integrating an emotion engine that analyzes the user's emotional state. This makes it possible to provide optimal plans and support according to the emotions of travelers so that foreign travelers can travel comfortably within Japan.

[2206] "Means for creating a traveler's account" is a general term for devices and software that allow a traveler to enter personal information and store that information in a database.

[2207] "Means for proposing travel plans" is a general term for devices and software that use a generation algorithm to generate optimal travel plans based on inputs from travelers regarding their purpose, duration, and areas of interest, and then display them so that users can customize them.

[2208] "Means for providing translation in multiple languages" is a general term for devices and software that have the function of translating text entered by a user into a specified language and displaying the translation result on a screen or playing it aloud.

[2209] "Means for calculating a travel route" is a general term for devices and software that calculate the optimal travel route based on the destination and current location information entered by the user and provide navigation information.

[2210] "Means for explaining local rules" is a general term for devices and software that provide users with information about local culture, customs, and manners.

[2211] "Means for linking with other domestic applications" is a general term for devices and software for sharing information and establishing linkages with other domestic applications used by travelers.

[2212] An "emotion engine" is a general term for devices and software that analyze user input and sensor data to determine an emotional state and then respond or make appropriate suggestions accordingly.

[2213] MODE FOR CARRYING OUT THE INVENTION

[2214] The present invention provides an information processing device that helps foreign travelers to have a comfortable trip within Japan. This device is a system that includes functions such as creating a travel account for the traveler, proposing travel plans, translating into multiple languages, calculating travel routes, explaining local rules, linking with other domestic applications, and integrating an emotion engine.

[2215] Account creation method

[2216] Device: Turn on the power, launch the application, and display the initial setup screen. The user selects a language and enters personal information such as name, email address, and password. The device temporarily stores this information and sends it to the server.

[2217] Server: Stores the received information in a database and generates a confirmation message for the account creation. The device displays a notification to the user that the account was created successfully and provides the option to proceed to the next step.

[2218] Travel plan proposal method

[2219] Terminal: Displays a travel plan creation screen and provides input fields for the purpose of the trip, duration, and genres of interest. The user enters this information, and the terminal sends it to the server.

[2220] Server: Based on the received information, the server uses a generative AI model to generate an optimal travel plan. The generated plan is sent to the device, where the user can view and edit it.

[2221] Translation tools

[2222] Terminal: Displays the translation function screen and provides the user with a text input field. The user enters the text to be translated and selects the language to specify. The terminal sends the entered text and the selected language to the server.

[2223] Server: Calls the translation API to translate the input text into the specified language. The translation result is sent to the device, which displays it on the screen and, if necessary, plays it aloud.

[2224] Travel route calculation method

[2225] Device: Displays a destination entry screen and gives the user the option to enter their current location and destination. The user can enter their current location and destination, or use the GPS function to automatically obtain their current location. The device then sends this information to the server.

[2226] Server: Executes the travel route calculation algorithm to calculate the optimal travel route. Sends the calculated route and navigation information to the device, which displays it.

[2227] Local Rules Explanation Method

[2228] Terminal: Displays a local rules explanation screen and provides information on local culture, customs, and manners. If a user is in trouble, they can press the emergency support button to request assistance. The terminal then sends the request to the server.

[2229] Server: Generates appropriate advice and support information based on the request and sends it to the terminal, which then displays it to the user.

[2230] Linking with other domestic applications

[2231] Terminal: The domestic application linkage setting screen is displayed, along with a list of applications to be linked. The user selects the application they want to link, and the terminal sends the setting information to the server.

[2232] Server: Calls the application integration API, shares user information with related applications, and establishes integration. The server sends a notification of integration completion to the device, and the device notifies the user.

[2233] Emotion engine integration

[2234] The system incorporates an emotion engine that analyzes user input and sensor data to determine the user's emotional state, and based on this, it makes suggestions and offers the following:

[2235] Using an emotion engine to suggest travel plans

[2236] When a user inputs a travel plan, the emotion engine analyzes the user's state. For example, if it determines that the user is tired, it will generate a plan that includes many relaxing spots. On the other hand, if it determines that the user is excited, it will suggest a plan that includes many active activities.

[2237] Example prompt sentence:

[2238] "I want to relax, so please tell me about a travel plan that includes places where I can relax."

[2239] Using emotion engine in translation function

[2240] The emotion engine also analyzes when a user uses the translation function. For example, if the user is feeling anxious, the system will provide a translation in a reassuring tone.

[2241] Example prompt sentence:

[2242] "I'm lost. Please tell me what to do."

[2243] Using the emotion engine to explain local rules

[2244] When a user requests emergency support, the emotion engine analyzes the user's state and, if it determines that the user is nervous, for example, provides a detailed and thorough explanation.

[2245] This will enable us to provide a comprehensive support system to help foreign travelers enjoy a comfortable trip within Japan.

[2246] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2247] Program processing flow

[2248] Account creation method

[2249] Step 1:

[2250] When the device is turned on, it launches the application and displays the initial setup screen. When the user selects a language, the device switches the interface based on that selection: input (language selected by the user), output (interface display in the selected language).

[2251] Step 2:

[2252] The user enters personal information such as name, email address, password, etc. Input (personal information entered by the user), output (personal information once input is complete).

[2253] Step 3:

[2254] The terminal temporarily stores the input information and sends it to a server via the Internet. Input (user's personal information), Output (sending information to the server).

[2255] Step 4:

[2256] The server stores the received information in a database and generates a confirmation message for creating an account. Data processing (storing personal information), data calculation (generating a confirmation message), output (confirmation message).

[2257] Step 5:

[2258] The terminal displays a notification to the user that the account was created successfully and provides the option to proceed to the next step. Input (confirmation message received from the server), Output (notification displayed to the user).

[2259] Travel plan proposal method

[2260] Step 1:

[2261] The terminal displays a travel plan creation screen and provides input fields for the purpose of the trip, duration, and genres of interest. Input (request for screen display), output (display of input fields).

[2262] Step 2:

[2263] The user inputs the purpose of the trip, the duration, and the genres they are interested in. Input (user's travel information), output (entered travel information).

[2264] Step 3:

[2265] The terminal collects this information and sends it to the server. Input (travel information), Output (sending information to the server).

[2266] Step 4:

[2267] The server uses a generative AI model based on the received information to generate an optimal travel plan, including data processing (analysis of travel information), data calculation (generation of optimal plan), and output (generated travel plan).

[2268] Step 5:

[2269] The server sends the generated itinerary to the terminal. Input (generated itinerary), Output (transmission to terminal).

[2270] Step 6:

[2271] The terminal displays the received travel plan to the user and allows editing and saving. Input (travel plan received from server), Output (providing display and editing options to the user).

[2272] Translation tools

[2273] Step 1:

[2274] The terminal displays a translation function screen and provides the user with a text input field. Input (request for screen display), output (display of input field).

[2275] Step 2:

[2276] The user enters the text to be translated and selects the language to be translated. Input (translation text and language selection), Output (entered text and selected language).

[2277] Step 3:

[2278] The terminal sends the entered text and the selected language to the server. Input (text and language), Output (send to server).

[2279] Step 4:

[2280] The server calls the translation API and translates the input text into the specified language. Data processing (text analysis), data calculation (translation processing), and output (translation results) are performed.

[2281] Step 5:

[2282] The server sends the translation results to the terminal. Input (translation results), output (transmission to terminal).

[2283] Step 6:

[2284] The device displays the translation results on the screen and plays them aloud if necessary. Input (translation results), output (screen display and audio playback).

[2285] Travel route calculation method

[2286] Step 1:

[2287] The device displays a destination entry screen and gives the user the option to enter their current location and destination. Input (request for screen display), Output (display of input field).

[2288] Step 2:

[2289] The user can input their current location and destination, or use the GPS function to automatically obtain their current location. Input (current location and destination, or GPS data), output (input location data).

[2290] Step 3:

[2291] The device sends the acquired current location information and destination information to the server. Input (current location and destination information), output (transmission to server).

[2292] Step 4:

[2293] The server executes the travel route calculation algorithm to calculate the optimal travel route, including data processing (analysis of location information), data calculation (route calculation), and output (calculated route data).

[2294] Step 5:

[2295] The server sends the calculated route and navigation information to the device.,Input (calculated route data), Output (transmission to device).

[2296] Step 6:

[2297] The device displays navigation information and maps on the screen, allowing the user to check the route. Input (route data), output (display of map and information).

[2298] Local Rules Explanation Method

[2299] Step 1:

[2300] The terminal displays a local rules explanation screen and provides information on the culture, customs, and manners of each region. Input (request for screen display), output (display of rules explanation).

[2301] Step 2:

[2302] When a user is in trouble, he / she presses the emergency support button to request assistance. Input (pressing the support request button), Output (support request data).

[2303] Step 3:

[2304] The terminal sends an emergency support request to the server. Input (support request data), Output (transmission to server).

[2305] Step 4:

[2306] The server generates appropriate advice and support information based on the request content and sends it to the terminal. Data processing (analysis of the request content), data calculation (generation of advice), output (generated support information).

[2307] Step 5:

[2308] The terminal displays the received advice and support information to the user. Input (received support information), output (display to user).

[2309] Linking with other domestic applications

[2310] Step 1:

[2311] The terminal displays the domestic application linkage setting screen and displays a list of applications to be linked. Input (request for screen display), output (display of setting screen).

[2312] Step 2:

[2313] The user selects the application they want to link. Input (selected application), output (selected information).

[2314] Step 3:

[2315] The terminal sends the link setting information of the selected application to the server. Input (link setting information), output (transmission to server).

[2316] Step 4:

[2317] The server calls the application integration API, shares user information with related applications, and establishes integration. Data processing (sharing user information), data calculation (establishing integration), and output (notifying that integration is complete) are performed.

[2318] Step 5:

[2319] The server sends a notification of completion of collaboration to the terminal. Input (notification of completion of collaboration), output (send to terminal).

[2320] Step 6:

[2321] The device notifies the user that the connection is complete, and synchronization with related applications becomes possible. Input (connection completion notification), Output (notification to the user).

[2322] Emotion engine integration

[2323] Step 1:

[2324] The server analyzes the user's input and sensor data to determine the emotional state. Input (user's input and sensor data), Output (determined emotional state).

[2325] Step 2:

[2326] When a user inputs their travel plan, the emotion engine analyzes the user's state: input (user's travel information), output (determining their emotional state).

[2327] Step 3:

[2328] The server generates a travel plan based on the user's emotional state. For example, if the server determines that the user is tired, it generates a plan that includes many relaxation spots. Data processing (analysis of emotional state), data calculation (plan generation), output (generated plan).

[2329] Step 4:

[2330] The server uses an emotion engine to translate in a reassuring tone if the user is feeling anxious. Data processing (analysis of emotional state), data calculation (translation processing), output (translation results).

[2331] Step 5:

[2332] The server analyzes the user's emotional state during emergency support and provides detailed and thorough explanations. Data processing (analysis of emotional state), data calculation (generation of support information), output (support information).

[2333] (Application example 2)

[2334] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2335] Previous travel support applications lacked comprehensive support for foreign travelers to have a comfortable stay in Japan, particularly due to language barriers and a lack of understanding of local rules. Furthermore, the guidance and product explanations within the virtual stores travelers visited were limited to providing general information and lacked personalized responses tailored to the individual emotional state of the travelers. This resulted in a decrease in traveler satisfaction.

[2336] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2337] In this invention, the server includes means for creating an account for a traveler, means for proposing a travel plan, means for providing translations in multiple languages, means for calculating travel routes, means for explaining local rules, means for linking with other domestic applications, means for recognizing a user's emotions and providing information in an appropriate tone accordingly, and means for proposing product guides and recommended products in a virtual store. This makes it possible to provide personalized guides and product explanations tailored to the emotional state of a traveler while overcoming language barriers, thereby improving traveler satisfaction.

[2338] "Means for creating a traveler's account" refers to the process by which a traveler inputs personal information and manages that information on the server.

[2339] The "means for proposing a travel plan" is a process for generating and proposing an optimal travel plan based on the traveler's purpose, period, and areas of interest.

[2340] A "means for providing translation in multiple languages" is a process for translating user-entered text into multiple languages ​​and providing the results.

[2341] The "means for calculating a travel route" is a process for calculating and providing guidance on the optimal travel route based on the user's current location and destination.

[2342] "Means of explaining local rules" refers to the process of explaining the rules and manners of each region in Japan to foreign tourists.

[2343] "Means of linking with other domestic applications" refers to the process by which travelers link with other domestic applications when using those applications.

[2344] "Means for recognizing the user's emotions and providing information in an appropriate tone" is a process for analyzing and determining the user's emotional state and providing information in the most appropriate tone based on that.

[2345] "Means for providing product information and suggesting recommended products within a virtual store" refers to the process of providing product information to users within a virtual store and suggesting recommended products based on the user's interests and emotions.

[2346] MODE FOR CARRYING OUT THE INVENTION

[2347] This invention provides a mobile terminal application system to support foreign travelers in enjoying a comfortable trip within Japan. This system can create a traveler's account, suggest travel plans, provide translations in multiple languages, calculate travel routes, explain local rules, link with other domestic applications, and even provide information in an appropriate tone according to the user's emotional state.

[2348] Hardware and software used

[2349] Hardware: Smartphones, smart glasses, head-mounted displays

[2350] Software: Python, TensorFlow (emotion engine), Google Translate API (translation), Google Maps API (route calculation)

[2351] Embodiments of each means of the system

[2352] 1. How to create a traveller account

[2353] The user enters personal information and sends it to the server, which stores it in a database and creates an account for the traveler.

[2354] 2. A way to suggest travel plans

[2355] The user inputs the purpose, duration, and areas of interest of the trip. The server runs a generation algorithm based on this input information to generate an optimal travel plan. In doing so, it uses an emotion engine to analyze the user's emotional state. For example, if the server determines that the user is "tired," it will suggest a plan that includes many relaxation spots.

[2356] 3. A way to provide translations in multiple languages

[2357] The user enters the text they want to translate. The device sends this text to the server. The server uses the Google Translate API to translate it and sends the results back to the device. The translation is delivered in an appropriate tone based on the emotion engine.

[2358] 4. Means of calculating travel routes

[2359] The user inputs a destination. The device sends the current location and destination information to the server. The server uses the Google Maps API to calculate the optimal route and provide navigation information.

[2360] 5. Means of Explaining Local Rules

[2361] By storing local rule information on the device, the information can be displayed when the user is in trouble. In addition, the server provides appropriate advice and support information in response to emergency support requests from the device.

[2362] 6. Means of linking with other domestic applications

[2363] On the device, select the application you want to link and send the link setting information to the server. The server then uses the link API to share information between the applications and establish the link.

[2364] 7. A way to recognize the user's emotions and deliver information in an appropriate tone.

[2365] The camera feed and audio data are fed into a TensorFlow model to analyze the user's emotional state. For example, if the system detects that the user is feeling anxious, it will provide information in a tone that will calm the user's anxiety.

[2366] 8. A means of product guidance and recommendations within a virtual store

[2367] When a user views product information in a virtual store, the system translates and displays guidance information. It recognizes the user's emotional state and provides recommendations and guidance accordingly.

[2368] Specific examples

[2369] Example 1: Travel plan suggestions

[2370] 1. A user plans a two-week sightseeing trip and indicates that they are interested in historical sites.

[2371] 2. The emotion engine determines that the user is tired.

[2372] 3. The server generates an itinerary that includes many relaxing spots.

[2373] 4. Display the generated plan on the device and allow the user to customize it.

[2374] Example 2: Applying the translation function

[2375] 1. The user types "I'm lost."

[2376] 2. The emotion engine determines that the user is feeling anxious.

[2377] 3. The server generates a translation in a calming tone.

[2378] 4. The device will display the message "Don't worry, the nearest station is right nearby" and play it aloud.

[2379] Example 3: Guidance in a virtual store

[2380] 1. A user views products in a virtual store.

[2381] 2. The emotion engine determines the user's emotional state and recognizes that they are feeling anxious.

[2382] 3. The server provides product descriptions in a reassuring translation tone.

[2383] 4. The device will display and play a message saying, "This product is a high-quality item that you can use with confidence."

[2384] Prompt Sentence Examples

[2385] If users are feeling unsure, provide a reassuring tone in your product description. Translate the product information below to create a reassuring message:

[2386] Product Name: High-quality bags

[2387] Description: This bag is durable and can be used for a long time. It is made of high-quality materials and is lightweight and easy to carry.

[2388] Example output: "This bag is of very high quality and will last a long time. You can use it with confidence."

[2389] The above is an embodiment of the present invention.

[2390] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2391] Step 1:

[2392] The user launches the smartphone application, selects a language, and enters their personal information. The device sends this information to the server, which then creates a traveler account based on the information entered and stores it in a database.

[2393] Input: Language selection, personal information

[2394] Output: Traveler account created and saved to database

[2395] Data processing: Convert input information into database format and save

[2396] Step 2:

[2397] The user opens the travel plan suggestion screen within the application and inputs the purpose of the trip, duration, and genres of interest. The device sends this input information to the server, which then runs a generation algorithm to generate the optimal travel plan. In doing so, the server uses an emotion engine to determine the user's emotional state and personalize the plan accordingly.

[2398] Input: purpose of trip, duration, genre of interest

[2399] Output: A personalized itinerary

[2400] Data processing: Input information into a generative algorithm, and an emotion engine determines the emotional state and optimizes the plan.

[2401] Step 3:

[2402] The user uses the translation function and inputs the text they want to translate. The device sends this text to the server, which translates it using the Google Translate API. Once the translation result is generated, the server adjusts the tone using an emotion engine and sends it to the device for display.

[2403] Input: The text you want to translate

[2404] Output: Translation result with adjusted tone

[2405] Data processing: Text translation, tone adjustment with emotion engine

[2406] Step 4:

[2407] The user inputs the destination they wish to visit. The device sends the current location and destination information to the server, which then uses the Google Maps API to calculate the optimal route. The resulting route and navigation information are then sent to the device and displayed.

[2408] Input: Destination information, current location information

[2409] Output: Travel route and navigation information

[2410] Data processing: Route calculation, navigation information generation

[2411] Step 5:

[2412] When a user requests an explanation of local rules, the terminal displays the information on the local rules. Furthermore, if the user encounters an emergency, the terminal can use the emergency support function to request assistance from the server. The server provides appropriate advice and assistance information, which is then sent to the terminal for display.

[2413] Input: Request for local rules clarification, emergency support request

[2414] Output: Local rules information, emergency support information

[2415] Data processing: information search and display, generation of supporting information

[2416] Step 6:

[2417] The user opens the link setting screen to link with other domestic applications. The user selects the application they want to link, and the device sends the link setting information to the server. The server uses the link API to share information and establish the link.

[2418] Input: Select the application you want to link

[2419] Output: Established integration settings

[2420] Data processing: Execution of linked API, information sharing settings

[2421] Step 7:

[2422] When a user wants to check product information in a virtual store, the device sends the product information to the server. The server translates the product information using a translation API and recognizes the user's emotional state using an emotion engine. It then provides a product description and recommended products in an appropriate tone and sends them to the device. The device can then display the information and play it back aloud.

[2423] Input: Product information request

[2424] Output: Translated product information, description and suggestions based on emotional state

[2425] Data processing: translating product information, analyzing emotional states, adjusting tone and generating recommendations

[2426] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2427] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2428] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2429] [Fourth embodiment]

[2430] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2431] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2432] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2433] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2434] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2435] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2436] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2437] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2438] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2439] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2440] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2441] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2442] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2443] System Overview

[2444] The present invention provides a smartphone application that helps foreign travelers enjoy a comfortable trip within Japan. This application is a system that includes functions such as account creation for travelers, travel plan suggestions, translation functions, route calculation, explanation of local rules, and linkage with other domestic applications. Details of each function and specific operating procedures are explained below.

[2445] Account creation method

[2446] 1. Initial Setup

[2447] Device: Turn on the power and display the initial setup screen.

[2448] User: Select a language and enter personal information.

[2449] Server: Receives the entered information, creates a traveler account, and stores it in a database.

[2450] Travel plan proposal method

[2451] 1. Travel plan creation

[2452] Device: Display the travel plan creation screen.

[2453] User: Enter the purpose of the trip, duration, and genres of interest.

[2454] Server: Based on this input information, an algorithm is used to generate the optimal travel plan.

[2455] Server: Sends the generated plan to the device and allows the user to view and edit the displayed plan.

[2456] Translation tools

[2457] 1. Real-time translation

[2458] Terminal: Display the translation function screen.

[2459] User: Enter the text you want to translate.

[2460] Server: Translates the input text and sends the results to the device.

[2461] Device: The translation results are displayed on the screen and played aloud if necessary.

[2462] Travel route calculation method

[2463] 1. Route calculation

[2464] Device: Display the destination input screen.

[2465] User: Enter destination.

[2466] Server: Calculates the optimal travel route based on current location information and destination information, and sends the results to the device.

[2467] Terminal: Displays travel route and navigation information on the screen.

[2468] Local Rules Explanation Method

[2469] 1. Provision of local rules

[2470] Terminal: Display the local rules explanation screen.

[2471] Terminal: Provides emergency support when users are in trouble.

[2472] Server: Sends appropriate advice and support information to the terminal in response to an emergency support request.

[2473] Linking with other domestic applications

[2474] 1. App Integration

[2475] Device: Display the domestic application integration settings screen.

[2476] User: Select the application you want to link and set up the link.

[2477] Server: Uses the application integration API to share user information and establish integration.

[2478] Terminal: Notifies the user that the connection is complete and makes the related application available.

[2479] Specific examples

[2480] Specific examples of travel plan proposals

[2481] 1. User operations

[2482] User: You are planning a two-week trip for sightseeing and you are interested in historical sites.

[2483] 2. Server Processing

[2484] Server: Based on this information, it generates a travel plan that includes recommended spots in Tokyo, Kyoto, Osaka, etc.

[2485] Example plans:

[2486] Day 1: Tokyo Tower, Sensoji Temple

[2487] Day 2: Kinkakuji Temple and Fushimi Inari Shrine in Kyoto

[2488] 3. Displaying the terminal

[2489] Terminal: Presents the generated plan to the user and allows the user to customize the plan.

[2490] Examples of translation functions

[2491] 1. User operations

[2492] User: Type "Where is the nearest train station?"

[2493] 2. Server Processing

[2494] Server: Translate this text into Japanese and make it "Where is the nearest station?"

[2495] 3. Displaying the terminal

[2496] Terminal: Helps users ask for directions by displaying translation results and playing them aloud.

[2497] As described above, the system of the present invention provides a multifunctional platform that meets the various needs of travelers and makes their travels more comfortable.

[2498] The processing flow will be explained below.

[2499] Account creation method

[2500] Step 1:

[2501] Terminal: When the application is launched for the first time, the initial setup screen is displayed.

[2502] Step 2:

[2503] User: Select the desired language from the language selection menu that appears.

[2504] Step 3:

[2505] Device: Displays the account creation screen based on the selected language.

[2506] Step 4:

[2507] User: Enter personal information such as name, email address, and password and press the "Register" button.

[2508] Step 5:

[2509] Terminal: Sends the entered information to the server.

[2510] Step 6:

[2511] Server: Creates a user account in the database based on the received information and sends a registration completion status to the terminal.

[2512] Step 7:

[2513] Terminal: Display a registration completion message to the user.

[2514] Travel plan proposal method

[2515] Step 1:

[2516] Device: Display the travel plan creation screen.

[2517] Step 2:

[2518] User: Enter the purpose of the trip (e.g. sightseeing), duration (e.g. 2 weeks), and genre of interest (e.g. historical buildings).

[2519] Step 3:

[2520] Terminal: Sends the entered information to the server.

[2521] Step 4:

[2522] Server: Runs the travel plan suggestion algorithm and generates the optimal travel plan based on the user's input information.

[2523] Step 5:

[2524] Server: Sends the generated travel plan to the terminal.

[2525] Step 6:

[2526] Terminal: Displays the generated travel plan to the user, allowing the user to review and edit the plan.

[2527] Translation tools

[2528] Step 1:

[2529] Terminal: Display the translation function screen.

[2530] Step 2:

[2531] User: Enter the text you want to translate.

[2532] Step 3:

[2533] Terminal: Sends entered text to the server.

[2534] Step 4:

[2535] Server: Uses the translation API to translate the input text into the specified language.

[2536] Step 5:

[2537] Server: Sends the translation results to the device.

[2538] Step 6:

[2539] Device: The translation results are displayed on the screen and played aloud if necessary.

[2540] Travel route calculation method

[2541] Step 1:

[2542] Device: Display the destination input screen.

[2543] Step 2:

[2544] User: Enter your destination (e.g. Kyoto Station).

[2545] Step 3:

[2546] Terminal: Sends current location information and destination information to the server.

[2547] Step 4:

[2548] Server: Executes the travel route calculation algorithm and calculates the optimal travel route.

[2549] Step 5:

[2550] Server: Sends calculated travel route data and navigation information to the terminal.

[2551] Step 6:

[2552] Terminal: Displays travel route and navigation information on the screen.

[2553] Local Rules Explanation Method

[2554] Step 1:

[2555] Terminal: Display the local rules explanation screen.

[2556] Step 2:

[2557] User: Check the local rules, or press the emergency support button if you need emergency support.

[2558] Step 3:

[2559] Terminal: When the emergency support button is pressed, the information is sent to the server.

[2560] Step 4:

[2561] Server: Provides specific advice and support information tailored to the user's situation and sends it to the terminal.

[2562] Step 5:

[2563] Terminal: Displays advice and assistance information received from the server to the user.

[2564] Linking with other domestic applications

[2565] Step 1:

[2566] Device: Display the domestic application integration settings screen.

[2567] Step 2:

[2568] User: Select the application you want to integrate.

[2569] Step 3:

[2570] Terminal: Sends the link setting information of the selected application to the server.

[2571] Step 4:

[2572] Server: Uses the application integration API to share user information and establish integration.

[2573] Step 5:

[2574] Server: Notifies the terminal that the connection is complete.

[2575] Step 6:

[2576] Terminal: Notifies the user that the linking is complete and makes the linked application available.

[2577] The above are the specific processing steps of the present invention.

[2578] Example 1

[2579] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2580] In order for foreign travelers to enjoy a comfortable trip within Japan, they need a wide range of information and support, but they often have difficulty gathering information and communicating due to differences in language and culture.The purpose of this invention is to alleviate difficulties during travel and ensure a comfortable travel experience by providing the information and support travelers need in a centralized manner.

[2581] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2582] In this invention, the server includes a means for creating a traveler's account, a means for inputting the traveler's purpose, period, and areas of interest and proposing an optimal travel plan using a generative AI model, and a means for translating the input text into multiple languages ​​and displaying and playing back the results by voice, thereby enabling travelers to easily complete complex procedures and obtain the necessary information in a timely manner.

[2583] The "means of creating a traveler's account" is the process by which a traveler inputs personal information, and the server uses that information to create and store a new account in the database.

[2584] "A means of having travelers input their purpose, duration, and areas of interest, and then using a generative AI model to suggest the optimal travel plan" refers to the process of having travelers input their purpose, duration, and areas of interest, and then using a generative AI model to suggest the optimal travel plan based on that data.

[2585] "Means for translating input text into multiple languages ​​and displaying and playing the results aloud" refers to a function that translates text input by a traveler into different languages ​​in real time, and not only displays the results on the device, but also plays them aloud if necessary.

[2586] "Means for calculating the optimal travel route based on the traveler's current location information and destination information and displaying the results" refers to the process in which, when a traveler inputs their current location and destination, the server calculates the optimal travel route and displays the results on the terminal.

[2587] "Means to explain local rules of the destination and provide emergency support" is a function that provides travelers with advance information about specific local rules of the destination they will be visiting, and provides appropriate support information and advice in real time in the event of an emergency.

[2588] "Means of linking with other domestic and international software and sharing user information" refers to the process of linking with other domestic and international applications via APIs, etc., and safely sharing user information.

[2589] This invention is software for mobile information terminals that helps foreign travelers to have a comfortable trip within Japan. This software has functions such as creating a travel account for travelers, proposing travel plans, translating, calculating travel routes, explaining local rules, and linking with other domestic and international software.

[2590] Account creation method

[2591] 1. Initial Setup

[2592] Terminal: When a traveler uses the application for the first time, an initial setup screen will be displayed.

[2593] User: Select your language and enter your personal information such as name, email address and password.

[2594] Server: Receives the personal information sent by the user and stores the new user account in the database. The server generates a unique user ID based on this information and returns it to the user.

[2595] Travel plan proposal method

[2596] 1. Create a travel plan

[2597] Terminal: Provides the user with a travel plan creation screen, prompting them to input the purpose of the trip, the duration, and the genres of interest (e.g., history, nature, shopping, etc.).

[2598] User: Enter the purpose of the trip, duration, and genres of interest, and press the send button.

[2599] Server: Receives this information and uses a generative AI model (e.g., GPT-4) to generate an optimal travel plan based on the user's preferences. Specifically, it extracts and combines relevant tourist spot information from the database to generate the optimal plan.

[2600] Server: Sends the generated travel plan to the terminal and displays it to the user.

[2601] Translation tools

[2602] 1. Real-time translation

[2603] Device: Display the translation function screen, and set up a text input field and a display area for the translation results.

[2604] User: Enter the text you want to translate and press the translate button.

[2605] Server: Receives the text sent by the user and translates the text using, for example, the Google Translate API.

[2606] Server: Sends the translation results to the device.

[2607] Terminal: The translation results are displayed on the screen and can also be played aloud if necessary.

[2608] Travel route calculation method

[2609] 1. Route calculation

[2610] Terminal: Provides a screen for entering a destination.

[2611] User: Enter the destination and press the route calculation button.

[2612] Server: Obtains the user's current location and destination information, and calculates the optimal route using, for example, the Google Maps API.

[2613] Server: Sends the calculation results to the terminal.

[2614] Device: Displays calculated route and navigation information on the screen.

[2615] Local Rules Explanation Method

[2616] 1. Provision of local rules

[2617] Terminal: Display the local rules explanation screen.

[2618] Users: Learn about the local rules of the places they visit.

[2619] Terminal: Also displays a screen that provides emergency support when the user is in trouble.

[2620] User: Press the emergency support button when in trouble.

[2621] Server: Receives emergency support requests, generates appropriate advice and assistance information, and sends it.

[2622] Terminal: Displays the support information sent from the server.

[2623] Linking with other domestic and international software

[2624] 1. App Integration

[2625] Device: Display the domestic and international software integration settings screen.

[2626] User: Select the software you want to link and set up the link.

[2627] Server: Uses the app integration API to securely share user information with selected software.

[2628] Terminal: Notifies the user that the connection is complete and displays the related software.

[2629] Specific examples

[2630] Specific examples of travel plan proposals

[2631] 1. User operations

[2632] User: You are planning a two-week trip for sightseeing and you are interested in historical sites.

[2633] 2. Server Processing

[2634] Server: Based on this information, a travel plan is generated that includes recommended spots in Tokyo, Kyoto, Osaka, etc.

[2635] Example plans:

[2636] Day 1: Tokyo Tower, Sensoji Temple

[2637] Day 2: Kinkakuji Temple and Fushimi Inari Shrine in Kyoto

[2638] 3. Displaying the terminal

[2639] Terminal: Presents the generated plan to the user and allows the user to customize the plan.

[2640] Examples of translation functions

[2641] 1. User operations

[2642] User: Type "Where is the nearest train station?"

[2643] 2. Server Processing

[2644] Server: Translate this text into Japanese and make it "Where is the nearest station?"

[2645] 3. Displaying the terminal

[2646] Terminal: Helps users ask for directions by displaying translation results and playing them aloud.

[2647] Prompt Sentence Examples

[2648] "I want to create software that suggests travel plans within Japan. Please explain the steps to generate the optimal travel plan when the user inputs the purpose of the trip, duration, and genres of interest."

[2649] The present invention allows travelers to easily complete complicated procedures and obtain necessary information in real time, making travel more comfortable and fulfilling.

[2650] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2651] Step 1:

[2652] Displaying the initial settings screen

[2653] Terminal: When a traveler launches the application, an initial setup screen appears. The terminal offers the user language selection, name, email address, and password entry fields.

[2654] Input: Personal information such as language, name, email address, and password

[2655] Output: Information entered on the initial setup screen

[2656] Step 2:

[2657] Entering and submitting personal information

[2658] User: The traveler selects a language, enters their name, email address, and password, and presses the send button.

[2659] Input: Personal information such as language, name, email address, and password

[2660] Output: Personal information is sent from the device to the server

[2661] Step 3:

[2662] Creating an account

[2663] Server: Saves a new user account in the database based on the received personal information. The server generates a unique user ID based on this information and returns it to the device.

[2664] Input: Personal information (language, name, email address, password)

[2665] Data processing: User ID generation, information storage in database

[2666] Output: Generated user ID

[2667] Step 4:

[2668] Display of travel plan creation screen

[2669] Terminal: Provides the user with a travel plan creation screen, displaying fields for inputting the purpose of the trip, duration, and genres of interest.

[2670] Input: None (screen input only)

[2671] Output: Travel plan creation screen

[2672] Step 5:

[2673] Enter and submit your travel purpose, duration, and genre

[2674] User: Enter the purpose of the trip, duration, and genres of interest, and press the send button.

[2675] Input: purpose of trip, duration, genre of interest

[2676] Output: Input information is sent from the device to the server

[2677] Step 6:

[2678] Generate a travel plan

[2679] Server: Based on the received travel information, a generative AI model (e.g., GPT-4) is used to generate an optimal travel plan based on the user's preferences. Specifically, it extracts and combines relevant tourist spot information from a database to create the optimal plan.

[2680] Input: purpose of trip, duration, areas of interest

[2681] Data Computing: Using generative AI models to generate travel plans

[2682] Output: Optimal travel plan

[2683] Step 7:

[2684] View and edit your travel plans

[2685] Terminal: Presents the generated travel plan to the user and provides an interface that allows the user to customize the plan.

[2686] Input: Optimal travel plan

[2687] Output: Customizable travel plan screen

[2688] Step 8:

[2689] Display of translation function screen

[2690] Device: Display the translation function screen, and set up a text input field and a display area for the translation results.

[2691] Input: None (screen input only)

[2692] Output: Translation function screen

[2693] Step 9:

[2694] Entering text and submitting translation requests

[2695] User: Enter the text you want to translate and press the send button.

[2696] Input: Text to translate

[2697] Output: The entered text is sent from the terminal to the server.

[2698] Step 10:

[2699] Text translation

[2700] Server: Receives the text sent by the user and translates the text using, for example, the Google Translate API.

[2701] Input: Text to translate

[2702] Data processing: Translate text using the translation API

[2703] Output: The translated text

[2704] Step 11:

[2705] Displaying translation results and playing audio

[2706] Terminal: The translation results are displayed on the screen and can also be played aloud if necessary.

[2707] Input: Translated text

[2708] Output: Display of translation results and audio playback

[2709] Step 12:

[2710] Display destination entry screen

[2711] Device: Displays a screen for entering your destination.

[2712] Input: None (screen input only)

[2713] Output: Destination input screen

[2714] Step 13:

[2715] Entering a destination and sending a route calculation request

[2716] User: Enter the destination and press the route calculation button.

[2717] Input: Destination information

[2718] Output: The entered destination information is sent from the device to the server.

[2719] Step 14:

[2720] Calculating the best route

[2721] Server: Obtains the user's current location and destination information, and calculates the optimal route using, for example, the Google Maps API.

[2722] Input: current location information, destination information

[2723] Data calculation: Calculate the optimal route using Google Maps API

[2724] Output: Optimal travel route

[2725] Step 15:

[2726] View route information

[2727] Device: Displays calculated route and navigation information on the screen.

[2728] Input: Optimal travel route

[2729] Output: Route information display screen

[2730] Step 16:

[2731] Display of local rules explanation screen

[2732] Terminal: Provides a screen explaining the local rules.

[2733] Input: None (screen input only)

[2734] Output: Local rules explanation screen

[2735] Step 17:

[2736] Display of emergency support screen

[2737] Terminal: Displays a screen that provides emergency support functions.

[2738] Input: None (screen input only)

[2739] Output: Emergency support screen

[2740] Step 18:

[2741] Submitting an Emergency Support Request

[2742] User: Press the emergency support button when in trouble.

[2743] Input: Emergency Support Request

[2744] Output: The request is sent to the server

[2745] Step 19:

[2746] Generate and submit support information

[2747] Server: Receives emergency support requests, generates appropriate advice and assistance information, and sends it to the device.

[2748] Input: Emergency Support Request

[2749] Data calculation: Generation of advice and assistance information according to the situation

[2750] Output: Support information

[2751] Step 20:

[2752] View support information

[2753] Terminal: Displays the support information sent from the server.

[2754] Input: Support Information

[2755] Output: Support information display screen

[2756] Step 21:

[2757] Display the app link setting screen

[2758] Device: Display the domestic and international software integration settings screen.

[2759] Input: None (screen input only)

[2760] Output: App link setting screen

[2761] Step 22:

[2762] Selecting and setting up linked apps

[2763] User: Select the software you want to link and set up the link.

[2764] Input: Selected linked application information

[2765] Output: Sending information from the device to the server

[2766] Step 23:

[2767] Sharing of User Information

[2768] Server: Uses the app integration API to securely share user information with selected software.

[2769] Input: Selected linked application information, user information

[2770] Data processing: Share user information using the collaboration API

[2771] Output: Connection completion information

[2772] Step 24:

[2773] Viewing integration notifications and related software

[2774] Terminal: Notifies the user that the connection is complete and displays the related software.

[2775] Input: Connection completion information

[2776] Output: Link notification and related software list

[2777] The above are the specific processing steps of the system. At each step, the user, terminal, and server cooperate to provide services that meet the needs of travelers.

[2778] (Application example 1)

[2779] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2780] Today's travelers need support not only in selecting tourist destinations and planning routes, but also in choosing food and learning dining etiquette based on local culture. Foreign travelers, in particular, are prone to experiencing inconveniences due to language barriers and cultural differences, and often have difficulty obtaining information about and choosing food. This can limit their travel experience and cause significant stress for travelers. Therefore, an integrated solution is needed to help travelers enjoy their meals while understanding local food culture, enabling them to have a comfortable trip.

[2781] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2782] In this invention, the server includes means for creating an account for a traveler, means for proposing a travel plan, means for providing translations in multiple languages, means for calculating a travel route, means for explaining local rules, means for linking with other domestic applications, means for proposing a meal plan based on the traveler's food preferences, means for calculating a travel route to a local restaurant, means for translating restaurant menus and dish descriptions, and means for explaining local rules regarding meals. This enables travelers to enjoy optimal travel plans and dining experiences without experiencing language or cultural barriers, and to have a comfortable and fulfilling trip.

[2783] "Means for creating a traveller's account" refers to a system whereby a traveller enters personal information and food preferences and stores them in a database.

[2784] The "means for suggesting travel plans" is a system that includes an algorithm that generates optimal travel plans based on the traveler's purpose, duration, and areas of interest. 【2785...

Claims

1. A smartphone application that supports travel within Japan for foreign tourists. a means for creating a traveller account; A means of proposing travel plans; a means of providing translations in multiple languages; a means for calculating a travel route; A means of explaining local rules; A means of linking with other domestic applications, A system including:

2. 2. The system according to claim 1, wherein the traveler's account creation involves inputting personal information of the traveler and storing it in a database.

3. 2. The system according to claim 1, wherein the travel plan proposal is based on input of the traveler's purpose, period, and areas of interest, and an optimal plan is generated by a generation algorithm.

Images