System

Abstract

A system is provided.SOLUTION: A system for providing multilingual tourism guidance, the system comprising: means for inputting user data; means for acquiring location information from a user; means for generating recommended tourism information based on the location information using a generative AI; means for translating the generated information into a language set by the user; and means for notifying the user's terminal of the generated multilingual information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Inbound tourists often do not understand the local language and are unable to effectively obtain information about tourist attractions and places to eat. Furthermore, manually creating and updating tourist guides in multiple languages ​​is time-consuming and costly. This results in the problem that it takes time for tourists to obtain information about their destination, resulting in a decrease in satisfaction. Conventional methods also do not adequately provide individual guides that utilize tourists' location information. A system that solves these problems is needed. [Means for solving the problem]

[0005] To solve the above-mentioned problems, the present invention proposes a system for providing local tourist information in multiple languages. This system includes a means for inputting user data, a means for acquiring location information from the user, a means for generating recommended tourist information based on the location information using generative AI, a means for translating the generated information into a language set by the user, and a means for notifying the user of the generated multilingual information. The system also includes a means for transmitting the location information acquired from the user to a server and updating the location information by the server, and a means for transmitting the generated tourist information to the user's device as a push notification and displaying the push notification, thereby enabling the provision of appropriate tourist information in real time in a language that tourists can easily understand.

[0006] "User data" refers to information about a user, including, for example, name, gender, age, and language preference.

[0007] "Location information" is information indicating the user's current location, and includes latitude and longitude data.

[0008] "Generative AI" is artificial intelligence that uses massive amounts of data to generate appropriate outputs from given inputs.

[0009] "Tourist information" refers to detailed information about recommended places such as tourist attractions, tourist spots, and restaurants.

[0010] "Translation means" refers to a function that converts the generated information into a language set by the user.

[0011] "Notification means" refers to a means for notifying the user's terminal of the generated tourist information, and includes push notifications, etc.

[0012] A "server" is a device or system that stores and manages data over a network and provides information in response to requests from clients (terminals).

[0013] A "terminal" is a device used by a user to access the system, including a smartphone, tablet, laptop, etc.

[0014] "Push notification" is a technology that sends information from a server to a device in real time to notify the user. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] ---

[0037] The present invention relates to a system that provides local tourist information in multiple languages ​​to inbound tourists. In an embodiment of the present invention, a user can use a device such as a smartphone or tablet to receive local tourist information and recommended dining spots.

[0038] This system mainly consists of the following components:

[0039] 1. User data input method:

[0040] The user starts the app and enters initial information such as their name, gender, age, preferred language, etc. on the initial setup screen. This information is sent to the server and stored in a database.

[0041] 2. Location information acquisition means:

[0042] The device periodically obtains the user's current location using the GPS function. This location information is sent to the server, where it is updated as the user's latest location information.

[0043] 3. Information generation using generative AI:

[0044] Based on the received location information, the server uses generative AI to generate information about nearby tourist spots and restaurants. This information is multilingual and can be translated into the language selected by the user.

[0045] 4. Multilingual translation tools:

[0046] The server translates the generated tourist information into the language set by the user (e.g., English, Japanese, Chinese, etc.). The translated information is stored in a database and provided to the user as needed.

[0047] 5. Means of notification:

[0048] The server sends the translated information to the user's device as a push notification. The device displays the received push notification to the user in real time. The user can tap the notification to access a page with more information.

[0049] Specific examples

[0050] Example 1: Tourist information

[0051] 1. The user launches the app and selects Japanese.

[0052] 2. The device sends its current location to the server, specifying Asakusa as its location.

[0053] 3. The server uses generative AI to generate information about tourist spots around Asakusa (e.g., Sensoji Temple, Kaminarimon Gate) and translates it into Japanese.

[0054] 4. The server creates a push notification saying "Explore Asakusa: Visit Sensoji Temple and Kaminarimon Gate!" and sends it to the device.

[0055] 5. The device displays a notification that the user can tap to access a page with more information.

[0056] Example 2: Dining recommendations

[0057] 1. The user launches the app and selects English.

[0058] 2. The device sends its current location to the server, specifying Ginza as its location.

[0059] 3. The server uses generative AI to generate information about restaurants around Ginza (e.g., high-end sushi restaurants) and translates it into English.

[0060] 4. The server creates a push notification saying "Explore Ginza: Try the premium sushi restaurants nearby!" and sends it to the device.

[0061] 5. The device displays a notification that the user can tap to access a page with more information.

[0062] This system allows inbound tourists to receive relevant tourist information in real time in a language they can understand.

[0063] The processing flow will be explained below.

[0064] ---

[0065] Step 1:

[0066] The user starts the app and enters their name, gender, age, and preferred language on the initial setup screen. The device then sends this information to the server.

[0067] Step 2:

[0068] The server stores the received user information in a database, generates a user ID and an authentication token, and sends them back to the device. The device stores the authentication token in local storage.

[0069] Step 3:

[0070] The device periodically acquires the user's location information using its GPS function, and the acquired location information (latitude and longitude) is sent to the server along with an authentication token.

[0071] Step 4:

[0072] The server updates the database with the received location information, stores the user's latest location information, and sends a request to the generative AI to generate information about nearby tourist attractions and restaurants based on the user's location information.

[0073] Step 5:

[0074] The generative AI generates information about tourist spots and restaurants in the area based on the location information it receives, and sends the generated information back to the server.

[0075] Step 6:

[0076] The server translates the generated information into the language selected by the user, and stores the translated information in a temporary database.

[0077] Step 7:

[0078] The server formats the translated information as a push notification message and prepares it to be sent to the user's device.

[0079] Step 8:

[0080] The server sends the push notification to the user's device, which then displays the received push notification to the user in real time.

[0081] Step 9:

[0082] Users can tap on the push notification to access a detailed information page, which displays detailed information about the recommended tourist spot or restaurant.

[0083] ---

[0084] Example 1

[0085] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0086] In recent years, the number of foreign tourists has increased, and there is a growing need to provide local tourist information in multiple languages. However, conventional methods often fail to adequately provide multilingual tourist information or real-time information. In addition, there are only a limited number of systems that provide appropriate tourist information based on the user's current location, and there is a need for an efficient guidance method that can help tourists avoid getting lost at their destination.

[0087] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0088] In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for translating the generated information into a language set by the user, means for sending the translated information to the user's device as a push notification, and means for the user to tap the push notification to access a detailed information page. This allows multilingual tourist information to be provided in real time, enabling the user to access appropriate tourist information based on their current location.

[0089] The "means for inputting user data" is an interface that allows a user to input initial information such as their name, gender, age, and preferred language, and then transmit that information to the server.

[0090] The "means for acquiring location information from the user" is a function for periodically acquiring the user's current location using the GPS function of the user's terminal and transmitting that information to the server.

[0091] "Means for generating recommended tourist information based on location information using generative AI" refers to the process in which a server receives location information and generates information about nearby tourist spots and restaurants using a generative AI model.

[0092] "Means for translating the generated information into the language set by the user" refers to the process for translating the generated tourist information into the language selected by the user, and uses translation technologies such as translation APIs.

[0093] The "means for sending translated information to the user's device as a push notification" is a system in which the server creates a push notification based on the translated tourist information and sends it to the user's device.

[0094] "Means for users to tap on a push notification to access the detailed information page" refers to an interface that allows users to access the detailed page of the generated tourist information by tapping on a push notification displayed on their device.

[0095] The present invention relates to a system that provides local tourist information in multiple languages. This system is realized by combining three main elements: a user, a terminal, and a server.

[0096] User Data Input

[0097] First, a user launches the app using a device such as a smartphone or tablet. When launching the app for the first time, the user enters initial information such as their name, gender, age, preferred language, etc. This information is sent via the device to the server, which then stores it in a database.

[0098] Hardware used: smartphone, tablet

[0099] Software used: Mobile applications, database systems (e.g., MySQL (registered trademark))

[0100] Specific prompt examples:

[0101] "Initial setup screen: Please enter your name, gender, age, and preferred language."

[0102] Location information acquisition

[0103] Next, the device periodically acquires the user's current location using its GPS function. This location information is sent from the device to the server, where it is stored in a database as the latest location information.

[0104] Hardware used: GPS-enabled smartphones and tablets

[0105] Software used: GPS API, location management system

[0106] Specific prompt examples:

[0107] "Retrieving your location. Please wait a moment."

[0108] Information Generation and Translation

[0109] Based on the received location information, the server uses a generative AI model (e.g., GPT-4 (registered trademark)) to generate information about nearby tourist attractions and restaurants. This generated information is multilingual, and the server translates it into the language selected by the user. The translated information is then stored in a database.

[0110] Hardware used: Server

[0111] Software used: Generative AI models (e.g., GPT-4), translation APIs (e.g., Google (registered trademark) Translation API, DeepL)

[0112] Specific prompt examples:

[0113] "The user is at Asakusa Station. Please tell me about nearby tourist spots in Japanese."

[0114] Sending push notifications

[0115] The server creates a push notification based on the translated information and sends it to the device, which displays the notification to the user in real time.

[0116] Hardware used: smartphone, tablet

[0117] Software used: Push notification system (e.g., Firebase Cloud Messaging)

[0118] Specific prompt examples:

[0119] "Sightseeing information for Sensoji Temple and Kaminarimon has been updated. Tap to see more details."

[0120] Access the details page

[0121] When a user taps on the received push notification, the device displays a page with detailed information about the related tourist spot. The server sends the requested detailed data to the device, which then displays the information.

[0122] Specific prompt examples:

[0123] "Visit our detailed information page. Check out the attractions and restaurants."

[0124] This system will enable inbound tourists to obtain relevant tourist information in real time in a language they can understand, greatly improving the efficiency and convenience of tourist guidance.

[0125] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0126] Step 1:

[0127] When a user launches the app for the first time, they enter their name, gender, age, and preferred language on the initial setup screen. Input: User's name, gender, age, and preferred language. Output: The input data is formatted within the application. The device sends this input data to the server. The server receives the data and stores it in a database.

[0128] Step 2:

[0129] The device periodically obtains the user's current location using its GPS function. Input: Current location information obtained from the GPS. Output: The obtained location information is formatted within the device and converted into a format that can be sent. The device sends the location information to the server, which updates it in the database.

[0130] Step 3:

[0131] The server sends a prompt to the generation AI based on the received location information. Input: Location information, prompt text to send to the generation AI. Output: Tourist information generated by the generation AI. The server receives the generated tourist information and begins the process of translating it into the language set by the user.

[0132] Step 4:

[0133] The server uses a translation API to translate the generated tourist information into the user's language. Input: Generated tourist information, language selected by the user. Output: Tourist information translated by the translation API. The translated information is saved in the database.

[0134] Step 5:

[0135] The server creates a push notification based on the translated information. Input: Translated tourist information. Output: Message in push notification format. The server sends the push notification to the device.

[0136] Step 6:

[0137] The device displays the received push notification to the user. Input: Push notification sent from the server. Output: Notification message displayed to the user by the device. The user taps the notification to access the detailed information page.

[0138] Step 7:

[0139] When a user taps on a push notification, the device displays a detailed information page for the related tourist attraction. Input: User operation (tap), request for detailed information page. Output: Detailed information page is displayed. The server sends the detailed information page data to the device, which then displays the detailed information.

[0140] Specifically, after the user launches the app for the first time and completes the setup, the server generates tourist information using the generative AI model, translates the information, and provides it to the user via push notification. By tapping the notification, the user can access detailed tourist information.

[0141] (Application example 1)

[0142] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0143] While systems exist that provide local tourist information and restaurant recommendations in multiple languages, until now they have been limited to personal devices such as smartphones and tablets. As a result, tourists had to operate their devices to receive real-time multilingual tourist information while on the move. This led to issues such as reduced convenience and user experience.

[0144] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0145] In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for translating the generated information into a language set by the user, means for notifying the user's means of transportation of the generated multilingual information, means for providing information using voice recognition and voice output, and means for displaying information on a display. This enables tourists to receive multilingual tourist information and recommended dining spots in real time by voice and on a display while in an autonomous vehicle.

[0146] "Multilingual" refers to the ability to provide information in multiple languages.

[0147] "Regional tourist guide" refers to a guide that provides tourist information and introduces tourist spots related to a specific region.

[0148] "User Data" refers to personal information such as the user's name, gender, age, and language preference.

[0149] "Location Information" refers to the current geographic location of a user or vehicle obtained using technologies such as GPS.

[0150] "Generative AI" refers to a system that uses artificial intelligence to analyze data and generate specific information or recommendations.

[0151] "Multilingual information" refers to a form in which generated information is provided in multiple languages.

[0152] "User's means of transportation" refers to the means of transportation used by tourists, in this case, particularly autonomous vehicles.

[0153] "Voice recognition" refers to the technology of analyzing the user's voice, understanding its meaning, and then performing operations or responding.

[0154] "Audio output" refers to a technique for conveying information to a user by voice.

[0155] "Display" refers to a device that visually displays information.

[0156] "Push notification" refers to a notification message that is automatically sent from a server to a user's device.

[0157] "Infotainment system" refers to a system that integrates information and entertainment, especially an in-vehicle system.

[0158] The present invention provides a system for providing local tourist information in multiple languages, and is particularly suitable for self-driving vehicles. Specific configurations and processes for implementing the invention are described below.

[0159] The system mainly consists of the following components:

[0160] 1. User data input method:

[0161] The user uses the vehicle's touch panel or voice input to enter initial information such as name, gender, age, and preferred language.

[0162] This information is stored on the on-board computer and transmitted to a server.

[0163] 2. Location information acquisition means:

[0164] The vehicle's GPS function is used to obtain the current location in real time and send it to a server.

[0165] 3. Information generation using generative AI:

[0166] The server receives the location information and uses a generative AI model (e.g., OpenAI® GPT-3®) to generate information about nearby tourist attractions and restaurants.

[0167] The server uses a multilingual translation API (e.g., Google Translate API) to translate the generated information into the user's language of choice.

[0168] 4. Means of notification:

[0169] The translated information is sent to the car's infotainment system as a push notification.

[0170] Information is provided to the user through the voice assistant and the display.

[0171] As a specific example, a process will be described in which the user gets on a vehicle and selects English.

[0172] Example 1: Tourist information

[0173] 1. User provides input:

[0174] Get on the vehicle and select English on the in-car display.

[0175] Enter the initial information using the touch panel or voice input.

[0176] 2. Get location information:

[0177] The vehicle recognizes its current location as being near Tokyo Tower.

[0178] Location information is sent to the server via GPS.

[0179] 3. Information Generation and Translation:

[0180] The server generates information about tourist spots around Tokyo Tower using a generative AI model.

[0181] Use the translation API to translate it into English.

[0182] 4. Notification of Results:

[0183] The server creates a notification saying "Explore Tokyo: Don't miss Tokyo Tower and Roppongi Hills!".

[0184] The information is pushed to the in-vehicle infotainment system, displayed on the display, and notified by the voice assistant.

[0185] Specific Example 2: Recommendations for Dining Places

[0186] 1. User Input:

[0187] Select Chinese in the in-vehicle voice assistant.

[0188] Enter the initial information.

[0189] 2. Obtain Location Information:

[0190] The vehicle recognizes the current location as Shinjuku.

[0191] The location information is sent to the server via GPS.

[0192] 3. Information Generation and Translation:

[0193] The server generates restaurant information around Shinjuku using the information generation AI model.

[0194] Use the translation API to translate it into Chinese.

[0195] 4. Notification of Results:

[0196] The server creates a notification saying "Try the popular barbecue restaurants here in Shinjuku!".

[0197] Information is pushed to the in-car infotainment system, displayed on the display, and notified via the voice assistant.

[0198] Prompt Sentence Examples

[0199] Prompt: "Your current location is Shinjuku. Please generate information about nearby tourist spots and translate it into Chinese."

[0200] Expected output: "Shinjuku Station Area has many attractions, including... (specific spot information)"

[0201] This system will enable tourists to receive real-time multilingual tourist information and restaurant recommendations while they are in an autonomous vehicle, greatly improving convenience.

[0202] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0203] Step 1:

[0204] A user gets into a vehicle and uses a touch panel or voice assistant to input initial information such as name, gender, age, and preferred language. This user data is then stored in the vehicle's on-board computer and sent to the server. The input comes from the user, and the output is a dataset of the initial information.

[0205] Step 2:

[0206] The vehicle's GPS function is used to obtain the vehicle's current location in real time. The terminal that receives location information from the GPS sends this location information to the server. The input is location data from the GPS, and the output is the location information sent to the server.

[0207] Step 3:

[0208] Based on the location information received by the server, a generative AI model is used to generate information about nearby tourist attractions and restaurants. The server generates a prompt sentence and provides it as input to the AI ​​model. The input is the location information and the prompt sentence, and the output is the generated tourist information.

[0209] Step 4:

[0210] The server uses a multilingual translation API to translate the generated tourist information into the specified language. It passes the information and the target language as input to the translation API and obtains the translated information. The input is the generated tourist information and the target language, and the output is the translated information.

[0211] Step 5:

[0212] The server sends the translated tourist information to the vehicle's infotainment system as a push notification. The in-vehicle infotainment system receives the information and displays it on the vehicle's display, while also notifying the user via voice using the voice assistant. The input is the translated tourist information, and the output is the display and voice notification.

[0213] Step 6:

[0214] Users can access more information by tapping on the information displayed on the screen or asking a question to the voice assistant. This action allows them to obtain more detailed information or decide on the next action. The input is the user's tap or voice command, and the output is the provision of more information.

[0215] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0216] ---

[0217] The present invention relates to a system that provides local tourist information in multiple languages ​​to inbound tourists and combines it with an emotion engine that recognizes user emotions. In an embodiment of the present invention, a user uses a device such as a smartphone or tablet to receive local tourist information and restaurant recommendations, and is also provided with information customized based on their emotions.

[0218] This system mainly consists of the following components:

[0219] 1. User data input method:

[0220] The user starts the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. This information is sent to the server and stored in a database.

[0221] 2. Location information acquisition means:

[0222] The device periodically obtains the user's current location using the GPS function. This location information is sent to the server, where it is updated as the user's latest location information.

[0223] 3. Emotion engine means:

[0224] The device uses cameras and sensors to analyze the user's facial expressions and voice to recognize emotions, and the emotional information is sent to a server with the user's permission.

[0225] 4. Information generation methods using generative AI:

[0226] The server uses generative AI to generate information about nearby tourist spots and restaurants based on the location and emotion information it receives. The generated information is tailored to the user's emotions.

[0227] 5. Multilingual translation tools:

[0228] The server translates the generated tourist information into the language set by the user (e.g., English, Japanese, Chinese, etc.) and stores the translated information in a database.

[0229] 6. Means of notification:

[0230] The server sends the translated information to the user's device as a push notification, which is then displayed to the user in real time.

[0231] Specific examples

[0232] Example 1: Tourist information

[0233] 1. The user launches the app, selects Japanese, and allows emotional information collection.

[0234] 2. The device sends its current location to the server, indicating that it is in Asakusa. The emotion engine then analyzes the user's facial expression and determines that the user is relaxed.

[0235] 3. The server uses generative AI to generate information about tourist spots around Asakusa (e.g., Sensoji Temple, Kaminarimon Gate), creates recommendations for enjoying a relaxing sightseeing experience, and translates them into Japanese.

[0236] 4. The server creates a push notification saying "Relax and explore Asakusa: Take a leisurely stroll around Sensoji Temple!" and sends it to the device.

[0237] 5. The device displays a notification that the user can tap to access a page with more information.

[0238] Example 2: Dining recommendations

[0239] 1. The user launches the app, selects English, and allows emotional information collection.

[0240] 2. The device sends its current location to the server, indicating that it is in Ginza. The emotion engine then analyzes the user's voice and determines that the user is excited.

[0241] 3. The server uses generative AI to generate information about restaurants around Ginza (e.g., high-end sushi restaurants), creates recommendations to provide an exciting dining experience, and translates them into English.

[0242] 4. The server creates a push notification saying "Explore Ginza: Enjoy the thrilling experience at premium sushi restaurants nearby!" and sends it to the device.

[0243] 5. The device displays a notification that the user can tap to access a page with more information.

[0244] This system allows inbound tourists to receive relevant tourist information in real time in a language they understand, and also provides information that is tailored to their emotions, further enhancing their tourist experience and increasing their satisfaction.

[0245] The processing flow will be explained below.

[0246] --

[0247] Step 1:

[0248] The user starts the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. The device then sends this information to the server.

[0249] Step 2:

[0250] The server stores the received user information in a database, generates a user ID and an authentication token, and sends them back to the device. The device stores the authentication token in local storage.

[0251] Step 3:

[0252] The device periodically obtains the user's current location using its GPS function, and the obtained location information (latitude and longitude) is sent to the server along with an authentication token.

[0253] Step 4:

[0254] The server updates the received location information in a database and stores the user's latest location information.

[0255] Step 5:

[0256] The device uses cameras and sensors to analyze the user's facial expressions and voice to recognize emotions, and the recognized emotional information is sent to the server along with an authentication token.

[0257] Step 6:

[0258] The server stores the received emotional information in a database and provides it to the generative AI, which then generates information about nearby tourist spots and restaurants based on the location and emotional information.

[0259] Step 7:

[0260] The server translates the generated tourist information into the language selected by the user, and the translated information is saved in a database.

[0261] Step 8:

[0262] The server formats the translated information as a push notification message and prepares it to be sent to the user's device.

[0263] Step 9:

[0264] The server sends the push notification to the user's device, which then displays the received push notification to the user in real time.

[0265] Step 10:

[0266] Users can tap on the push notification to access a detailed information page, which displays detailed information about the recommended tourist spot or restaurant.

[0267] --

[0268] Example 2

[0269] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0270] Conventional tourist information systems were able to provide tourist information based on the user's location information, but lacked customization based on the user's emotional state. As a result, while there was a demand for information provision optimized for the user's emotions, there were issues with not being able to sufficiently increase user satisfaction. Other issues included a lack of multilingual support and the inability to provide information smoothly in real time.

[0271] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for acquiring and analyzing emotional information of the user, means for customizing the generated tourist information based on the emotional information, means for translating the generated information into a language set by the user, and means for notifying the user's terminal of the generated multilingual information. This makes it possible to provide tourist information in multiple languages ​​in real time according to the user's emotional state.

[0272] ---

[0273] ---

[0274] "User Data" refers to information such as a user's name, gender, age, language preference, and preferences regarding collection of emotional information.

[0275] "Location information" refers to information about the user's current location obtained by the GPS function of the device.

[0276] "Generative AI" refers to artificial intelligence that generates tourist information based on received input information and prompts.

[0277] "Emotion information" refers to information about the emotional state of the user that is acquired by analyzing the user's facial expressions and voice.

[0278] "Notification means" refers to a mechanism for sending and displaying the generated information as a push notification to the user's device.

[0279] "Multilingual information" refers to tourist information and recommendations translated into the language set by the user.

[0280] "Server" refers to a central processing unit that stores user data, analyzes location and emotional information, generates information using generative AI, and translates and notifies users of that information.

[0281] "Terminal" refers to a mobile information terminal such as a smartphone or tablet used by a user.

[0282] ---

[0283] The above are definitions of important words included in the scope of the patent claims.

[0284] ---

[0285] System Configuration

[0286] This invention is a system that provides multilingual local tourist information for inbound tourists, recognizing user emotions and providing customized information based on those emotions. Users access the system using devices such as smartphones or tablets to receive local tourist information and recommended dining spots. The main components of this system are a user data input means, a location information acquisition means, an emotion information acquisition means, an information generation means using generative AI, a multilingual translation means, and a notification means.

[0287] User Data Input

[0288] The user starts the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. This information is sent from the device to the server and stored in a database.

[0289] Location information acquisition

[0290] The device periodically acquires the user's current location using its GPS function, and this location information is sent to the server and updated in the server's database.

[0291] Emotional information acquisition

[0292] The device uses sensors such as a camera and microphone to collect the user's facial expressions and voice, and analyzes the emotional information. The analyzed emotional information is sent to a server and stored in a database.

[0293] Information generation by generative AI

[0294] The server uses a generative AI model to generate tourist information and restaurant recommendations based on the received location and emotion information. The generated information is optimized to match the user's emotions.

[0295] Prompt Sentence Examples

[0296] Please suggest some tourist spots in Asakusa for users who are relaxing.

[0297] Suggest a high-end sushi restaurant in Ginza for excited users.

[0298] Multilingual Translation

[0299] The generated information is translated by the server into the language set by the user, and the translated information is stored in a database.

[0300] Notifications and Indications

[0301] The server creates a push notification message from the translated information and sends it to the device. The device displays the received push notification to the user in real time. The user can tap the notification to access a page with more information.

[0302] Specific examples

[0303] Example 1: Tourist information

[0304] 1. The user launches the app, selects Japanese, and allows the collection of emotion information.

[0305] 2. The device sends its current location to the server, specifying Asakusa as its location.

[0306] 3. The device uses a camera to analyze the user's facial expressions and recognizes that they are relaxed.

[0307] 4. The server inputs a prompt to the generative AI model: "Please suggest tourist spots in Asakusa for a user who is relaxing."

[0308] 5. The generative AI model generates information about relaxing spots such as Sensoji Temple and Kaminarimon Gate.

[0309] 6. The server translates the generated information into Japanese and stores it in the database.

[0310] 7. The server creates a push notification saying "Relax and explore Asakusa: Take a leisurely stroll around Sensoji Temple!" and sends it to the device.

[0311] 8. The device displays a notification, and the user can tap the notification to access a page with more information.

[0312] Example 2: Dining recommendations

[0313] 1. The user launches the app, selects English, and allows emotional information collection.

[0314] 2. The device sends its current location to the server, specifying Ginza as its location.

[0315] 3. The device analyzes the user's voice using a microphone and recognizes that they are excited.

[0316] 4. The server inputs a prompt to the generative AI model: "Please suggest high-end sushi restaurants in Ginza for an excited user."

[0317] 5. A generative AI model generates restaurant information that provides exciting experiences, such as high-end sushi restaurants.

[0318] 6. The server translates the generated information into English and stores it in a database.

[0319] 7. The server creates a push notification saying "Explore Ginza: Enjoy the thrilling experience at premium sushi restaurants nearby!" and sends it to the device.

[0320] 8. The device displays a notification, and the user can tap the notification to access a page with more information.

[0321] This system allows inbound tourists to receive appropriate tourist information in real time in a language they can understand, and also provides information that is optimized to suit their emotions, further enriching their tourist experience and increasing their satisfaction.

[0322] ---

[0323] The above is the form for carrying out the invention.

[0324] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0325] ---

[0326] Step 1:

[0327] The user launches the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. Input data: name, gender, age, language, permission to collect emotional information. Output data: user data sent to the server. Specifically, the user enters the required information in each field and taps the send button. This causes the device to send the user data to the server, which then stores it in a database.

[0328] Step 2:

[0329] The device periodically obtains the user's current location using its GPS function. Input data: GPS information of the current location. Output data: Location information sent to the server. Specifically, the device periodically obtains GPS data in the background and sends it to the server. The server then stores the location information in a database and updates it with the latest location information.

[0330] Step 3:

[0331] The device uses a camera and microphone to collect the user's facial expressions and voice and recognizes emotions. Input data: user's facial expression data, voice data. Output data: emotional information sent to the server. Specifically, the device collects data in real time using the camera and microphone and identifies the user's emotions using an emotion analysis algorithm. The analysis results are sent to the server and stored in a database.

[0332] Step 4:

[0333] The server inputs a prompt sentence into the generative AI model based on the received location information and emotion information. Input data: location information, emotion information. Output data: generated tourist information. Specifically, the server generates a prompt sentence and inputs it into the generative AI model (e.g., "Please suggest tourist spots in Asakusa for a user who is relaxing"). The generative AI model generates appropriate tourist information and returns it to the server.

[0334] Step 5:

[0335] The server translates the generated tourist information into the language set by the user. Input data: tourist information. Output data: translated multilingual tourist information. Specifically, the server uses a translation engine to translate the generated information into the user's language (e.g., English, Japanese, Chinese, etc.). The translated information is stored in a database.

[0336] Step 6:

[0337] The server creates a push notification message from the translated information and sends it to the device. Input data: Translated multilingual tourist information. Output data: Push notification sent to the user's device. Specifically, the server creates the push notification text based on the translated information and sends it to the user's device as a push notification. The device displays the received notification to the user in real time.

[0338] Step 7:

[0339] The user taps the displayed notification to access the detailed information page. Input data: The body of the push notification. Output data: The display of the detailed information page. Specifically, when the user taps the notification, the device sends a request to the server to display the detailed information page. The server sends the detailed information to the device, and the device displays it to the user.

[0340] ---

[0341] The above is an explanation of the specific flow of the program processing of this system and each step.

[0342] (Application example 2)

[0343] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0344] Modern tourists, especially inbound tourists, face the challenge of finding it difficult to receive accurate and up-to-date tourist information in their native language. Another problem is that tourist information is generally generic, making it impossible to provide customized information tailored to the feelings and circumstances of individual tourists. This leads to a decline in the quality of the tourist experience and a lack of satisfaction.

[0345] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0346] In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for translating the generated information, means for notifying the generated multilingual information, means for using an emotion engine that recognizes the user's emotions, and means for customizing the tourist information based on the emotion information. This makes it possible to provide tourist information customized in multiple languages ​​in real time according to the emotions and state of each tourist.

[0347] "Means for inputting user data" refers to an interface through which a user inputs their name, gender, age, preferred language, and permission for collection of emotional information.

[0348] "Means for obtaining location information from a user" refers to a device or system that uses GPS functionality to automatically obtain the user's current location.

[0349] "Generative AI" refers to artificial intelligence technology for generating new information based on large amounts of data, and specifically includes models that perform natural language processing.

[0350] "Means for translating generated information" refers to a multilingual translation engine that translates information generated by generative AI into the user's native language.

[0351] "Means for notifying the generated multilingual information" refers to a system for sending the generated and translated information to the user's device as a push notification.

[0352] An "emotion engine that recognizes user emotions" refers to technology that uses cameras and sensors to analyze a user's facial expressions and voice and identify their emotions.

[0353] "Means for customizing tourist information based on emotional information" refers to a system that provides tourist information and recommended information appropriate to the user's state based on their emotional data.

[0354] System configuration and main processing

[0355] The present invention is a system that provides local tourist information in multiple languages ​​and also recognizes the user's emotions to provide customized information. The system consists of the following main components:

[0356] 1. User data input method

[0357] When users enter an autonomous vehicle or launch an application, they enter their name, gender, age, preferred language, and permission to collect emotional information through the interface. This information is sent to the vehicle's computer and stored in a database.

[0358] 2. Location information acquisition means

[0359] The device periodically obtains the user's current location using GPS, and this location information is sent to the vehicle's computer system for updates.

[0360] 3. Emotional Engine Means

[0361] The device uses a camera and microphone installed in the vehicle to capture the user's facial expressions and voice, which are then analyzed by an emotion recognition engine (e.g., Microsoft® Azure® Face API, Google Cloud Vision API) to generate emotion data, which is then sent to the vehicle's computer.

[0362] 4. Information generation methods using generative AI

[0363] The server uses generative AI (e.g., OpenAI GPT-4) to generate tourist information appropriate for the user based on the received location and emotion information. An example of input used for generation is a prompt sentence such as "Tell me where I can relax around Ginza."

[0364] 5. Multilingual Translation Tools

[0365] The server translates the generated information into the language set by the user using a multilingual translation engine (e.g., Microsoft Translator, Google Translate), and the translated information is stored in a database.

[0366] 6. Means of notification

[0367] The device or vehicle's interface system will send the translated information as a push notification, and the user will receive the information as audio or text and can select the tourist attractions or restaurants they want to visit.

[0368] Natural language explanations

[0369] The user data input means consists of an interface implementation that allows users to input their name, gender, language preference, etc. when entering an application or an autonomous vehicle. This information is sent to the server and used for subsequent customization processing.

[0370] The location information acquisition means acquires the user's current location using a GPS module and transmits that information to the server. This allows the location information to be constantly updated in real time, and tourist information is provided based on the latest information.

[0371] The emotion engine means uses a camera and microphone to capture the user's facial expressions and voice, and processes them with an emotion recognition engine such as Microsoft Azure Face API or Google Cloud Vision API. This data is sent to a server as emotion information, and the user's emotional state is measured.

[0372] The information generation method using generative AI generates tourist information based on location information and emotion information using generative AI such as OpenAI GPT-4. For example, it uses the prompt sentence "Tell me about places where I can relax around Ginza" to generate tourist information that is optimal for the user.

[0373] The multilingual translation means translates the generated tourist information into the language set by the user, using a multilingual translation engine such as Microsoft Translator or Google Translate, and stores the translated information in a database.

[0374] The notification means sends the translated information to the user's device or the interface system of the autonomous vehicle as a push notification, allowing the user to receive appropriate tourist information in real time.

[0375] Specific examples

[0376] 1. A Japanese tourist gets into a self-driving vehicle, selects Japanese, and allows emotional information to be collected.

[0377] 2. The vehicle's GPS verifies its current location as Ginza and transmits this information to the computer system.

[0378] 3. The emotion engine recognizes a relaxed state from facial expressions.

[0379] 4. The computer system inputs "places to relax around Ginza" as a prompt sentence into the generative AI, and generates appropriate tourist information.

[0380] 5. The generated information is translated into Japanese using a multilingual translation engine and stored in a database.

[0381] 6. The vehicle's interface system presents the location to the user as a "quiet and enjoyable place in Ginza."

[0382] This system allows users to receive the most appropriate tourist information based on their emotions and current location in real time in multiple languages.

[0383] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0384] Step 1:

[0385] When a user enters a vehicle, they enter their name, gender, age, preferred language, and permission to collect emotional information on a tablet or interface.

[0386] Input: User's name, gender, age, preferred language, and permission to collect emotional information.

[0387] Output: The input information is stored in the vehicle's database.

[0388] In this step, basic user data is collected and stored in a database for subsequent customization.

[0389] Step 2:

[0390] The device uses its GPS function to obtain the user's location information in real time and transmits it to the server.

[0391] Input: None (automatic location acquisition via GPS).

[0392] Output: Current location data (latitude, longitude) is sent to the server and location information is updated.

[0393] This process allows the user's current location to be determined in real time, allowing tourist information to be provided based on appropriate location information.

[0394] Step 3:

[0395] The device's camera and microphone are used to capture the user's facial expressions and voice, which are then analyzed using an emotion recognition engine.

[0396] Input: User's facial expression image data, voice data.

[0397] Output: Emotional data (e.g., relaxed state, excited state) is generated and sent to the server.

[0398] Here, raw data captured by a camera and microphone is processed by an emotion recognition engine and converted into concrete data representing the user's current emotional state.

[0399] Step 4:

[0400] The server integrates location information and emotional information and uses generative AI to generate tourist guides based on that information.

[0401] Input: location information (latitude, longitude), emotional information (e.g., relaxation state).

[0402] Output: Generated tourist information (text format).

[0403] The server uses a generative AI model (e.g., OpenAI GPT-4) to generate specific tourist information based on the input data, providing instructions to the AI ​​in the form of a prompt such as "Tell me where I can relax around Ginza."

[0404] Step 5:

[0405] The server translates the generated tourist information into the language set by the user.

[0406] Input: Generated tourist information (original language).

[0407] Output: Translated tourist information (user-defined language, e.g. Japanese).

[0408] Using a translation engine (e.g., Microsoft Translator, Google Translate), the generated tourist information is translated into the language set by the user and stored in a database.

[0409] Step 6:

[0410] The interface system of the terminal or vehicle sends the translated tourist information to the user as a push notification.

[0411] Input: Translated tourist information (user-preferred language).

[0412] Output: Tourist information is displayed on the user's device as a push notification.

[0413] Users can receive notifications and tap to access more information, including detailed directions to tourist attractions and restaurants.

[0414] This series of steps creates a system that can provide multilingual tourist information customized according to the user's emotions and current location.

[0415] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0416] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0417] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0418] [Second embodiment]

[0419] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0420] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0421] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0422] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0423] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0424] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0425] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0426] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0427] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0428] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0429] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0430] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0431] ---

[0432] The present invention relates to a system that provides local tourist information in multiple languages ​​to inbound tourists. In an embodiment of the present invention, a user can use a device such as a smartphone or tablet to receive local tourist information and recommended dining spots.

[0433] This system mainly consists of the following components:

[0434] 1. User data input method:

[0435] The user starts the app and enters initial information such as their name, gender, age, preferred language, etc. on the initial setup screen. This information is sent to the server and stored in a database.

[0436] 2. Location information acquisition means:

[0437] The device periodically obtains the user's current location using the GPS function. This location information is sent to the server, where it is updated as the user's latest location information.

[0438] 3. Information generation using generative AI:

[0439] Based on the received location information, the server uses generative AI to generate information about nearby tourist spots and restaurants. This information is multilingual and can be translated into the language selected by the user.

[0440] 4. Multilingual translation tools:

[0441] The server translates the generated tourist information into the language set by the user (e.g., English, Japanese, Chinese, etc.). The translated information is stored in a database and provided to the user as needed.

[0442] 5. Means of notification:

[0443] The server sends the translated information to the user's device as a push notification. The device displays the received push notification to the user in real time. The user can tap the notification to access a page with more information.

[0444] Specific examples

[0445] Example 1: Tourist information

[0446] 1. The user launches the app and selects Japanese.

[0447] 2. The device sends its current location to the server, specifying Asakusa as its location.

[0448] 3. The server uses generative AI to generate information about tourist spots around Asakusa (e.g., Sensoji Temple, Kaminarimon Gate) and translates it into Japanese.

[0449] 4. The server creates a push notification saying "Explore Asakusa: Visit Sensoji Temple and Kaminarimon Gate!" and sends it to the device.

[0450] 5. The device displays a notification that the user can tap to access a page with more information.

[0451] Example 2: Dining recommendations

[0452] 1. The user launches the app and selects English.

[0453] 2. The device sends its current location to the server, specifying Ginza as its location.

[0454] 3. The server uses generative AI to generate information about restaurants around Ginza (e.g., high-end sushi restaurants) and translates it into English.

[0455] 4. The server creates a push notification saying "Explore Ginza: Try the premium sushi restaurants nearby!" and sends it to the device.

[0456] 5. The device displays a notification that the user can tap to access a page with more information.

[0457] This system allows inbound tourists to receive relevant tourist information in real time in a language they can understand.

[0458] The processing flow will be explained below.

[0459] ---

[0460] Step 1:

[0461] The user starts the app and enters their name, gender, age, and preferred language on the initial setup screen. The device then sends this information to the server.

[0462] Step 2:

[0463] The server stores the received user information in a database, generates a user ID and an authentication token, and sends them back to the device. The device stores the authentication token in local storage.

[0464] Step 3:

[0465] The device periodically acquires the user's location information using its GPS function, and the acquired location information (latitude and longitude) is sent to the server along with an authentication token.

[0466] Step 4:

[0467] The server updates the database with the received location information, stores the user's latest location information, and sends a request to the generative AI to generate information about nearby tourist attractions and restaurants based on the user's location information.

[0468] Step 5:

[0469] The generative AI generates information about tourist spots and restaurants in the area based on the location information it receives, and sends the generated information back to the server.

[0470] Step 6:

[0471] The server translates the generated information into the language selected by the user, and stores the translated information in a temporary database.

[0472] Step 7:

[0473] The server formats the translated information as a push notification message and prepares it to be sent to the user's device.

[0474] Step 8:

[0475] The server sends the push notification to the user's device, which then displays the received push notification to the user in real time.

[0476] Step 9:

[0477] Users can tap on the push notification to access a detailed information page, which displays detailed information about the recommended tourist spot or restaurant.

[0478] ---

[0479] Example 1

[0480] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0481] In recent years, the number of foreign tourists has increased, and there is a growing need to provide local tourist information in multiple languages. However, conventional methods often fail to adequately provide multilingual tourist information or real-time information. In addition, there are only a limited number of systems that provide appropriate tourist information based on the user's current location, and there is a need for an efficient guidance method that can help tourists avoid getting lost at their destination.

[0482] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0483] In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for translating the generated information into a language set by the user, means for sending the translated information to the user's device as a push notification, and means for the user to tap the push notification to access a detailed information page. This allows multilingual tourist information to be provided in real time, enabling the user to access appropriate tourist information based on their current location.

[0484] The "means for inputting user data" is an interface that allows a user to input initial information such as their name, gender, age, and preferred language, and then transmit that information to the server.

[0485] The "means for acquiring location information from the user" is a function for periodically acquiring the user's current location using the GPS function of the user's terminal and transmitting that information to the server.

[0486] "Means for generating recommended tourist information based on location information using generative AI" refers to the process in which a server receives location information and generates information about nearby tourist spots and restaurants using a generative AI model.

[0487] "Means for translating the generated information into the language set by the user" refers to the process for translating the generated tourist information into the language selected by the user, and uses translation technologies such as translation APIs.

[0488] The "means for sending translated information to the user's device as a push notification" is a system in which the server creates a push notification based on the translated tourist information and sends it to the user's device.

[0489] "Means for users to tap on a push notification to access the detailed information page" refers to an interface that allows users to access the detailed page of the generated tourist information by tapping on a push notification displayed on their device.

[0490] The present invention relates to a system that provides local tourist information in multiple languages. This system is realized by combining three main elements: a user, a terminal, and a server.

[0491] User Data Input

[0492] First, a user launches the app using a device such as a smartphone or tablet. When launching the app for the first time, the user enters initial information such as their name, gender, age, preferred language, etc. This information is sent via the device to the server, which then stores it in a database.

[0493] Hardware used: smartphone, tablet

[0494] Software used: Mobile applications, database systems (e.g., MySQL)

[0495] Specific prompt examples:

[0496] "Initial setup screen: Please enter your name, gender, age, and preferred language."

[0497] Location information acquisition

[0498] Next, the device periodically acquires the user's current location using its GPS function. This location information is sent from the device to the server, where it is stored in a database as the latest location information.

[0499] Hardware used: GPS-enabled smartphones and tablets

[0500] Software used: GPS API, location management system

[0501] Specific prompt examples:

[0502] "Retrieving your location. Please wait a moment."

[0503] Information Generation and Translation

[0504] Based on the received location information, the server uses a generative AI model (e.g., GPT-4) to generate information about nearby tourist attractions and restaurants. This generated information is multilingual, and the server translates it into the language selected by the user. The translated information is then stored in a database.

[0505] Hardware used: Server

[0506] Software used: Generative AI models (e.g., GPT-4), translation APIs (e.g., Google Translate API, DeepL)

[0507] Specific prompt examples:

[0508] "The user is at Asakusa Station. Please tell me about nearby tourist spots in Japanese."

[0509] Sending push notifications

[0510] The server creates a push notification based on the translated information and sends it to the device, which displays the notification to the user in real time.

[0511] Hardware used: smartphone, tablet

[0512] Software used: Push notification system (e.g., Firebase Cloud Messaging)

[0513] Specific prompt examples:

[0514] "Sightseeing information for Sensoji Temple and Kaminarimon has been updated. Tap to see more details."

[0515] Access the details page

[0516] When a user taps on the received push notification, the device displays a page with detailed information about the related tourist spot. The server sends the requested detailed data to the device, which then displays the information.

[0517] Specific prompt examples:

[0518] "Visit our detailed information page. Check out the attractions and restaurants."

[0519] This system will enable inbound tourists to obtain relevant tourist information in real time in a language they can understand, greatly improving the efficiency and convenience of tourist guidance.

[0520] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0521] Step 1:

[0522] When a user launches the app for the first time, they enter their name, gender, age, and preferred language on the initial setup screen. Input: User's name, gender, age, and preferred language. Output: The input data is formatted within the application. The device sends this input data to the server. The server receives the data and stores it in a database.

[0523] Step 2:

[0524] The device periodically obtains the user's current location using its GPS function. Input: Current location information obtained from the GPS. Output: The obtained location information is formatted within the device and converted into a format that can be sent. The device sends the location information to the server, which updates it in the database.

[0525] Step 3:

[0526] The server sends a prompt to the generation AI based on the received location information. Input: Location information, prompt text to send to the generation AI. Output: Tourist information generated by the generation AI. The server receives the generated tourist information and begins the process of translating it into the language set by the user.

[0527] Step 4:

[0528] The server uses a translation API to translate the generated tourist information into the user's language. Input: Generated tourist information, language selected by the user. Output: Tourist information translated by the translation API. The translated information is saved in the database.

[0529] Step 5:

[0530] The server creates a push notification based on the translated information. Input: Translated tourist information. Output: Message in push notification format. The server sends the push notification to the device.

[0531] Step 6:

[0532] The device displays the received push notification to the user. Input: Push notification sent from the server. Output: Notification message displayed to the user by the device. The user taps the notification to access the detailed information page.

[0533] Step 7:

[0534] When a user taps on a push notification, the device displays a detailed information page for the related tourist attraction. Input: User operation (tap), request for detailed information page. Output: Detailed information page is displayed. The server sends the detailed information page data to the device, which then displays the detailed information.

[0535] Specifically, after the user launches the app for the first time and completes the setup, the server generates tourist information using the generative AI model, translates the information, and provides it to the user via push notification. By tapping the notification, the user can access detailed tourist information.

[0536] (Application example 1)

[0537] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0538] While systems exist that provide local tourist information and restaurant recommendations in multiple languages, until now they have been limited to personal devices such as smartphones and tablets. As a result, tourists had to operate their devices to receive real-time multilingual tourist information while on the move. This led to issues such as reduced convenience and user experience.

[0539] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0540] In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for translating the generated information into a language set by the user, means for notifying the user's means of transportation of the generated multilingual information, means for providing information using voice recognition and voice output, and means for displaying information on a display. This enables tourists to receive multilingual tourist information and recommended dining spots in real time by voice and on a display while in an autonomous vehicle.

[0541] "Multilingual" refers to the ability to provide information in multiple languages.

[0542] "Regional tourist guide" refers to a guide that provides tourist information and introduces tourist spots related to a specific region.

[0543] "User Data" refers to personal information such as the user's name, gender, age, and language preference.

[0544] "Location Information" refers to the current geographic location of a user or vehicle obtained using technologies such as GPS.

[0545] "Generative AI" refers to a system that uses artificial intelligence to analyze data and generate specific information or recommendations.

[0546] "Multilingual information" refers to a form in which generated information is provided in multiple languages.

[0547] "User's means of transportation" refers to the means of transportation used by tourists, in this case, particularly autonomous vehicles.

[0548] "Voice recognition" refers to the technology of analyzing the user's voice, understanding its meaning, and then performing operations or responding.

[0549] "Audio output" refers to a technique for conveying information to a user by voice.

[0550] "Display" refers to a device that visually displays information.

[0551] "Push notification" refers to a notification message that is automatically sent from a server to a user's device.

[0552] "Infotainment system" refers to a system that integrates information and entertainment, especially an in-vehicle system.

[0553] The present invention provides a system for providing local tourist information in multiple languages, and is particularly suitable for self-driving vehicles. Specific configurations and processes for implementing the invention are described below.

[0554] The system mainly consists of the following components:

[0555] 1. User data input method:

[0556] The user uses the vehicle's touch panel or voice input to enter initial information such as name, gender, age, and preferred language.

[0557] This information is stored on the on-board computer and transmitted to a server.

[0558] 2. Location information acquisition means:

[0559] The vehicle's GPS function is used to obtain the current location in real time and send it to a server.

[0560] 3. Information generation using generative AI:

[0561] The server receives the location information and uses a generative AI model (e.g., OpenAI GPT-3) to generate information about nearby tourist attractions and restaurants.

[0562] The server uses a multilingual translation API (e.g., Google Translate API) to translate the generated information into the user's language of choice.

[0563] 4. Means of notification:

[0564] The translated information is sent to the car's infotainment system as a push notification.

[0565] Information is provided to the user through the voice assistant and the display.

[0566] As a specific example, a process will be described in which the user gets on a vehicle and selects English.

[0567] Example 1: Tourist information

[0568] 1. User provides input:

[0569] Get on the vehicle and select English on the in-car display.

[0570] Enter the initial information using the touch panel or voice input.

[0571] 2. Get location information:

[0572] The vehicle recognizes its current location as being near Tokyo Tower.

[0573] Location information is sent to the server via GPS.

[0574] 3. Information Generation and Translation:

[0575] The server generates information about tourist spots around Tokyo Tower using a generative AI model.

[0576] Translate into English using the translation API.

[0577] 4. Notification of Results:

[0578] The server creates a notification saying "Explore Tokyo: Don't miss Tokyo Tower and Roppongi Hills!"

[0579] Information is pushed to the in-car infotainment system, displayed on the display, and notified via the voice assistant.

[0580] Example 2: Dining recommendations

[0581] 1. User provides input:

[0582] Select Chinese on the in-car voice assistant.

[0583] Enter the initial information.

[0584] 2. Obtain location information:

[0585] The vehicle recognizes its current location as Shinjuku.

[0586] Location information is sent to the server via GPS.

[0587] 3. Information generation and translation:

[0588] The server generates restaurant information around Shinjuku using a generation AI model.

[0589] Translate it into Chinese using a translation API.

[0590] 4. Notification of results:

[0591] The server creates a notification saying "Try the popular barbecue restaurants here in Shinjuku!".

[0592] The information is pushed to the in-vehicle infotainment system, displayed on the display, and notified by the voice assistant.

[0593] Example of prompt text

[0594] Prompt: "The current location is Shinjuku. Generate tourist spot information around it and translate it into Chinese."

[0595] Expected output: "There are many places worth visiting around Shinjuku Station, including... (specific spot information)"

[0596] With this system, even while tourists are riding in an autonomous vehicle, it is possible to provide real-time multilingual tourist information and recommended information on dining places, significantly improving convenience.

[0597] The flow of specific processing in Application Example 1 will be described using FIG. 12.

[0598] Step 1:

[0599] A user gets into a vehicle and uses a touch panel or voice assistant to input initial information such as name, gender, age, and preferred language. This user data is then stored in the vehicle's on-board computer and sent to the server. The input comes from the user, and the output is a dataset of the initial information.

[0600] Step 2:

[0601] The vehicle's GPS function is used to obtain the vehicle's current location in real time. The terminal that receives location information from the GPS sends this location information to the server. The input is location data from the GPS, and the output is the location information sent to the server.

[0602] Step 3:

[0603] Based on the location information received by the server, a generative AI model is used to generate information about nearby tourist attractions and restaurants. The server generates a prompt sentence and provides it as input to the AI ​​model. The input is the location information and the prompt sentence, and the output is the generated tourist information.

[0604] Step 4:

[0605] The server uses a multilingual translation API to translate the generated tourist information into the specified language. It passes the information and the target language as input to the translation API and obtains the translated information. The input is the generated tourist information and the target language, and the output is the translated information.

[0606] Step 5:

[0607] The server sends the translated tourist information to the vehicle's infotainment system as a push notification. The in-vehicle infotainment system receives the information and displays it on the vehicle's display, while also notifying the user via voice using the voice assistant. The input is the translated tourist information, and the output is the display and voice notification.

[0608] Step 6:

[0609] Users can access more information by tapping on the information displayed on the screen or asking a question to the voice assistant. This action allows them to obtain more detailed information or decide on the next action. The input is the user's tap or voice command, and the output is the provision of more information.

[0610] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0611] ---

[0612] The present invention relates to a system that provides local tourist information in multiple languages ​​to inbound tourists and combines it with an emotion engine that recognizes user emotions. In an embodiment of the present invention, a user uses a device such as a smartphone or tablet to receive local tourist information and restaurant recommendations, and is also provided with information customized based on their emotions.

[0613] This system mainly consists of the following components:

[0614] 1. User data input method:

[0615] The user starts the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. This information is sent to the server and stored in a database.

[0616] 2. Location information acquisition means:

[0617] The device periodically obtains the user's current location using the GPS function. This location information is sent to the server, where it is updated as the user's latest location information.

[0618] 3. Emotion engine means:

[0619] The device uses cameras and sensors to analyze the user's facial expressions and voice to recognize emotions, and the emotional information is sent to a server with the user's permission.

[0620] 4. Information generation methods using generative AI:

[0621] The server uses generative AI to generate information about nearby tourist spots and restaurants based on the location and emotion information it receives. The generated information is tailored to the user's emotions.

[0622] 5. Multilingual translation tools:

[0623] The server translates the generated tourist information into the language set by the user (e.g., English, Japanese, Chinese, etc.) and stores the translated information in a database.

[0624] 6. Means of notification:

[0625] The server sends the translated information to the user's device as a push notification, which is then displayed to the user in real time.

[0626] Specific examples

[0627] Example 1: Tourist information

[0628] 1. The user launches the app, selects Japanese, and allows emotional information collection.

[0629] 2. The device sends its current location to the server, indicating that it is in Asakusa. The emotion engine then analyzes the user's facial expression and determines that the user is relaxed.

[0630] 3. The server uses generative AI to generate information about tourist spots around Asakusa (e.g., Sensoji Temple, Kaminarimon Gate), creates recommendations for enjoying a relaxing sightseeing experience, and translates them into Japanese.

[0631] 4. The server creates a push notification saying "Relax and explore Asakusa: Take a leisurely stroll around Sensoji Temple!" and sends it to the device.

[0632] 5. The device displays a notification that the user can tap to access a page with more information.

[0633] Example 2: Dining recommendations

[0634] 1. The user launches the app, selects English, and allows emotional information collection.

[0635] 2. The device sends its current location to the server, indicating that it is in Ginza. The emotion engine then analyzes the user's voice and determines that the user is excited.

[0636] 3. The server uses generative AI to generate information about restaurants around Ginza (e.g., high-end sushi restaurants), creates recommendations to provide an exciting dining experience, and translates them into English.

[0637] 4. The server creates a push notification saying "Explore Ginza: Enjoy the thrilling experience at premium sushi restaurants nearby!" and sends it to the device.

[0638] 5. The device displays a notification that the user can tap to access a page with more information.

[0639] This system allows inbound tourists to receive relevant tourist information in real time in a language they understand, and also provides information that is tailored to their emotions, further enhancing their tourist experience and increasing their satisfaction.

[0640] The processing flow will be explained below.

[0641] --

[0642] Step 1:

[0643] The user starts the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. The device then sends this information to the server.

[0644] Step 2:

[0645] The server stores the received user information in a database, generates a user ID and an authentication token, and sends them back to the device. The device stores the authentication token in local storage.

[0646] Step 3:

[0647] The device periodically obtains the user's current location using its GPS function, and the obtained location information (latitude and longitude) is sent to the server along with an authentication token.

[0648] Step 4:

[0649] The server updates the received location information in a database and stores the user's latest location information.

[0650] Step 5:

[0651] The device uses cameras and sensors to analyze the user's facial expressions and voice to recognize emotions, and the recognized emotional information is sent to the server along with an authentication token.

[0652] Step 6:

[0653] The server stores the received emotional information in a database and provides it to the generative AI, which then generates information about nearby tourist spots and restaurants based on the location and emotional information.

[0654] Step 7:

[0655] The server translates the generated tourist information into the language selected by the user, and the translated information is saved in a database.

[0656] Step 8:

[0657] The server formats the translated information as a push notification message and prepares it to be sent to the user's device.

[0658] Step 9:

[0659] The server sends the push notification to the user's device, which then displays the received push notification to the user in real time.

[0660] Step 10:

[0661] Users can tap on the push notification to access a detailed information page, which displays detailed information about the recommended tourist spot or restaurant.

[0662] --

[0663] Example 2

[0664] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0665] Conventional tourist information systems were able to provide tourist information based on the user's location information, but lacked customization based on the user's emotional state. As a result, while there was a demand for information provision optimized for the user's emotions, there were issues with not being able to sufficiently increase user satisfaction. Other issues included a lack of multilingual support and the inability to provide information smoothly in real time.

[0666] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for acquiring and analyzing emotional information of the user, means for customizing the generated tourist information based on the emotional information, means for translating the generated information into a language set by the user, and means for notifying the user's terminal of the generated multilingual information. This makes it possible to provide tourist information in multiple languages ​​in real time according to the user's emotional state.

[0667] ---

[0668] ---

[0669] "User Data" refers to information such as a user's name, gender, age, language preference, and preferences regarding collection of emotional information.

[0670] "Location information" refers to information about the user's current location obtained by the GPS function of the device.

[0671] "Generative AI" refers to artificial intelligence that generates tourist information based on received input information and prompts.

[0672] "Emotion information" refers to information about the emotional state of the user that is acquired by analyzing the user's facial expressions and voice.

[0673] "Notification means" refers to a mechanism for sending and displaying the generated information as a push notification to the user's device.

[0674] "Multilingual information" refers to tourist information and recommendations translated into the language set by the user.

[0675] "Server" refers to a central processing unit that stores user data, analyzes location and emotional information, generates information using generative AI, and translates and notifies users of that information.

[0676] "Terminal" refers to a mobile information terminal such as a smartphone or tablet used by a user.

[0677] ---

[0678] The above are definitions of important words included in the scope of the patent claims.

[0679] ---

[0680] System Configuration

[0681] This invention is a system that provides multilingual local tourist information for inbound tourists, recognizing user emotions and providing customized information based on those emotions. Users access the system using devices such as smartphones or tablets to receive local tourist information and recommended dining spots. The main components of this system are a user data input means, a location information acquisition means, an emotion information acquisition means, an information generation means using generative AI, a multilingual translation means, and a notification means.

[0682] User Data Input

[0683] The user starts the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. This information is sent from the device to the server and stored in a database.

[0684] Location information acquisition

[0685] The device periodically acquires the user's current location using its GPS function, and this location information is sent to the server and updated in the server's database.

[0686] Emotional information acquisition

[0687] The device uses sensors such as a camera and microphone to collect the user's facial expressions and voice, and analyzes the emotional information. The analyzed emotional information is sent to a server and stored in a database.

[0688] Information generation by generative AI

[0689] The server uses a generative AI model to generate tourist information and restaurant recommendations based on the received location and emotion information. The generated information is optimized to match the user's emotions.

[0690] Prompt Sentence Examples

[0691] Please suggest some tourist spots in Asakusa for users who are relaxing.

[0692] Suggest a high-end sushi restaurant in Ginza for excited users.

[0693] Multilingual Translation

[0694] The generated information is translated by the server into the language set by the user, and the translated information is stored in a database.

[0695] Notifications and Indications

[0696] The server creates a push notification message from the translated information and sends it to the device. The device displays the received push notification to the user in real time. The user can tap the notification to access a page with more information.

[0697] Specific examples

[0698] Example 1: Tourist information

[0699] 1. The user launches the app, selects Japanese, and allows the collection of emotion information.

[0700] 2. The device sends its current location to the server, specifying Asakusa as its location.

[0701] 3. The device uses a camera to analyze the user's facial expressions and recognizes that they are relaxed.

[0702] 4. The server inputs a prompt to the generative AI model: "Please suggest tourist spots in Asakusa for a user who is relaxing."

[0703] 5. The generative AI model generates information about relaxing spots such as Sensoji Temple and Kaminarimon Gate.

[0704] 6. The server translates the generated information into Japanese and stores it in the database.

[0705] 7. The server creates a push notification saying "Relax and explore Asakusa: Take a leisurely stroll around Sensoji Temple!" and sends it to the device.

[0706] 8. The device displays a notification, and the user can tap the notification to access a page with more information.

[0707] Example 2: Dining recommendations

[0708] 1. The user launches the app, selects English, and allows emotional information collection.

[0709] 2. The device sends its current location to the server, specifying Ginza as its location.

[0710] 3. The device analyzes the user's voice using a microphone and recognizes that they are excited.

[0711] 4. The server inputs a prompt to the generative AI model: "Please suggest high-end sushi restaurants in Ginza for an excited user."

[0712] 5. A generative AI model generates restaurant information that provides exciting experiences, such as high-end sushi restaurants.

[0713] 6. The server translates the generated information into English and stores it in a database.

[0714] 7. The server creates a push notification saying "Explore Ginza: Enjoy the thrilling experience at premium sushi restaurants nearby!" and sends it to the device.

[0715] 8. The device displays a notification, and the user can tap the notification to access a page with more information.

[0716] This system allows inbound tourists to receive appropriate tourist information in real time in a language they can understand, and also provides information that is optimized to suit their emotions, further enriching their tourist experience and increasing their satisfaction.

[0717] ---

[0718] The above is the form for carrying out the invention.

[0719] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0720] ---

[0721] Step 1:

[0722] The user launches the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. Input data: name, gender, age, language, permission to collect emotional information. Output data: user data sent to the server. Specifically, the user enters the required information in each field and taps the send button. This causes the device to send the user data to the server, which then stores it in a database.

[0723] Step 2:

[0724] The device periodically obtains the user's current location using its GPS function. Input data: GPS information of the current location. Output data: Location information sent to the server. Specifically, the device periodically obtains GPS data in the background and sends it to the server. The server then stores the location information in a database and updates it with the latest location information.

[0725] Step 3:

[0726] The device uses a camera and microphone to collect the user's facial expressions and voice and recognizes emotions. Input data: user's facial expression data, voice data. Output data: emotional information sent to the server. Specifically, the device collects data in real time using the camera and microphone and identifies the user's emotions using an emotion analysis algorithm. The analysis results are sent to the server and stored in a database.

[0727] Step 4:

[0728] The server inputs a prompt sentence into the generative AI model based on the received location information and emotion information. Input data: location information, emotion information. Output data: generated tourist information. Specifically, the server generates a prompt sentence and inputs it into the generative AI model (e.g., "Please suggest tourist spots in Asakusa for a user who is relaxing"). The generative AI model generates appropriate tourist information and returns it to the server.

[0729] Step 5:

[0730] The server translates the generated tourist information into the language set by the user. Input data: tourist information. Output data: translated multilingual tourist information. Specifically, the server uses a translation engine to translate the generated information into the user's language (e.g., English, Japanese, Chinese, etc.). The translated information is stored in a database.

[0731] Step 6:

[0732] The server creates a push notification message from the translated information and sends it to the device. Input data: Translated multilingual tourist information. Output data: Push notification sent to the user's device. Specifically, the server creates the push notification text based on the translated information and sends it to the user's device as a push notification. The device displays the received notification to the user in real time.

[0733] Step 7:

[0734] The user taps the displayed notification to access the detailed information page. Input data: The body of the push notification. Output data: The display of the detailed information page. Specifically, when the user taps the notification, the device sends a request to the server to display the detailed information page. The server sends the detailed information to the device, and the device displays it to the user.

[0735] ---

[0736] The above is an explanation of the specific flow of the program processing of this system and each step.

[0737] (Application example 2)

[0738] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0739] Modern tourists, especially inbound tourists, face the challenge of finding it difficult to receive accurate and up-to-date tourist information in their native language. Another problem is that tourist information is generally generic, making it impossible to provide customized information tailored to the feelings and circumstances of individual tourists. This leads to a decline in the quality of the tourist experience and a lack of satisfaction.

[0740] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0741] In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for translating the generated information, means for notifying the generated multilingual information, means for using an emotion engine that recognizes the user's emotions, and means for customizing the tourist information based on the emotion information. This makes it possible to provide tourist information customized in multiple languages ​​in real time according to the emotions and state of each tourist.

[0742] "Means for inputting user data" refers to an interface through which a user inputs their name, gender, age, preferred language, and permission for collection of emotional information.

[0743] "Means for obtaining location information from a user" refers to a device or system that uses GPS functionality to automatically obtain the user's current location.

[0744] "Generative AI" refers to artificial intelligence technology for generating new information based on large amounts of data, and specifically includes models that perform natural language processing.

[0745] "Means for translating generated information" refers to a multilingual translation engine that translates information generated by generative AI into the user's native language.

[0746] "Means for notifying the generated multilingual information" refers to a system for sending the generated and translated information to the user's device as a push notification.

[0747] An "emotion engine that recognizes user emotions" refers to technology that uses cameras and sensors to analyze a user's facial expressions and voice and identify their emotions.

[0748] "Means for customizing tourist information based on emotional information" refers to a system that provides tourist information and recommended information appropriate to the user's state based on their emotional data.

[0749] System configuration and main processing

[0750] The present invention is a system that provides local tourist information in multiple languages ​​and also recognizes the user's emotions to provide customized information. The system consists of the following main components:

[0751] 1. User data input method

[0752] When users enter an autonomous vehicle or launch an application, they enter their name, gender, age, preferred language, and permission to collect emotional information through the interface. This information is sent to the vehicle's computer and stored in a database.

[0753] 2. Location information acquisition means

[0754] The device periodically obtains the user's current location using GPS, and this location information is sent to the vehicle's computer system for updates.

[0755] 3. Emotional Engine Means

[0756] The device uses a camera and microphone installed in the vehicle to capture the user's facial expressions and voice, which are then analyzed by an emotion recognition engine (e.g., Microsoft Azure Face API, Google Cloud Vision API) to generate emotion data, which is then sent to the vehicle's computer.

[0757] 4. Information generation methods using generative AI

[0758] The server uses generative AI (e.g., OpenAI GPT-4) to generate tourist information appropriate for the user based on the received location and emotion information. An example of input used for generation is a prompt sentence such as "Tell me where I can relax around Ginza."

[0759] 5. Multilingual Translation Tools

[0760] The server translates the generated information into the language set by the user using a multilingual translation engine (e.g., Microsoft Translator, Google Translate), and the translated information is stored in a database.

[0761] 6. Means of notification

[0762] The device or vehicle's interface system will send the translated information as a push notification, and the user will receive the information as audio or text and can select the tourist attractions or restaurants they want to visit.

[0763] Natural language explanations

[0764] The user data input means consists of an interface implementation that allows users to input their name, gender, language preference, etc. when entering an application or an autonomous vehicle. This information is sent to the server and used for subsequent customization processing.

[0765] The location information acquisition means acquires the user's current location using a GPS module and transmits that information to the server. This allows the location information to be constantly updated in real time, and tourist information is provided based on the latest information.

[0766] The emotion engine means uses a camera and microphone to capture the user's facial expressions and voice, and processes them with an emotion recognition engine such as Microsoft Azure Face API or Google Cloud Vision API. This data is sent to a server as emotion information, and the user's emotional state is measured.

[0767] The information generation method using generative AI generates tourist information based on location information and emotion information using generative AI such as OpenAI GPT-4. For example, it uses the prompt sentence "Tell me about places where I can relax around Ginza" to generate tourist information that is optimal for the user.

[0768] The multilingual translation means translates the generated tourist information into the language set by the user, using a multilingual translation engine such as Microsoft Translator or Google Translate, and stores the translated information in a database.

[0769] The notification means sends the translated information to the user's device or the interface system of the autonomous vehicle as a push notification, allowing the user to receive appropriate tourist information in real time.

[0770] Specific examples

[0771] 1. A Japanese tourist gets into a self-driving vehicle, selects Japanese, and allows emotional information to be collected.

[0772] 2. The vehicle's GPS verifies its current location as Ginza and transmits this information to the computer system.

[0773] 3. The emotion engine recognizes a relaxed state from facial expressions.

[0774] 4. The computer system inputs "places to relax around Ginza" as a prompt sentence into the generative AI, and generates appropriate tourist information.

[0775] 5. The generated information is translated into Japanese using a multilingual translation engine and stored in a database.

[0776] 6. The vehicle's interface system presents the location to the user as a "quiet and enjoyable place in Ginza."

[0777] This system allows users to receive the most appropriate tourist information based on their emotions and current location in real time in multiple languages.

[0778] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0779] Step 1:

[0780] When a user enters a vehicle, they enter their name, gender, age, preferred language, and permission to collect emotional information on a tablet or interface.

[0781] Input: User's name, gender, age, preferred language, and permission to collect emotional information.

[0782] Output: The input information is stored in the vehicle's database.

[0783] In this step, basic user data is collected and stored in a database for subsequent customization.

[0784] Step 2:

[0785] The device uses its GPS function to obtain the user's location information in real time and transmits it to the server.

[0786] Input: None (automatic location acquisition via GPS).

[0787] Output: Current location data (latitude, longitude) is sent to the server and location information is updated.

[0788] This process allows the user's current location to be determined in real time, allowing tourist information to be provided based on appropriate location information.

[0789] Step 3:

[0790] The device's camera and microphone are used to capture the user's facial expressions and voice, which are then analyzed using an emotion recognition engine.

[0791] Input: User's facial expression image data, voice data.

[0792] Output: Emotional data (e.g., relaxed state, excited state) is generated and sent to the server.

[0793] Here, raw data captured by a camera and microphone is processed by an emotion recognition engine and converted into concrete data representing the user's current emotional state.

[0794] Step 4:

[0795] The server integrates location information and emotional information and uses generative AI to generate tourist guides based on that information.

[0796] Input: location information (latitude, longitude), emotional information (e.g., relaxation state).

[0797] Output: Generated tourist information (text format).

[0798] The server uses a generative AI model (e.g., OpenAI GPT-4) to generate specific tourist information based on the input data, providing instructions to the AI ​​in the form of a prompt such as "Tell me where I can relax around Ginza."

[0799] Step 5:

[0800] The server translates the generated tourist information into the language set by the user.

[0801] Input: Generated tourist information (original language).

[0802] Output: Translated tourist information (user-defined language, e.g. Japanese).

[0803] Using a translation engine (e.g., Microsoft Translator, Google Translate), the generated tourist information is translated into the language set by the user and stored in a database.

[0804] Step 6:

[0805] The interface system of the terminal or vehicle sends the translated tourist information to the user as a push notification.

[0806] Input: Translated tourist information (user-preferred language).

[0807] Output: Tourist information is displayed on the user's device as a push notification.

[0808] Users can receive notifications and tap to access more information, including detailed directions to tourist attractions and restaurants.

[0809] This series of steps creates a system that can provide multilingual tourist information customized according to the user's emotions and current location.

[0810] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0811] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0812] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0813] [Third embodiment]

[0814] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0815] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0816] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0817] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0818] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0819] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0820] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0821] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0822] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0823] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0824] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0825] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0826] ---

[0827] The present invention relates to a system that provides local tourist information in multiple languages ​​to inbound tourists. In an embodiment of the present invention, a user can use a device such as a smartphone or tablet to receive local tourist information and recommended dining spots.

[0828] This system mainly consists of the following components:

[0829] 1. User data input method:

[0830] The user starts the app and enters initial information such as their name, gender, age, preferred language, etc. on the initial setup screen. This information is sent to the server and stored in a database.

[0831] 2. Location information acquisition means:

[0832] The device periodically obtains the user's current location using the GPS function. This location information is sent to the server, where it is updated as the user's latest location information.

[0833] 3. Information generation using generative AI:

[0834] Based on the received location information, the server uses generative AI to generate information about nearby tourist spots and restaurants. This information is multilingual and can be translated into the language selected by the user.

[0835] 4. Multilingual translation tools:

[0836] The server translates the generated tourist information into the language set by the user (e.g., English, Japanese, Chinese, etc.). The translated information is stored in a database and provided to the user as needed.

[0837] 5. Means of notification:

[0838] The server sends the translated information to the user's device as a push notification. The device displays the received push notification to the user in real time. The user can tap the notification to access a page with more information.

[0839] Specific examples

[0840] Example 1: Tourist information

[0841] 1. The user launches the app and selects Japanese.

[0842] 2. The device sends its current location to the server, specifying Asakusa as its location.

[0843] 3. The server uses generative AI to generate information about tourist spots around Asakusa (e.g., Sensoji Temple, Kaminarimon Gate) and translates it into Japanese.

[0844] 4. The server creates a push notification saying "Explore Asakusa: Visit Sensoji Temple and Kaminarimon Gate!" and sends it to the device.

[0845] 5. The device displays a notification that the user can tap to access a page with more information.

[0846] Example 2: Dining recommendations

[0847] 1. The user launches the app and selects English.

[0848] 2. The device sends its current location to the server, specifying Ginza as its location.

[0849] 3. The server uses generative AI to generate information about restaurants around Ginza (e.g., high-end sushi restaurants) and translates it into English.

[0850] 4. The server creates a push notification saying "Explore Ginza: Try the premium sushi restaurants nearby!" and sends it to the device.

[0851] 5. The device displays a notification that the user can tap to access a page with more information.

[0852] This system allows inbound tourists to receive relevant tourist information in real time in a language they can understand.

[0853] The processing flow will be explained below.

[0854] ---

[0855] Step 1:

[0856] The user starts the app and enters their name, gender, age, and preferred language on the initial setup screen. The device then sends this information to the server.

[0857] Step 2:

[0858] The server stores the received user information in a database, generates a user ID and an authentication token, and sends them back to the device. The device stores the authentication token in local storage.

[0859] Step 3:

[0860] The device periodically acquires the user's location information using its GPS function, and the acquired location information (latitude and longitude) is sent to the server along with an authentication token.

[0861] Step 4:

[0862] The server updates the database with the received location information, stores the user's latest location information, and sends a request to the generative AI to generate information about nearby tourist attractions and restaurants based on the user's location information.

[0863] Step 5:

[0864] The generative AI generates information about tourist spots and restaurants in the area based on the location information it receives, and sends the generated information back to the server.

[0865] Step 6:

[0866] The server translates the generated information into the language selected by the user, and stores the translated information in a temporary database.

[0867] Step 7:

[0868] The server formats the translated information as a push notification message and prepares it to be sent to the user's device.

[0869] Step 8:

[0870] The server sends the push notification to the user's device, which then displays the received push notification to the user in real time.

[0871] Step 9:

[0872] Users can tap on the push notification to access a detailed information page, which displays detailed information about the recommended tourist spot or restaurant.

[0873] ---

[0874] Example 1

[0875] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0876] In recent years, the number of foreign tourists has increased, and there is a growing need to provide local tourist information in multiple languages. However, conventional methods often fail to adequately provide multilingual tourist information or real-time information. In addition, there are only a limited number of systems that provide appropriate tourist information based on the user's current location, and there is a need for an efficient guidance method that can help tourists avoid getting lost at their destination.

[0877] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0878] In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for translating the generated information into a language set by the user, means for sending the translated information to the user's device as a push notification, and means for the user to tap the push notification to access a detailed information page. This allows multilingual tourist information to be provided in real time, enabling the user to access appropriate tourist information based on their current location.

[0879] The "means for inputting user data" is an interface that allows a user to input initial information such as their name, gender, age, and preferred language, and then transmit that information to the server.

[0880] The "means for acquiring location information from the user" is a function for periodically acquiring the user's current location using the GPS function of the user's terminal and transmitting that information to the server.

[0881] "Means for generating recommended tourist information based on location information using generative AI" refers to the process in which a server receives location information and generates information about nearby tourist spots and restaurants using a generative AI model.

[0882] "Means for translating the generated information into the language set by the user" refers to the process for translating the generated tourist information into the language selected by the user, and uses translation technologies such as translation APIs.

[0883] The "means for sending translated information to the user's device as a push notification" is a system in which the server creates a push notification based on the translated tourist information and sends it to the user's device.

[0884] "Means for users to tap on a push notification to access the detailed information page" refers to an interface that allows users to access the detailed page of the generated tourist information by tapping on a push notification displayed on their device.

[0885] The present invention relates to a system that provides local tourist information in multiple languages. This system is realized by combining three main elements: a user, a terminal, and a server.

[0886] User Data Input

[0887] First, a user launches the app using a device such as a smartphone or tablet. When launching the app for the first time, the user enters initial information such as their name, gender, age, preferred language, etc. This information is sent via the device to the server, which then stores it in a database.

[0888] Hardware used: smartphone, tablet

[0889] Software used: Mobile applications, database systems (e.g., MySQL)

[0890] Specific prompt examples:

[0891] "Initial setup screen: Please enter your name, gender, age, and preferred language."

[0892] Location information acquisition

[0893] Next, the device periodically acquires the user's current location using its GPS function. This location information is sent from the device to the server, where it is stored in a database as the latest location information.

[0894] Hardware used: GPS-enabled smartphones and tablets

[0895] Software used: GPS API, location management system

[0896] Specific prompt examples:

[0897] "Retrieving your location. Please wait a moment."

[0898] Information Generation and Translation

[0899] Based on the received location information, the server uses a generative AI model (e.g., GPT-4) to generate information about nearby tourist attractions and restaurants. This generated information is multilingual, and the server translates it into the language selected by the user. The translated information is then stored in a database.

[0900] Hardware used: Server

[0901] Software used: Generative AI models (e.g., GPT-4), translation APIs (e.g., Google Translate API, DeepL)

[0902] Specific prompt examples:

[0903] "The user is at Asakusa Station. Please tell me about nearby tourist spots in Japanese."

[0904] Sending push notifications

[0905] The server creates a push notification based on the translated information and sends it to the device, which displays the notification to the user in real time.

[0906] Hardware used: smartphone, tablet

[0907] Software used: Push notification system (e.g., Firebase Cloud Messaging)

[0908] Specific prompt examples:

[0909] "Sightseeing information for Sensoji Temple and Kaminarimon has been updated. Tap to see more details."

[0910] Access the details page

[0911] When a user taps on the received push notification, the device displays a page with detailed information about the related tourist spot. The server sends the requested detailed data to the device, which then displays the information.

[0912] Specific prompt examples:

[0913] "Visit our detailed information page. Check out the attractions and restaurants."

[0914] This system will enable inbound tourists to obtain relevant tourist information in real time in a language they can understand, greatly improving the efficiency and convenience of tourist guidance.

[0915] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0916] Step 1:

[0917] When a user launches the app for the first time, they enter their name, gender, age, and preferred language on the initial setup screen. Input: User's name, gender, age, and preferred language. Output: The input data is formatted within the application. The device sends this input data to the server. The server receives the data and stores it in a database.

[0918] Step 2:

[0919] The device periodically obtains the user's current location using its GPS function. Input: Current location information obtained from the GPS. Output: The obtained location information is formatted within the device and converted into a format that can be sent. The device sends the location information to the server, which updates it in the database.

[0920] Step 3:

[0921] The server sends a prompt to the generation AI based on the received location information. Input: Location information, prompt text to send to the generation AI. Output: Tourist information generated by the generation AI. The server receives the generated tourist information and begins the process of translating it into the language set by the user.

[0922] Step 4:

[0923] The server uses a translation API to translate the generated tourist information into the user's language. Input: Generated tourist information, language selected by the user. Output: Tourist information translated by the translation API. The translated information is saved in the database.

[0924] Step 5:

[0925] The server creates a push notification based on the translated information. Input: Translated tourist information. Output: Message in push notification format. The server sends the push notification to the device.

[0926] Step 6:

[0927] The device displays the received push notification to the user. Input: Push notification sent from the server. Output: Notification message displayed to the user by the device. The user taps the notification to access the detailed information page.

[0928] Step 7:

[0929] When a user taps on a push notification, the device displays a detailed information page for the related tourist attraction. Input: User operation (tap), request for detailed information page. Output: Detailed information page is displayed. The server sends the detailed information page data to the device, which then displays the detailed information.

[0930] Specifically, after the user launches the app for the first time and completes the setup, the server generates tourist information using the generative AI model, translates the information, and provides it to the user via push notification. By tapping the notification, the user can access detailed tourist information.

[0931] (Application example 1)

[0932] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0933] While systems exist that provide local tourist information and restaurant recommendations in multiple languages, until now they have been limited to personal devices such as smartphones and tablets. As a result, tourists had to operate their devices to receive real-time multilingual tourist information while on the move. This led to issues such as reduced convenience and user experience.

[0934] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0935] In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for translating the generated information into a language set by the user, means for notifying the user's means of transportation of the generated multilingual information, means for providing information using voice recognition and voice output, and means for displaying information on a display. This enables tourists to receive multilingual tourist information and recommended dining spots in real time by voice and on a display while in an autonomous vehicle.

[0936] "Multilingual" refers to the ability to provide information in multiple languages.

[0937] "Regional tourist guide" refers to a guide that provides tourist information and introduces tourist spots related to a specific region.

[0938] "User Data" refers to personal information such as the user's name, gender, age, and language preference.

[0939] "Location Information" refers to the current geographic location of a user or vehicle obtained using technologies such as GPS.

[0940] "Generative AI" refers to a system that uses artificial intelligence to analyze data and generate specific information or recommendations.

[0941] "Multilingual information" refers to a form in which generated information is provided in multiple languages.

[0942] "User's means of transportation" refers to the means of transportation used by tourists, in this case, particularly autonomous vehicles.

[0943] "Voice recognition" refers to the technology of analyzing the user's voice, understanding its meaning, and then performing operations or responding.

[0944] "Audio output" refers to a technique for conveying information to a user by voice.

[0945] "Display" refers to a device that visually displays information.

[0946] "Push notification" refers to a notification message that is automatically sent from a server to a user's device.

[0947] "Infotainment system" refers to a system that integrates information and entertainment, especially an in-vehicle system.

[0948] The present invention provides a system for providing local tourist information in multiple languages, and is particularly suitable for self-driving vehicles. Specific configurations and processes for implementing the invention are described below.

[0949] The system mainly consists of the following components:

[0950] 1. User data input method:

[0951] The user uses the vehicle's touch panel or voice input to enter initial information such as name, gender, age, and preferred language.

[0952] This information is stored on the on-board computer and transmitted to a server.

[0953] 2. Location information acquisition means:

[0954] The vehicle's GPS function is used to obtain the current location in real time and send it to a server.

[0955] 3. Information generation using generative AI:

[0956] The server receives the location information and uses a generative AI model (e.g., OpenAI GPT-3) to generate information about nearby tourist attractions and restaurants.

[0957] The server uses a multilingual translation API (e.g., Google Translate API) to translate the generated information into the user's language of choice.

[0958] 4. Means of notification:

[0959] The translated information is sent to the car's infotainment system as a push notification.

[0960] Information is provided to the user through the voice assistant and the display.

[0961] As a specific example, a process will be described in which the user gets on a vehicle and selects English.

[0962] Example 1: Tourist information

[0963] 1. User provides input:

[0964] Get on the vehicle and select English on the in-car display.

[0965] Enter the initial information using the touch panel or voice input.

[0966] 2. Get location information:

[0967] The vehicle recognizes its current location as being near Tokyo Tower.

[0968] Location information is sent to the server via GPS.

[0969] 3. Information Generation and Translation:

[0970] The server generates information about tourist spots around Tokyo Tower using a generative AI model.

[0971] Translate into English using the translation API.

[0972] 4. Notification of Results:

[0973] The server creates a notification saying "Explore Tokyo: Don't miss Tokyo Tower and Roppongi Hills!"

[0974] Information is pushed to the in-car infotainment system, displayed on the display, and notified via the voice assistant.

[0975] Example 2: Dining recommendations

[0976] 1. User provides input:

[0977] Select Chinese on the in-car voice assistant.

[0978] Enter the initial information.

[0979] 2. Obtain location information:

[0980] The vehicle recognizes its current location as Shinjuku.

[0981] The location information is sent to the server via GPS.

[0982] 3. Information generation and translation:

[0983] The server generates restaurant information around Shinjuku using a generation AI model.

[0984] It is translated into Chinese using a translation API.

[0985] 4. Notification of results:

[0986] The server creates a notification saying "Try the popular barbecue restaurants here in Shinjuku!".

[0987] The information is pushed to the in-vehicle infotainment system, displayed on the display, and notified by the voice assistant.

[0988] Example of prompt text

[0989] Prompt: "The current location is Shinjuku. Generate sightseeing spot information around it and translate it into Chinese."

[0990] Expected output: "There are many places worth visiting around Shinjuku Station, including... (specific spot information)"

[0991] With this system, even when tourists are in an autonomous vehicle, it is possible to provide real-time multilingual sightseeing information and recommended information on dining places, significantly improving convenience.

[0992] The flow of specific processing in Application Example 1 will be described using Fig. 12.

[0993] Step 1:

[0994] A user gets into a vehicle and uses a touch panel or voice assistant to input initial information such as name, gender, age, and preferred language. This user data is then stored in the vehicle's on-board computer and sent to the server. The input comes from the user, and the output is a dataset of the initial information.

[0995] Step 2:

[0996] The vehicle's GPS function is used to obtain the vehicle's current location in real time. The terminal that receives location information from the GPS sends this location information to the server. The input is location data from the GPS, and the output is the location information sent to the server.

[0997] Step 3:

[0998] Based on the location information received by the server, a generative AI model is used to generate information about nearby tourist attractions and restaurants. The server generates a prompt sentence and provides it as input to the AI ​​model. The input is the location information and the prompt sentence, and the output is the generated tourist information.

[0999] Step 4:

[1000] The server uses a multilingual translation API to translate the generated tourist information into the specified language. It passes the information and the target language as input to the translation API and obtains the translated information. The input is the generated tourist information and the target language, and the output is the translated information.

[1001] Step 5:

[1002] The server sends the translated tourist information to the vehicle's infotainment system as a push notification. The in-vehicle infotainment system receives the information and displays it on the vehicle's display, while also notifying the user via voice using the voice assistant. The input is the translated tourist information, and the output is the display and voice notification.

[1003] Step 6:

[1004] Users can access more information by tapping on the information displayed on the screen or asking a question to the voice assistant. This action allows them to obtain more detailed information or decide on the next action. The input is the user's tap or voice command, and the output is the provision of more information.

[1005] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1006] ---

[1007] The present invention relates to a system that provides local tourist information in multiple languages ​​to inbound tourists and combines it with an emotion engine that recognizes user emotions. In an embodiment of the present invention, a user uses a device such as a smartphone or tablet to receive local tourist information and restaurant recommendations, and is also provided with information customized based on their emotions.

[1008] This system mainly consists of the following components:

[1009] 1. User data input method:

[1010] The user starts the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. This information is sent to the server and stored in a database.

[1011] 2. Location information acquisition means:

[1012] The device periodically obtains the user's current location using the GPS function. This location information is sent to the server, where it is updated as the user's latest location information.

[1013] 3. Emotion engine means:

[1014] The device uses cameras and sensors to analyze the user's facial expressions and voice to recognize emotions, and the emotional information is sent to a server with the user's permission.

[1015] 4. Information generation methods using generative AI:

[1016] The server uses generative AI to generate information about nearby tourist spots and restaurants based on the location and emotion information it receives. The generated information is tailored to the user's emotions.

[1017] 5. Multilingual translation tools:

[1018] The server translates the generated tourist information into the language set by the user (e.g., English, Japanese, Chinese, etc.) and stores the translated information in a database.

[1019] 6. Means of notification:

[1020] The server sends the translated information to the user's device as a push notification, which is then displayed to the user in real time.

[1021] Specific examples

[1022] Example 1: Tourist information

[1023] 1. The user launches the app, selects Japanese, and allows emotional information collection.

[1024] 2. The device sends its current location to the server, indicating that it is in Asakusa. The emotion engine then analyzes the user's facial expression and determines that the user is relaxed.

[1025] 3. The server uses generative AI to generate information about tourist spots around Asakusa (e.g., Sensoji Temple, Kaminarimon Gate), creates recommendations for enjoying a relaxing sightseeing experience, and translates them into Japanese.

[1026] 4. The server creates a push notification saying "Relax and explore Asakusa: Take a leisurely stroll around Sensoji Temple!" and sends it to the device.

[1027] 5. The device displays a notification that the user can tap to access a page with more information.

[1028] Example 2: Dining recommendations

[1029] 1. The user launches the app, selects English, and allows emotional information collection.

[1030] 2. The device sends its current location to the server, indicating that it is in Ginza. The emotion engine then analyzes the user's voice and determines that the user is excited.

[1031] 3. The server uses generative AI to generate information about restaurants around Ginza (e.g., high-end sushi restaurants), creates recommendations to provide an exciting dining experience, and translates them into English.

[1032] 4. The server creates a push notification saying "Explore Ginza: Enjoy the thrilling experience at premium sushi restaurants nearby!" and sends it to the device.

[1033] 5. The device displays a notification that the user can tap to access a page with more information.

[1034] This system allows inbound tourists to receive relevant tourist information in real time in a language they understand, and also provides information that is tailored to their emotions, further enhancing their tourist experience and increasing their satisfaction.

[1035] The processing flow will be explained below.

[1036] --

[1037] Step 1:

[1038] The user starts the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. The device then sends this information to the server.

[1039] Step 2:

[1040] The server stores the received user information in a database, generates a user ID and an authentication token, and sends them back to the device. The device stores the authentication token in local storage.

[1041] Step 3:

[1042] The device periodically obtains the user's current location using its GPS function, and the obtained location information (latitude and longitude) is sent to the server along with an authentication token.

[1043] Step 4:

[1044] The server updates the received location information in a database and stores the user's latest location information.

[1045] Step 5:

[1046] The device uses cameras and sensors to analyze the user's facial expressions and voice to recognize emotions, and the recognized emotional information is sent to the server along with an authentication token.

[1047] Step 6:

[1048] The server stores the received emotional information in a database and provides it to the generative AI, which then generates information about nearby tourist spots and restaurants based on the location and emotional information.

[1049] Step 7:

[1050] The server translates the generated tourist information into the language selected by the user, and the translated information is saved in a database.

[1051] Step 8:

[1052] The server formats the translated information as a push notification message and prepares it to be sent to the user's device.

[1053] Step 9:

[1054] The server sends the push notification to the user's device, which then displays the received push notification to the user in real time.

[1055] Step 10:

[1056] Users can tap on the push notification to access a detailed information page, which displays detailed information about the recommended tourist spot or restaurant.

[1057] --

[1058] Example 2

[1059] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1060] Conventional tourist information systems were able to provide tourist information based on the user's location information, but lacked customization based on the user's emotional state. As a result, while there was a demand for information provision optimized for the user's emotions, there were issues with not being able to sufficiently increase user satisfaction. Other issues included a lack of multilingual support and the inability to provide information smoothly in real time.

[1061] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for acquiring and analyzing emotional information of the user, means for customizing the generated tourist information based on the emotional information, means for translating the generated information into a language set by the user, and means for notifying the user's terminal of the generated multilingual information. This makes it possible to provide tourist information in multiple languages ​​in real time according to the user's emotional state.

[1062] ---

[1063] ---

[1064] "User Data" refers to information such as a user's name, gender, age, language preference, and preferences regarding collection of emotional information.

[1065] "Location information" refers to information about the user's current location obtained by the GPS function of the device.

[1066] "Generative AI" refers to artificial intelligence that generates tourist information based on received input information and prompts.

[1067] "Emotion information" refers to information about the emotional state of the user that is acquired by analyzing the user's facial expressions and voice.

[1068] "Notification means" refers to a mechanism for sending and displaying the generated information as a push notification to the user's device.

[1069] "Multilingual information" refers to tourist information and recommendations translated into the language set by the user.

[1070] "Server" refers to a central processing unit that stores user data, analyzes location and emotional information, generates information using generative AI, and translates and notifies users of that information.

[1071] "Terminal" refers to a mobile information terminal such as a smartphone or tablet used by a user.

[1072] ---

[1073] The above are definitions of important words included in the scope of the patent claims.

[1074] ---

[1075] System Configuration

[1076] This invention is a system that provides multilingual local tourist information for inbound tourists, recognizing user emotions and providing customized information based on those emotions. Users access the system using devices such as smartphones or tablets to receive local tourist information and recommended dining spots. The main components of this system are a user data input means, a location information acquisition means, an emotion information acquisition means, an information generation means using generative AI, a multilingual translation means, and a notification means.

[1077] User Data Input

[1078] The user starts the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. This information is sent from the device to the server and stored in a database.

[1079] Location information acquisition

[1080] The device periodically acquires the user's current location using its GPS function, and this location information is sent to the server and updated in the server's database.

[1081] Emotional information acquisition

[1082] The device uses sensors such as a camera and microphone to collect the user's facial expressions and voice, and analyzes the emotional information. The analyzed emotional information is sent to a server and stored in a database.

[1083] Information generation by generative AI

[1084] The server uses a generative AI model to generate tourist information and restaurant recommendations based on the received location and emotion information. The generated information is optimized to match the user's emotions.

[1085] Prompt Sentence Examples

[1086] Please suggest some tourist spots in Asakusa for users who are relaxing.

[1087] Suggest a high-end sushi restaurant in Ginza for excited users.

[1088] Multilingual Translation

[1089] The generated information is translated by the server into the language set by the user, and the translated information is stored in a database.

[1090] Notifications and Indications

[1091] The server creates a push notification message from the translated information and sends it to the device. The device displays the received push notification to the user in real time. The user can tap the notification to access a page with more information.

[1092] Specific examples

[1093] Example 1: Tourist information

[1094] 1. The user launches the app, selects Japanese, and allows the collection of emotion information.

[1095] 2. The device sends its current location to the server, specifying Asakusa as its location.

[1096] 3. The device uses a camera to analyze the user's facial expressions and recognizes that they are relaxed.

[1097] 4. The server inputs a prompt to the generative AI model: "Please suggest tourist spots in Asakusa for a user who is relaxing."

[1098] 5. The generative AI model generates information about relaxing spots such as Sensoji Temple and Kaminarimon Gate.

[1099] 6. The server translates the generated information into Japanese and stores it in the database.

[1100] 7. The server creates a push notification saying "Relax and explore Asakusa: Take a leisurely stroll around Sensoji Temple!" and sends it to the device.

[1101] 8. The device displays a notification, and the user can tap the notification to access a page with more information.

[1102] Example 2: Dining recommendations

[1103] 1. The user launches the app, selects English, and allows emotional information collection.

[1104] 2. The device sends its current location to the server, specifying Ginza as its location.

[1105] 3. The device analyzes the user's voice using a microphone and recognizes that they are excited.

[1106] 4. The server inputs a prompt to the generative AI model: "Please suggest high-end sushi restaurants in Ginza for an excited user."

[1107] 5. A generative AI model generates restaurant information that provides exciting experiences, such as high-end sushi restaurants.

[1108] 6. The server translates the generated information into English and stores it in a database.

[1109] 7. The server creates a push notification saying "Explore Ginza: Enjoy the thrilling experience at premium sushi restaurants nearby!" and sends it to the device.

[1110] 8. The device displays a notification, and the user can tap the notification to access a page with more information.

[1111] This system allows inbound tourists to receive appropriate tourist information in real time in a language they can understand, and also provides information that is optimized to suit their emotions, further enriching their tourist experience and increasing their satisfaction.

[1112] ---

[1113] The above is the form for carrying out the invention.

[1114] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1115] ---

[1116] Step 1:

[1117] The user launches the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. Input data: name, gender, age, language, permission to collect emotional information. Output data: user data sent to the server. Specifically, the user enters the required information in each field and taps the send button. This causes the device to send the user data to the server, which then stores it in a database.

[1118] Step 2:

[1119] The device periodically obtains the user's current location using its GPS function. Input data: GPS information of the current location. Output data: Location information sent to the server. Specifically, the device periodically obtains GPS data in the background and sends it to the server. The server then stores the location information in a database and updates it with the latest location information.

[1120] Step 3:

[1121] The device uses a camera and microphone to collect the user's facial expressions and voice and recognizes emotions. Input data: user's facial expression data, voice data. Output data: emotional information sent to the server. Specifically, the device collects data in real time using the camera and microphone and identifies the user's emotions using an emotion analysis algorithm. The analysis results are sent to the server and stored in a database.

[1122] Step 4:

[1123] The server inputs a prompt sentence into the generative AI model based on the received location information and emotion information. Input data: location information, emotion information. Output data: generated tourist information. Specifically, the server generates a prompt sentence and inputs it into the generative AI model (e.g., "Please suggest tourist spots in Asakusa for a user who is relaxing"). The generative AI model generates appropriate tourist information and returns it to the server.

[1124] Step 5:

[1125] The server translates the generated tourist information into the language set by the user. Input data: tourist information. Output data: translated multilingual tourist information. Specifically, the server uses a translation engine to translate the generated information into the user's language (e.g., English, Japanese, Chinese, etc.). The translated information is stored in a database.

[1126] Step 6:

[1127] The server creates a push notification message from the translated information and sends it to the device. Input data: Translated multilingual tourist information. Output data: Push notification sent to the user's device. Specifically, the server creates the push notification text based on the translated information and sends it to the user's device as a push notification. The device displays the received notification to the user in real time.

[1128] Step 7:

[1129] The user taps the displayed notification to access the detailed information page. Input data: The body of the push notification. Output data: The display of the detailed information page. Specifically, when the user taps the notification, the device sends a request to the server to display the detailed information page. The server sends the detailed information to the device, and the device displays it to the user.

[1130] ---

[1131] The above is an explanation of the specific flow of the program processing of this system and each step.

[1132] (Application example 2)

[1133] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1134] Modern tourists, especially inbound tourists, face the challenge of finding it difficult to receive accurate and up-to-date tourist information in their native language. Another problem is that tourist information is generally generic, making it impossible to provide customized information tailored to the feelings and circumstances of individual tourists. This leads to a decline in the quality of the tourist experience and a lack of satisfaction.

[1135] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1136] In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for translating the generated information, means for notifying the generated multilingual information, means for using an emotion engine that recognizes the user's emotions, and means for customizing the tourist information based on the emotion information. This makes it possible to provide tourist information customized in multiple languages ​​in real time according to the emotions and state of each tourist.

[1137] "Means for inputting user data" refers to an interface through which a user inputs their name, gender, age, preferred language, and permission for collection of emotional information.

[1138] "Means for obtaining location information from a user" refers to a device or system that uses GPS functionality to automatically obtain the user's current location.

[1139] "Generative AI" refers to artificial intelligence technology for generating new information based on large amounts of data, and specifically includes models that perform natural language processing.

[1140] "Means for translating generated information" refers to a multilingual translation engine that translates information generated by generative AI into the user's native language.

[1141] "Means for notifying the generated multilingual information" refers to a system for sending the generated and translated information to the user's device as a push notification.

[1142] An "emotion engine that recognizes user emotions" refers to technology that uses cameras and sensors to analyze a user's facial expressions and voice and identify their emotions.

[1143] "Means for customizing tourist information based on emotional information" refers to a system that provides tourist information and recommended information appropriate to the user's state based on their emotional data.

[1144] System configuration and main processing

[1145] The present invention is a system that provides local tourist information in multiple languages ​​and also recognizes the user's emotions to provide customized information. The system consists of the following main components:

[1146] 1. User data input method

[1147] When users enter an autonomous vehicle or launch an application, they enter their name, gender, age, preferred language, and permission to collect emotional information through the interface. This information is sent to the vehicle's computer and stored in a database.

[1148] 2. Location information acquisition means

[1149] The device periodically obtains the user's current location using GPS, and this location information is sent to the vehicle's computer system for updates.

[1150] 3. Emotional Engine Means

[1151] The device uses a camera and microphone installed in the vehicle to capture the user's facial expressions and voice, which are then analyzed by an emotion recognition engine (e.g., Microsoft Azure Face API, Google Cloud Vision API) to generate emotion data, which is then sent to the vehicle's computer.

[1152] 4. Information generation methods using generative AI

[1153] The server uses generative AI (e.g., OpenAI GPT-4) to generate tourist information appropriate for the user based on the received location and emotion information. An example of input used for generation is a prompt sentence such as "Tell me where I can relax around Ginza."

[1154] 5. Multilingual Translation Tools

[1155] The server translates the generated information into the language set by the user using a multilingual translation engine (e.g., Microsoft Translator, Google Translate), and the translated information is stored in a database.

[1156] 6. Means of notification

[1157] The device or vehicle's interface system will send the translated information as a push notification, and the user will receive the information as audio or text and can select the tourist attractions or restaurants they want to visit.

[1158] Natural language explanations

[1159] The user data input means consists of an interface implementation that allows users to input their name, gender, language preference, etc. when entering an application or an autonomous vehicle. This information is sent to the server and used for subsequent customization processing.

[1160] The location information acquisition means acquires the user's current location using a GPS module and transmits that information to the server. This allows the location information to be constantly updated in real time, and tourist information is provided based on the latest information.

[1161] The emotion engine means uses a camera and microphone to capture the user's facial expressions and voice, and processes them with an emotion recognition engine such as Microsoft Azure Face API or Google Cloud Vision API. This data is sent to a server as emotion information, and the user's emotional state is measured.

[1162] The information generation method using generative AI generates tourist information based on location information and emotion information using generative AI such as OpenAI GPT-4. For example, it uses the prompt sentence "Tell me about places where I can relax around Ginza" to generate tourist information that is optimal for the user.

[1163] The multilingual translation means translates the generated tourist information into the language set by the user, using a multilingual translation engine such as Microsoft Translator or Google Translate, and stores the translated information in a database.

[1164] The notification means sends the translated information to the user's device or the interface system of the autonomous vehicle as a push notification, allowing the user to receive appropriate tourist information in real time.

[1165] Specific examples

[1166] 1. A Japanese tourist gets into a self-driving vehicle, selects Japanese, and allows emotional information to be collected.

[1167] 2. The vehicle's GPS verifies its current location as Ginza and transmits this information to the computer system.

[1168] 3. The emotion engine recognizes a relaxed state from facial expressions.

[1169] 4. The computer system inputs "places to relax around Ginza" as a prompt sentence into the generative AI, and generates appropriate tourist information.

[1170] 5. The generated information is translated into Japanese using a multilingual translation engine and stored in a database.

[1171] 6. The vehicle's interface system presents the location to the user as a "quiet and enjoyable place in Ginza."

[1172] This system allows users to receive the most appropriate tourist information based on their emotions and current location in real time in multiple languages.

[1173] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1174] Step 1:

[1175] When a user enters a vehicle, they enter their name, gender, age, preferred language, and permission to collect emotional information on a tablet or interface.

[1176] Input: User's name, gender, age, preferred language, and permission to collect emotional information.

[1177] Output: The input information is stored in the vehicle's database.

[1178] In this step, basic user data is collected and stored in a database for subsequent customization.

[1179] Step 2:

[1180] The device uses its GPS function to obtain the user's location information in real time and transmits it to the server.

[1181] Input: None (automatic location acquisition via GPS).

[1182] Output: Current location data (latitude, longitude) is sent to the server and location information is updated.

[1183] This process allows the user's current location to be determined in real time, allowing tourist information to be provided based on appropriate location information.

[1184] Step 3:

[1185] The device's camera and microphone are used to capture the user's facial expressions and voice, which are then analyzed using an emotion recognition engine.

[1186] Input: User's facial expression image data, voice data.

[1187] Output: Emotional data (e.g., relaxed state, excited state) is generated and sent to the server.

[1188] Here, raw data captured by a camera and microphone is processed by an emotion recognition engine and converted into concrete data representing the user's current emotional state.

[1189] Step 4:

[1190] The server integrates location information and emotional information and uses generative AI to generate tourist guides based on that information.

[1191] Input: location information (latitude, longitude), emotional information (e.g., relaxation state).

[1192] Output: Generated tourist information (text format).

[1193] The server uses a generative AI model (e.g., OpenAI GPT-4) to generate specific tourist information based on the input data, providing instructions to the AI ​​in the form of a prompt such as "Tell me where I can relax around Ginza."

[1194] Step 5:

[1195] The server translates the generated tourist information into the language set by the user.

[1196] Input: Generated tourist information (original language).

[1197] Output: Translated tourist information (user-defined language, e.g. Japanese).

[1198] Using a translation engine (e.g., Microsoft Translator, Google Translate), the generated tourist information is translated into the language set by the user and stored in a database.

[1199] Step 6:

[1200] The interface system of the terminal or vehicle sends the translated tourist information to the user as a push notification.

[1201] Input: Translated tourist information (user-preferred language).

[1202] Output: Tourist information is displayed on the user's device as a push notification.

[1203] Users can receive notifications and tap to access more information, including detailed directions to tourist attractions and restaurants.

[1204] This series of steps creates a system that can provide multilingual tourist information customized according to the user's emotions and current location.

[1205] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1206] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1207] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1208] [Fourth embodiment]

[1209] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1210] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1211] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1212] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1213] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1214] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1215] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1216] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1217] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1218] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1219] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1220] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1221] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1222] ---

[1223] The present invention relates to a system that provides local tourist information in multiple languages ​​to inbound tourists. In an embodiment of the present invention, a user can use a device such as a smartphone or tablet to receive local tourist information and recommended dining spots.

[1224] This system mainly consists of the following components:

[1225] 1. User data input method:

[1226] The user starts the app and enters initial information such as their name, gender, age, preferred language, etc. on the initial setup screen. This information is sent to the server and stored in a database.

[1227] 2. Location information acquisition means:

[1228] The device periodically obtains the user's current location using the GPS function. This location information is sent to the server, where it is updated as the user's latest location information.

[1229] 3. Information generation using generative AI:

[1230] Based on the received location information, the server uses generative AI to generate information about nearby tourist spots and restaurants. This information is multilingual and can be translated into the language selected by the user.

[1231] 4. Multilingual translation tools:

[1232] The server translates the generated tourist information into the language set by the user (e.g., English, Japanese, Chinese, etc.). The translated information is stored in a database and provided to the user as needed.

[1233] 5. Means of notification:

[1234] The server sends the translated information to the user's device as a push notification. The device displays the received push notification to the user in real time. The user can tap the notification to access a page with more information.

[1235] Specific examples

[1236] Example 1: Tourist information

[1237] 1. The user launches the app and selects Japanese.

[1238] 2. The device sends its current location to the server, specifying Asakusa as its location.

[1239] 3. The server uses generative AI to generate information about tourist spots around Asakusa (e.g., Sensoji Temple, Kaminarimon Gate) and translates it into Japanese.

[1240] 4. The server creates a push notification saying "Explore Asakusa: Visit Sensoji Temple and Kaminarimon Gate!" and sends it to the device.

[1241] 5. The device displays a notification that the user can tap to access a page with more information.

[1242] Example 2: Dining recommendations

[1243] 1. The user launches the app and selects English.

[1244] 2. The device sends its current location to the server, specifying Ginza as its location.

[1245] 3. The server uses generative AI to generate information about restaurants around Ginza (e.g., high-end sushi restaurants) and translates it into English.

[1246] 4. The server creates a push notification saying "Explore Ginza: Try the premium sushi restaurants nearby!" and sends it to the device.

[1247] 5. The device displays a notification that the user can tap to access a page with more information.

[1248] This system allows inbound tourists to receive relevant tourist information in real time in a language they can understand.

[1249] The processing flow will be explained below.

[1250] ---

[1251] Step 1:

[1252] The user starts the app and enters their name, gender, age, and preferred language on the initial setup screen. The device then sends this information to the server.

[1253] Step 2:

[1254] The server stores the received user information in a database, generates a user ID and an authentication token, and sends them back to the device. The device stores the authentication token in local storage.

[1255] Step 3:

[1256] The device periodically acquires the user's location information using its GPS function, and the acquired location information (latitude and longitude) is sent to the server along with an authentication token.

[1257] Step 4:

[1258] The server updates the database with the received location information, stores the user's latest location information, and sends a request to the generative AI to generate information about nearby tourist attractions and restaurants based on the user's location information.

[1259] Step 5:

[1260] The generative AI generates information about tourist spots and restaurants in the area based on the location information it receives, and sends the generated information back to the server.

[1261] Step 6:

[1262] The server translates the generated information into the language selected by the user, and stores the translated information in a temporary database.

[1263] Step 7:

[1264] The server formats the translated information as a push notification message and prepares it to be sent to the user's device.

[1265] Step 8:

[1266] The server sends the push notification to the user's device, which then displays the received push notification to the user in real time.

[1267] Step 9:

[1268] Users can tap on the push notification to access a detailed information page, which displays detailed information about the recommended tourist spot or restaurant.

[1269] ---

[1270] Example 1

[1271] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1272] In recent years, the number of foreign tourists has increased, and there is a growing need to provide local tourist information in multiple languages. However, conventional methods often fail to adequately provide multilingual tourist information or real-time information. In addition, there are only a limited number of systems that provide appropriate tourist information based on the user's current location, and there is a need for an efficient guidance method that can help tourists avoid getting lost at their destination.

[1273] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1274] In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for translating the generated information into a language set by the user, means for sending the translated information to the user's device as a push notification, and means for the user to tap the push notification to access a detailed information page. This allows multilingual tourist information to be provided in real time, enabling the user to access appropriate tourist information based on their current location.

[1275] The "means for inputting user data" is an interface that allows a user to input initial information such as their name, gender, age, and preferred language, and then transmit that information to the server.

[1276] The "means for acquiring location information from the user" is a function for periodically acquiring the user's current location using the GPS function of the user's terminal and transmitting that information to the server.

[1277] "Means for generating recommended tourist information based on location information using generative AI" refers to the process in which a server receives location information and generates information about nearby tourist spots and restaurants using a generative AI model.

[1278] "Means for translating the generated information into the language set by the user" refers to the process for translating the generated tourist information into the language selected by the user, and uses translation technologies such as translation APIs.

[1279] The "means for sending translated information to the user's device as a push notification" is a system in which the server creates a push notification based on the translated tourist information and sends it to the user's device.

[1280] "Means for users to tap on a push notification to access the detailed information page" refers to an interface that allows users to access the detailed page of the generated tourist information by tapping on a push notification displayed on their device.

[1281] The present invention relates to a system that provides local tourist information in multiple languages. This system is realized by combining three main elements: a user, a terminal, and a server.

[1282] User Data Input

[1283] First, a user launches the app using a device such as a smartphone or tablet. When launching the app for the first time, the user enters initial information such as their name, gender, age, preferred language, etc. This information is sent via the device to the server, which then stores it in a database.

[1284] Hardware used: smartphone, tablet

[1285] Software used: Mobile applications, database systems (e.g., MySQL)

[1286] Specific prompt examples:

[1287] "Initial setup screen: Please enter your name, gender, age, and preferred language."

[1288] Location information acquisition

[1289] Next, the device periodically acquires the user's current location using its GPS function. This location information is sent from the device to the server, where it is stored in a database as the latest location information.

[1290] Hardware used: GPS-enabled smartphones and tablets

[1291] Software used: GPS API, location management system

[1292] Specific prompt examples:

[1293] "Retrieving your location. Please wait a moment."

[1294] Information Generation and Translation

[1295] Based on the received location information, the server uses a generative AI model (e.g., GPT-4) to generate information about nearby tourist attractions and restaurants. This generated information is multilingual, and the server translates it into the language selected by the user. The translated information is then stored in a database.

[1296] Hardware used: Server

[1297] Software used: Generative AI models (e.g., GPT-4), translation APIs (e.g., Google Translate API, DeepL)

[1298] Specific prompt examples:

[1299] "The user is at Asakusa Station. Please tell me about nearby tourist spots in Japanese."

[1300] Sending push notifications

[1301] The server creates a push notification based on the translated information and sends it to the device, which displays the notification to the user in real time.

[1302] Hardware used: smartphone, tablet

[1303] Software used: Push notification system (e.g., Firebase Cloud Messaging)

[1304] Specific prompt examples:

[1305] "Sightseeing information for Sensoji Temple and Kaminarimon has been updated. Tap to see more details."

[1306] Access the details page

[1307] When a user taps on the received push notification, the device displays a page with detailed information about the related tourist spot. The server sends the requested detailed data to the device, which then displays the information.

[1308] Specific prompt examples:

[1309] "Visit our detailed information page. Check out the attractions and restaurants."

[1310] This system will enable inbound tourists to obtain relevant tourist information in real time in a language they can understand, greatly improving the efficiency and convenience of tourist guidance.

[1311] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1312] Step 1:

[1313] When a user launches the app for the first time, they enter their name, gender, age, and preferred language on the initial setup screen. Input: User's name, gender, age, and preferred language. Output: The input data is formatted within the application. The device sends this input data to the server. The server receives the data and stores it in a database.

[1314] Step 2:

[1315] The device periodically obtains the user's current location using its GPS function. Input: Current location information obtained from the GPS. Output: The obtained location information is formatted within the device and converted into a format that can be sent. The device sends the location information to the server, which updates it in the database.

[1316] Step 3:

[1317] The server sends a prompt to the generation AI based on the received location information. Input: Location information, prompt text to send to the generation AI. Output: Tourist information generated by the generation AI. The server receives the generated tourist information and begins the process of translating it into the language set by the user.

[1318] Step 4:

[1319] The server uses a translation API to translate the generated tourist information into the user's language. Input: Generated tourist information, language selected by the user. Output: Tourist information translated by the translation API. The translated information is saved in the database.

[1320] Step 5:

[1321] The server creates a push notification based on the translated information. Input: Translated tourist information. Output: Message in push notification format. The server sends the push notification to the device.

[1322] Step 6:

[1323] The device displays the received push notification to the user. Input: Push notification sent from the server. Output: Notification message displayed to the user by the device. The user taps the notification to access the detailed information page.

[1324] Step 7:

[1325] When a user taps on a push notification, the device displays a detailed information page for the related tourist attraction. Input: User operation (tap), request for detailed information page. Output: Detailed information page is displayed. The server sends the detailed information page data to the device, which then displays the detailed information.

[1326] Specifically, after the user launches the app for the first time and completes the setup, the server generates tourist information using the generative AI model, translates the information, and provides it to the user via push notification. By tapping the notification, the user can access detailed tourist information.

[1327] (Application example 1)

[1328] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1329] While systems exist that provide local tourist information and restaurant recommendations in multiple languages, until now they have been limited to personal devices such as smartphones and tablets. As a result, tourists had to operate their devices to receive real-time multilingual tourist information while on the move. This led to issues such as reduced convenience and user experience.

[1330] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1331] In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for translating the generated information into a language set by the user, means for notifying the user's means of transportation of the generated multilingual information, means for providing information using voice recognition and voice output, and means for displaying information on a display. This enables tourists to receive multilingual tourist information and recommended dining spots in real time by voice and on a display while in an autonomous vehicle.

[1332] "Multilingual" refers to the ability to provide information in multiple languages.

[1333] "Regional tourist guide" refers to a guide that provides tourist information and introduces tourist spots related to a specific region.

[1334] "User Data" refers to personal information such as the user's name, gender, age, and language preference.

[1335] "Location Information" refers to the current geographic location of a user or vehicle obtained using technologies such as GPS.

[1336] "Generative AI" refers to a system that uses artificial intelligence to analyze data and generate specific information or recommendations.

[1337] "Multilingual information" refers to a form in which generated information is provided in multiple languages.

[1338] "User's means of transportation" refers to the means of transportation used by tourists, in this case, particularly autonomous vehicles.

[1339] "Voice recognition" refers to the technology of analyzing the user's voice, understanding its meaning, and then performing operations or responding.

[1340] "Audio output" refers to a technique for conveying information to a user by voice.

[1341] "Display" refers to a device that visually displays information.

[1342] "Push notification" refers to a notification message that is automatically sent from a server to a user's device.

[1343] "Infotainment system" refers to a system that integrates information and entertainment, especially an in-vehicle system.

[1344] The present invention provides a system for providing local tourist information in multiple languages, and is particularly suitable for self-driving vehicles. Specific configurations and processes for implementing the invention are described below.

[1345] The system mainly consists of the following components:

[1346] 1. User data input method:

[1347] The user uses the vehicle's touch panel or voice input to enter initial information such as name, gender, age, and preferred language.

[1348] This information is stored on the on-board computer and transmitted to a server.

[1349] 2. Location information acquisition means:

[1350] The vehicle's GPS function is used to obtain the current location in real time and send it to a server.

[1351] 3. Information generation using generative AI:

[1352] The server receives the location information and uses a generative AI model (e.g., OpenAI GPT-3) to generate information about nearby tourist attractions and restaurants.

[1353] The server uses a multilingual translation API (e.g., Google Translate API) to translate the generated information into the user's language of choice.

[1354] 4. Means of notification:

[1355] The translated information is sent to the car's infotainment system as a push notification.

[1356] Information is provided to the user through the voice assistant and the display.

[1357] As a specific example, a process will be described in which the user gets on a vehicle and selects English.

[1358] Example 1: Tourist information

[1359] 1. User provides input:

[1360] Get on the vehicle and select English on the in-car display.

[1361] Enter the initial information using the touch panel or voice input.

[1362] 2. Get location information:

[1363] The vehicle recognizes its current location as being near Tokyo Tower.

[1364] Location information is sent to the server via GPS.

[1365] 3. Information Generation and Translation:

[1366] The server generates information about tourist spots around Tokyo Tower using a generative AI model.

[1367] Translate into English using the translation API.

[1368] 4. Notification of Results:

[1369] The server creates a notification saying "Explore Tokyo: Don't miss Tokyo Tower and Roppongi Hills!".

[1370] The information is pushed to the in-vehicle infotainment system, displayed on the display, and announced by the voice assistant.

[1371] Specific Example 2: Recommendations for Dining Places

[1372] 1. The user makes an input:

[1373] The user selects Chinese on the in-vehicle voice assistant.

[1374] The user enters initial information.

[1375] 2. Obtain location information:

[1376] The vehicle recognizes the current location as Shinjuku.

[1377] The location information is sent to the server via GPS.

[1378] 3. Information generation and translation:

[1379] The server generates restaurant information around Shinjuku using a generation AI model.

[1380] It is translated into Chinese using a translation API.

[1381] 4. Notification of results:

[1382] The server creates a notification saying "Try the popular barbecue restaurants here in Shinjuku!".

[1383] The information is pushed to the in-vehicle infotainment system, displayed on the display, and announced by the voice assistant.

[1384] Examples of prompt sentences

[1385] Prompt: "Your current location is Shinjuku. Please generate information about nearby tourist spots and translate it into Chinese."

[1386] Expected output: "Shinjuku Station Area has many attractions, including... (specific spot information)"

[1387] This system will enable tourists to receive real-time multilingual tourist information and restaurant recommendations while they are in an autonomous vehicle, greatly improving convenience.

[1388] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1389] Step 1:

[1390] A user gets into a vehicle and uses a touch panel or voice assistant to input initial information such as name, gender, age, and preferred language. This user data is then stored in the vehicle's on-board computer and sent to the server. The input comes from the user, and the output is a dataset of the initial information.

[1391] Step 2:

[1392] The vehicle's GPS function is used to obtain the vehicle's current location in real time. The terminal that receives location information from the GPS sends this location information to the server. The input is location data from the GPS, and the output is the location information sent to the server.

[1393] Step 3:

[1394] Based on the location information received by the server, a generative AI model is used to generate information about nearby tourist attractions and restaurants. The server generates a prompt sentence and provides it as input to the AI ​​model. The input is the location information and the prompt sentence, and the output is the generated tourist information.

[1395] Step 4:

[1396] The server uses a multilingual translation API to translate the generated tourist information into the specified language. It passes the information and the target language as input to the translation API and obtains the translated information. The input is the generated tourist information and the target language, and the output is the translated information.

[1397] Step 5:

[1398] The server sends the translated tourist information to the vehicle's infotainment system as a push notification. The in-vehicle infotainment system receives the information and displays it on the vehicle's display, while also notifying the user via voice using the voice assistant. The input is the translated tourist information, and the output is the display and voice notification.

[1399] Step 6:

[1400] Users can access more information by tapping on the information displayed on the screen or asking a question to the voice assistant. This action allows them to obtain more detailed information or decide on the next action. The input is the user's tap or voice command, and the output is the provision of more information.

[1401] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1402] ---

[1403] The present invention relates to a system that provides local tourist information in multiple languages ​​to inbound tourists and combines it with an emotion engine that recognizes user emotions. In an embodiment of the present invention, a user uses a device such as a smartphone or tablet to receive local tourist information and restaurant recommendations, and is also provided with information customized based on their emotions.

[1404] This system mainly consists of the following components:

[1405] 1. User data input method:

[1406] The user starts the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. This information is sent to the server and stored in a database.

[1407] 2. Location information acquisition means:

[1408] The device periodically obtains the user's current location using the GPS function. This location information is sent to the server, where it is updated as the user's latest location information.

[1409] 3. Emotion engine means:

[1410] The device uses cameras and sensors to analyze the user's facial expressions and voice to recognize emotions, and the emotional information is sent to a server with the user's permission.

[1411] 4. Information generation methods using generative AI:

[1412] The server uses generative AI to generate information about nearby tourist spots and restaurants based on the location and emotion information it receives. The generated information is tailored to the user's emotions.

[1413] 5. Multilingual translation tools:

[1414] The server translates the generated tourist information into the language set by the user (e.g., English, Japanese, Chinese, etc.) and stores the translated information in a database.

[1415] 6. Means of notification:

[1416] The server sends the translated information to the user's device as a push notification, which is then displayed to the user in real time.

[1417] Specific examples

[1418] Example 1: Tourist information

[1419] 1. The user launches the app, selects Japanese, and allows emotional information collection.

[1420] 2. The device sends its current location to the server, indicating that it is in Asakusa. The emotion engine then analyzes the user's facial expression and determines that the user is relaxed.

[1421] 3. The server uses generative AI to generate information about tourist spots around Asakusa (e.g., Sensoji Temple, Kaminarimon Gate), creates recommendations for enjoying a relaxing sightseeing experience, and translates them into Japanese.

[1422] 4. The server creates a push notification saying "Relax and explore Asakusa: Take a leisurely stroll around Sensoji Temple!" and sends it to the device.

[1423] 5. The device displays a notification that the user can tap to access a page with more information.

[1424] Example 2: Dining recommendations

[1425] 1. The user launches the app, selects English, and allows emotional information collection.

[1426] 2. The device sends its current location to the server, indicating that it is in Ginza. The emotion engine then analyzes the user's voice and determines that the user is excited.

[1427] 3. The server uses generative AI to generate information about restaurants around Ginza (e.g., high-end sushi restaurants), creates recommendations to provide an exciting dining experience, and translates them into English.

[1428] 4. The server creates a push notification saying "Explore Ginza: Enjoy the thrilling experience at premium sushi restaurants nearby!" and sends it to the device.

[1429] 5. The device displays a notification that the user can tap to access a page with more information.

[1430] This system allows inbound tourists to receive relevant tourist information in real time in a language they understand, and also provides information that is tailored to their emotions, further enhancing their tourist experience and increasing their satisfaction.

[1431] The processing flow will be explained below.

[1432] --

[1433] Step 1:

[1434] The user starts the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. The device then sends this information to the server.

[1435] Step 2:

[1436] The server stores the received user information in a database, generates a user ID and an authentication token, and sends them back to the device. The device stores the authentication token in local storage.

[1437] Step 3:

[1438] The device periodically obtains the user's current location using its GPS function, and the obtained location information (latitude and longitude) is sent to the server along with an authentication token.

[1439] Step 4:

[1440] The server updates the received location information in a database and stores the user's latest location information.

[1441] Step 5:

[1442] The device uses cameras and sensors to analyze the user's facial expressions and voice to recognize emotions, and the recognized emotional information is sent to the server along with an authentication token.

[1443] Step 6:

[1444] The server stores the received emotional information in a database and provides it to the generative AI, which then generates information about nearby tourist spots and restaurants based on the location and emotional information.

[1445] Step 7:

[1446] The server translates the generated tourist information into the language selected by the user, and the translated information is saved in a database.

[1447] Step 8:

[1448] The server formats the translated information as a push notification message and prepares it to be sent to the user's device.

[1449] Step 9:

[1450] The server sends the push notification to the user's device, which then displays the received push notification to the user in real time.

[1451] Step 10:

[1452] Users can tap on the push notification to access a detailed information page, which displays detailed information about the recommended tourist spot or restaurant.

[1453] --

[1454] Example 2

[1455] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1456] Conventional tourist information systems were able to provide tourist information based on the user's location information, but lacked customization based on the user's emotional state. As a result, while there was a demand for information provision optimized for the user's emotions, there were issues with not being able to sufficiently increase user satisfaction. Other issues included a lack of multilingual support and the inability to provide information smoothly in real time.

[1457] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for acquiring and analyzing emotional information of the user, means for customizing the generated tourist information based on the emotional information, means for translating the generated information into a language set by the user, and means for notifying the user's terminal of the generated multilingual information. This makes it possible to provide tourist information in multiple languages ​​in real time according to the user's emotional state.

[1458] ---

[1459] ---

[1460] "User Data" refers to information such as a user's name, gender, age, language preference, and preferences regarding collection of emotional information.

[1461] "Location information" refers to information about the user's current location obtained by the GPS function of the device.

[1462] "Generative AI" refers to artificial intelligence that generates tourist information based on received input information and prompts.

[1463] "Emotion information" refers to information about the emotional state of the user that is acquired by analyzing the user's facial expressions and voice.

[1464] "Notification means" refers to a mechanism for sending and displaying the generated information as a push notification to the user's device.

[1465] "Multilingual information" refers to tourist information and recommendations translated into the language set by the user.

[1466] "Server" refers to a central processing unit that stores user data, analyzes location and emotional information, generates information using generative AI, and translates and notifies users of that information.

[1467] "Terminal" refers to a mobile information terminal such as a smartphone or tablet used by a user.

[1468] ---

[1469] The above are definitions of important words included in the scope of the patent claims.

[1470] ---

[1471] System Configuration

[1472] This invention is a system that provides multilingual local tourist information for inbound tourists, recognizing user emotions and providing customized information based on those emotions. Users access the system using devices such as smartphones or tablets to receive local tourist information and recommended dining spots. The main components of this system are a user data input means, a location information acquisition means, an emotion information acquisition means, an information generation means using generative AI, a multilingual translation means, and a notification means.

[1473] User Data Input

[1474] The user starts the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. This information is sent from the device to the server and stored in a database.

[1475] Location information acquisition

[1476] The device periodically acquires the user's current location using its GPS function, and this location information is sent to the server and updated in the server's database.

[1477] Emotional information acquisition

[1478] The device uses sensors such as a camera and microphone to collect the user's facial expressions and voice, and analyzes the emotional information. The analyzed emotional information is sent to a server and stored in a database.

[1479] Information generation by generative AI

[1480] The server uses a generative AI model to generate tourist information and restaurant recommendations based on the received location and emotion information. The generated information is optimized to match the user's emotions.

[1481] Prompt Sentence Examples

[1482] Please suggest some tourist spots in Asakusa for users who are relaxing.

[1483] Suggest a high-end sushi restaurant in Ginza for excited users.

[1484] Multilingual Translation

[1485] The generated information is translated by the server into the language set by the user, and the translated information is stored in a database.

[1486] Notifications and Indications

[1487] The server creates a push notification message from the translated information and sends it to the device. The device displays the received push notification to the user in real time. The user can tap the notification to access a page with more information.

[1488] Specific examples

[1489] Example 1: Tourist information

[1490] 1. The user launches the app, selects Japanese, and allows the collection of emotion information.

[1491] 2. The device sends its current location to the server, specifying Asakusa as its location.

[1492] 3. The device uses a camera to analyze the user's facial expressions and recognizes that they are relaxed.

[1493] 4. The server inputs a prompt to the generative AI model: "Please suggest tourist spots in Asakusa for a user who is relaxing."

[1494] 5. The generative AI model generates information about relaxing spots such as Sensoji Temple and Kaminarimon Gate.

[1495] 6. The server translates the generated information into Japanese and stores it in the database.

[1496] 7. The server creates a push notification saying "Relax and explore Asakusa: Take a leisurely stroll around Sensoji Temple!" and sends it to the device.

[1497] 8. The device displays a notification, and the user can tap the notification to access a page with more information.

[1498] Example 2: Dining recommendations

[1499] 1. The user launches the app, selects English, and allows emotional information collection.

[1500] 2. The device sends its current location to the server, specifying Ginza as its location.

[1501] 3. The device analyzes the user's voice using a microphone and recognizes that they are excited.

[1502] 4. The server inputs a prompt to the generative AI model: "Please suggest high-end sushi restaurants in Ginza for an excited user."

[1503] 5. A generative AI model generates restaurant information that provides exciting experiences, such as high-end sushi restaurants.

[1504] 6. The server translates the generated information into English and stores it in a database.

[1505] 7. The server creates a push notification saying "Explore Ginza: Enjoy the thrilling experience at premium sushi restaurants nearby!" and sends it to the device.

[1506] 8. The device displays a notification, and the user can tap the notification to access a page with more information.

[1507] This system allows inbound tourists to receive appropriate tourist information in real time in a language they can understand, and also provides information that is optimized to suit their emotions, further enriching their tourist experience and increasing their satisfaction.

[1508] ---

[1509] The above is the form for carrying out the invention.

[1510] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1511] ---

[1512] Step 1:

[1513] The user launches the app and enters their name, gender, age, preferred language, and permission to collect emotional information on the initial setup screen. Input data: name, gender, age, language, permission to collect emotional information. Output data: user data sent to the server. Specifically, the user enters the required information in each field and taps the send button. This causes the device to send the user data to the server, which then stores it in a database.

[1514] Step 2:

[1515] The device periodically obtains the user's current location using its GPS function. Input data: GPS information of the current location. Output data: Location information sent to the server. Specifically, the device periodically obtains GPS data in the background and sends it to the server. The server then stores the location information in a database and updates it with the latest location information.

[1516] Step 3:

[1517] The device uses a camera and microphone to collect the user's facial expressions and voice and recognizes emotions. Input data: user's facial expression data, voice data. Output data: emotional information sent to the server. Specifically, the device collects data in real time using the camera and microphone and identifies the user's emotions using an emotion analysis algorithm. The analysis results are sent to the server and stored in a database.

[1518] Step 4:

[1519] The server inputs a prompt sentence into the generative AI model based on the received location information and emotion information. Input data: location information, emotion information. Output data: generated tourist information. Specifically, the server generates a prompt sentence and inputs it into the generative AI model (e.g., "Please suggest tourist spots in Asakusa for a user who is relaxing"). The generative AI model generates appropriate tourist information and returns it to the server.

[1520] Step 5:

[1521] The server translates the generated tourist information into the language set by the user. Input data: tourist information. Output data: translated multilingual tourist information. Specifically, the server uses a translation engine to translate the generated information into the user's language (e.g., English, Japanese, Chinese, etc.). The translated information is stored in a database.

[1522] Step 6:

[1523] The server creates a push notification message from the translated information and sends it to the device. Input data: Translated multilingual tourist information. Output data: Push notification sent to the user's device. Specifically, the server creates the push notification text based on the translated information and sends it to the user's device as a push notification. The device displays the received notification to the user in real time.

[1524] Step 7:

[1525] The user taps the displayed notification to access the detailed information page. Input data: The body of the push notification. Output data: The display of the detailed information page. Specifically, when the user taps the notification, the device sends a request to the server to display the detailed information page. The server sends the detailed information to the device, and the device displays it to the user.

[1526] ---

[1527] The above is an explanation of the specific flow of the program processing of this system and each step.

[1528] (Application example 2)

[1529] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1530] Modern tourists, especially inbound tourists, face the challenge of finding it difficult to receive accurate and up-to-date tourist information in their native language. Another problem is that tourist information is generally generic, making it impossible to provide customized information tailored to the feelings and circumstances of individual tourists. This leads to a decline in the quality of the tourist experience and a lack of satisfaction.

[1531] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1532] In this invention, the server includes means for inputting user data, means for acquiring location information from the user, means for generating recommended tourist information based on the location information using generative AI, means for translating the generated information, means for notifying the generated multilingual information, means for using an emotion engine that recognizes the user's emotions, and means for customizing the tourist information based on the emotion information. This makes it possible to provide tourist information customized in multiple languages ​​in real time according to the emotions and state of each tourist.

[1533] "Means for inputting user data" refers to an interface through which a user inputs their name, gender, age, preferred language, and permission for collection of emotional information.

[1534] "Means for obtaining location information from a user" refers to a device or system that uses GPS functionality to automatically obtain the user's current location.

[1535] "Generative AI" refers to artificial intelligence technology for generating new information based on large amounts of data, and specifically includes models that perform natural language processing.

[1536] "Means for translating generated information" refers to a multilingual translation engine that translates information generated by generative AI into the user's native language.

[1537] "Means for notifying the generated multilingual information" refers to a system for sending the generated and translated information to the user's device as a push notification.

[1538] An "emotion engine that recognizes user emotions" refers to technology that uses cameras and sensors to analyze a user's facial expressions and voice and identify their emotions.

[1539] "Means for customizing tourist information based on emotional information" refers to a system that provides tourist information and recommended information appropriate to the user's state based on their emotional data.

[1540] System configuration and main processing

[1541] The present invention is a system that provides local tourist information in multiple languages ​​and also recognizes the user's emotions to provide customized information. The system consists of the following main components:

[1542] 1. User data input method

[1543] When users enter an autonomous vehicle or launch an application, they enter their name, gender, age, preferred language, and permission to collect emotional information through the interface. This information is sent to the vehicle's computer and stored in a database.

[1544] 2. Location information acquisition means

[1545] The device periodically obtains the user's current location using GPS, and this location information is sent to the vehicle's computer system for updates.

[1546] 3. Emotional Engine Means

[1547] The device uses a camera and microphone installed in the vehicle to capture the user's facial expressions and voice, which are then analyzed by an emotion recognition engine (e.g., Microsoft Azure Face API, Google Cloud Vision API) to generate emotion data, which is then sent to the vehicle's computer.

[1548] 4. Information generation methods using generative AI

[1549] The server uses generative AI (e.g., OpenAI GPT-4) to generate tourist information appropriate for the user based on the received location and emotion information. An example of input used for generation is a prompt sentence such as "Tell me where I can relax around Ginza."

[1550] 5. Multilingual Translation Tools

[1551] The server translates the generated information into the language set by the user using a multilingual translation engine (e.g., Microsoft Translator, Google Translate), and the translated information is stored in a database.

[1552] 6. Means of notification

[1553] The device or vehicle's interface system will send the translated information as a push notification, and the user will receive the information as audio or text and can select the tourist attractions or restaurants they want to visit.

[1554] Natural language explanations

[1555] The user data input means consists of an interface implementation that allows users to input their name, gender, language preference, etc. when entering an application or an autonomous vehicle. This information is sent to the server and used for subsequent customization processing.

[1556] The location information acquisition means acquires the user's current location using a GPS module and transmits that information to the server. This allows the location information to be constantly updated in real time, and tourist information is provided based on the latest information.

[1557] The emotion engine means uses a camera and microphone to capture the user's facial expressions and voice, and processes them with an emotion recognition engine such as Microsoft Azure Face API or Google Cloud Vision API. This data is sent to a server as emotion information, and the user's emotional state is measured.

[1558] The information generation method using generative AI generates tourist information based on location information and emotion information using generative AI such as OpenAI GPT-4. For example, it uses the prompt sentence "Tell me about places where I can relax around Ginza" to generate tourist information that is optimal for the user.

[1559] The multilingual translation means translates the generated tourist information into the language set by the user, using a multilingual translation engine such as Microsoft Translator or Google Translate, and stores the translated information in a database.

[1560] The notification means sends the translated information to the user's device or the interface system of the autonomous vehicle as a push notification, allowing the user to receive appropriate tourist information in real time.

[1561] Specific examples

[1562] 1. A Japanese tourist gets into a self-driving vehicle, selects Japanese, and allows emotional information to be collected.

[1563] 2. The vehicle's GPS verifies its current location as Ginza and transmits this information to the computer system.

[1564] 3. The emotion engine recognizes a relaxed state from facial expressions.

[1565] 4. The computer system inputs "places to relax around Ginza" as a prompt sentence into the generative AI, and generates appropriate tourist information.

[1566] 5. The generated information is translated into Japanese using a multilingual translation engine and stored in a database.

[1567] 6. The vehicle's interface system presents the location to the user as a "quiet and enjoyable place in Ginza."

[1568] This system allows users to receive the most appropriate tourist information based on their emotions and current location in real time in multiple languages.

[1569] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1570] Step 1:

[1571] When a user enters a vehicle, they enter their name, gender, age, preferred language, and permission to collect emotional information on a tablet or interface.

[1572] Input: User's name, gender, age, preferred language, and permission to collect emotional information.

[1573] Output: The input information is stored in the vehicle's database.

[1574] In this step, basic user data is collected and stored in a database for subsequent customization.

[1575] Step 2:

[1576] The device uses its GPS function to obtain the user's location information in real time and transmits it to the server.

[1577] Input: None (automatic location acquisition via GPS).

[1578] Output: Current location data (latitude, longitude) is sent to the server and location information is updated.

[1579] This process allows the user's current location to be determined in real time, allowing tourist information to be provided based on appropriate location information.

[1580] Step 3:

[1581] The device's camera and microphone are used to capture the user's facial expressions and voice, which are then analyzed using an emotion recognition engine.

[1582] Input: User's facial expression image data, voice data.

[1583] Output: Emotional data (e.g., relaxed state, excited state) is generated and sent to the server.

[1584] Here, raw data captured by a camera and microphone is processed by an emotion recognition engine and converted into concrete data representing the user's current emotional state.

[1585] Step 4:

[1586] The server integrates location information and emotional information and uses generative AI to generate tourist guides based on that information.

[1587] Input: location information (latitude, longitude), emotional information (e.g., relaxation state).

[1588] Output: Generated tourist information (text format).

[1589] The server uses a generative AI model (e.g., OpenAI GPT-4) to generate specific tourist information based on the input data, providing instructions to the AI ​​in the form of a prompt such as "Tell me where I can relax around Ginza."

[1590] Step 5:

[1591] The server translates the generated tourist information into the language set by the user.

[1592] Input: Generated tourist information (original language).

[1593] Output: Translated tourist information (user-defined language, e.g. Japanese).

[1594] Using a translation engine (e.g., Microsoft Translator, Google Translate), the generated tourist information is translated into the language set by the user and stored in a database.

[1595] Step 6:

[1596] The interface system of the terminal or vehicle sends the translated tourist information to the user as a push notification.

[1597] Input: Translated tourist information (user-preferred language).

[1598] Output: Tourist information is displayed on the user's device as a push notification.

[1599] Users can receive notifications and tap to access more information, including detailed directions to tourist attractions and restaurants.

[1600] This series of steps creates a system that can provide multilingual tourist information customized according to the user's emotions and current location.

[1601] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1602] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1603] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1604] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1605] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1606] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1607] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1608] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1609] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1610] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1611] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1612] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1613] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1614] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1615] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1616] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1617] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1618] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1619] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1620] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1621] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1622] The following is further disclosed regarding the above embodiment.

[1623] (Claim 1)

[1624] A system that provides local tourist information in multiple languages,

[1625] means for inputting user data;

[1626] means for obtaining location information from a user;

[1627] A means of generating recommended tourist information based on location information using generative AI;

[1628] means for translating the generated information into a language selected by the user;

[1629] The system includes a means for notifying the generated multilingual information to a user's terminal.

[1630] (Claim 2)

[1631] means for transmitting location information acquired from a user to a server;

[1632] 10. The system of claim 1, further comprising means for updating location information by a server.

[1633] (Claim 3)

[1634] means for transmitting the generated tourist information to a user's terminal as a push notification;

[1635] 10. The system of claim 1, further comprising: means for displaying the push notification.

[1636] "Example 1"

[1637] (Claim 1)

[1638] means for inputting user data;

[1639] means for obtaining location information from a user;

[1640] A means of generating recommended tourist information based on location information using generative AI;

[1641] means for translating the generated information into a language selected by the user;

[1642] means for sending the translated information to the user's device as a push notification;

[1643] A way for users to tap on the push notification to access a page with more information;

[1644] A system including:

[1645] (Claim 2)

[1646] means for transmitting location information acquired from a user to a server;

[1647] 10. The system of claim 1, wherein the location information is updated by a server.

[1648] (Claim 3)

[1649] means for transmitting the generated tourist information to a user's terminal as a push notification;

[1650] 10. The system of claim 1, wherein the system displays a push notification.

[1651] "Application Example 1"

[1652] (Claim 1)

[1653] A system that provides local tourist information in multiple languages,

[1654] means for inputting user data;

[1655] means for obtaining location information from a user;

[1656] A means of generating recommended tourist information based on location information using generative AI;

[1657] means for translating the generated information into a language selected by the user;

[1658] a means for notifying the generated multilingual information to the user's means of transportation;

[1659] means for providing information using speech recognition and speech output;

[1660] A system including means for displaying information on a display.

[1661] (Claim 2)

[1662] means for transmitting location information acquired from a user to a server;

[1663] 10. The system of claim 1, further comprising means for updating location information by a server.

[1664] (Claim 3)

[1665] means for transmitting the generated tourist information as a push notification to an infotainment system of the user's vehicle;

[1666] 10. The system of claim 1, further comprising means for presenting the push notification by voice and on a display.

[1667] "Example 2: Combining Emotion Engines"

[1668] ---

[1669] (Claim 1)

[1670] means for inputting user data;

[1671] means for obtaining location information from a user;

[1672] A means of generating recommended tourist information based on location information using generative AI;

[1673] A means for acquiring and analyzing user emotion information;

[1674] a means for customizing the generated tourist information based on the emotion information;

[1675] means for translating the generated information into a language selected by the user;

[1676] The system includes a means for notifying the generated multilingual information to a user's terminal.

[1677] (Claim 2)

[1678] means for transmitting location information acquired from a user to a server;

[1679] 10. The system of claim 1, further comprising means for updating location information by a server.

[1680] (Claim 3)

[1681] means for transmitting the generated tourist information to a user's terminal as a push notification;

[1682] 10. The system of claim 1, further comprising: means for displaying the push notification.

[1683] ---

[1684] Now, we have identified the scope of the patent claim by utilizing a hypernym, without using proper nouns or company names.

[1685] "Application example 2 when combining emotion engines"

[1686] (Claim 1)

[1687] A system that provides local tourist information in multiple languages,

[1688] means for inputting user data;

[1689] means for obtaining location information from a user;

[1690] A means of generating recommended tourist information based on location information using generative AI;

[1691] means for translating the generated information into a language selected by the user;

[1692] means for notifying the generated multilingual information to a user's terminal;

[1693] means for using an emotion engine to recognize the emotion of a user;

[1694] A system including means for customizing tourist information based on emotion information.

[1695] (Claim 2)

[1696] means for transmitting location information acquired from a user to a server;

[1697] 10. The system of claim 1, further comprising means for updating location information by a server.

[1698] (Claim 3)

[1699] means for transmitting the generated tourist information to a user's terminal as a push notification;

[1700] 10. The system of claim 1, further comprising: means for displaying the push notification. [Explanation of symbols]

[1701] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A system that provides local tourist information in multiple languages, means for inputting user data; means for obtaining location information from a user; A means of generating recommended tourist information based on location information using generative AI; means for translating the generated information into a language selected by the user; The system includes a means for notifying the generated multilingual information to a user's terminal.

2. means for transmitting location information acquired from a user to a server; 10. The system of claim 1, further comprising means for updating location information by a server.

3. means for transmitting the generated tourist information to a user's terminal as a push notification; The system of claim 1 , further comprising: means for displaying the push notification.

Images