System

Abstract

A system is provided.SOLUTION: A system comprising: means for acquiring position information of users and transmitting the position information to a server; means for searching a database of sightseeing spots, restaurants, and accommodation facilities based on a language set by the users and providing information in multiple languages; means for providing detailed transfer guidance of public transportation; means for collecting feedback from the users and updating the database; and means for translating the language using a generation-related AI.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Existing train transfer guide apps are designed for users with a certain level of knowledge about public transportation, making them difficult to use for people who do not regularly use public transportation or for foreign tourists visiting Japan. This makes it difficult to travel and sightsee smoothly, and users who are not fluent in Japanese have particular difficulty obtaining information and making reservations. Furthermore, the existing system does not fully utilize user feedback, and the accuracy of the service is not being improved. [Means for solving the problem]

[0005] To solve this problem, the present invention provides the following means. It provides a system that includes means for acquiring a user's location information and sending it to a server, and searches a database of tourist spots, restaurants, and accommodations based on the language set by the user. The search results are then provided in multiple languages. It also includes means for providing users with detailed public transportation transfer information. It also aims to improve the accuracy of the information by collecting feedback from users and using means for continuously updating the database. It also introduces means for automatically translating languages ​​using generative AI, making it multilingual compatible. This provides a system that supports smooth and comfortable travel and sightseeing for foreign visitors to Japan and people who do not regularly use public transportation.

[0006] "User location information" refers to geographic coordinate data obtained by devices such as smartphones using GPS or other means.

[0007] A "server" is a computer system that sends, receives, and processes data over a network.

[0008] "Language settings" are settings that allow users to select their own language and change the display and output of the entire system to that language.

[0009] "Tourist spots" is a general term for places and attractions that tourists visit.

[0010] A "restaurant" is a facility or store established to serve meals.

[0011] "Accommodation" means a place or facility provided for the temporary lodging of travelers.

[0012] A "database" is a system that efficiently stores large amounts of data and manages it in a way that makes it searchable and usable.

[0013] "Multilingual" refers to the ability of a system to provide information in multiple languages.

[0014] "Generative AI" is a type of artificial intelligence technology, an algorithm that generates natural language expressions in response to specific input.

[0015] "Public transportation" is a general term for transportation means such as trains, buses, and taxis that provide transportation services available to the general public.

[0016] "Transfer information" provides detailed information on which lines and means of transport to use when traveling from one point to another by public transport.

[0017] "Feedback" refers to opinions and ratings provided by users regarding services and experiences.

[0018] A "discount coupon" is a special coupon that offers a discount on usage fees under certain conditions. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention is a system that provides detailed transfer information, tourist attractions, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation. The system provides optimal information based on the user's location information and achieves multilingual support using generative AI.

[0041] System configuration

[0042] 1. Server-side processing

[0043] The server is responsible for the following processes:

[0044] 1. Building and updating the database

[0045] The server collects information on tourist spots, restaurants, and accommodations and stores it in a database.

[0046] Periodically retrieve the latest information from multiple external data sources and update the database.

[0047] 2. Multilingual support

[0048] Generative AI is used to translate collected information into multiple languages.

[0049] Select the appropriate translation data based on the language set by the user.

[0050] 3. Demand forecasting

[0051] The collected data will be analyzed and tourism demand will be predicted.

[0052] Based on the results of demand forecasting, we propose new business opportunities.

[0053] 2. Terminal processing

[0054] The device (such as a smartphone) is responsible for the following processes:

[0055] 1. Providing a user interface

[0056] Provide an interface designed for user ease of use.

[0057] You can change the language in the settings screen and search for destinations and points of interest.

[0058] 2. Obtaining location information

[0059] The device uses GPS to obtain the user's current location.

[0060] 3. Displaying Data

[0061] Display multilingual information received from the server.

[0062] Displays detailed information about tourist spots and specific transfer information.

[0063] 3. User-side processing

[0064] The user performs the following operations using the app.

[0065] 1. Installing the app and initial settings

[0066] The user downloads and installs the app.

[0067] After launching the app, set the language.

[0068] 2. Search and provide feedback

[0069] Users can search for their destination and check detailed information and transfer information.

[0070] After using the service, provide feedback and receive discount coupons.

[0071] Specific examples of program processing

[0072] Example: A visit to Sensoji Temple

[0073] 1. Information collection and multilingual translation

[0074] The server collects tourist spot information and translates it into multiple languages ​​using generative AI.

[0075] Information about Sensoji Temple will be stored in a database and made available in multiple languages, including English and Chinese.

[0076] 2. User Search

[0077] The user launches the app and sets the language to English.

[0078] Enter "Sensoji Temple" in the app's search bar and tap the search button.

[0079] 3. Displaying Information

[0080] The device obtains the user's current location and sends it to the server.

[0081] The server sends multilingual information about Sensoji Temple to the terminal, which then displays the information in English.

[0082] 4. Route guidance

[0083] When the user requests route guidance, the device again obtains the current location and sends it to the server.

[0084] The server calculates the optimal transfer route and sends it to the terminal.

[0085] The device displays detailed transfer information, allowing the user to check train and bus transfer information.

[0086] 5. Feedback and Discount Coupons

[0087] After visiting a tourist attraction, users provide feedback within the app.

[0088] The server receives the feedback and updates the database.

[0089] The server issues a discount coupon and sends it to the terminal, which then displays the coupon to the user.

[0090] In this way, the present invention is a system that provides users with appropriate tourist information and transportation guidance in multiple languages ​​and utilizes feedback to improve services.

[0091] The processing flow will be explained below.

[0092] Step 1:

[0093] The user downloads and installs the app.

[0094] Step 2:

[0095] The user launches the app and selects the language they want to use (e.g., English) on the language settings screen.

[0096] Step 3:

[0097] The device will save the user's language preference and change all displays to the selected language.

[0098] Step 4:

[0099] The user enters the location they want to go to (for example, "Sensoji Temple") in the app's search bar and taps the search button.

[0100] Step 5:

[0101] The device obtains the user's current location information using its GPS function and sends that information and the search query ("Sensoji Temple") to the server.

[0102] Step 6:

[0103] Based on the current location information and search query received by the server, the server searches the database for relevant tourist spot information.

[0104] Step 7:

[0105] The tourist spot information acquired by the server is translated into the language set by the user (English) using generative AI.

[0106] Step 8:

[0107] The server transmits the translated tourist spot information to the terminal.

[0108] Step 9:

[0109] The information received by the device is displayed on the app screen, providing the user with detailed information about the tourist spot (location, opening hours, photos, etc.).

[0110] Step 10:

[0111] The user taps the "Directions" button to request directions to their destination.

[0112] Step 11:

[0113] The device again acquires the user's current location and sends it to the server.

[0114] Step 12:

[0115] The server calculates the optimal route from the current location to the destination and generates detailed transfer information.

[0116] Step 13:

[0117] The server transmits the calculated route guidance information to the terminal.

[0118] Step 14:

[0119] The terminal displays the route guidance information it receives on the screen, providing the user with details such as the means of transport to be used, the line to be taken, transfer stations, and travel time.

[0120] Step 15:

[0121] After a user visits a tourist attraction, they tap the "Feedback" button in the app to open a feedback form.

[0122] Step 16:

[0123] The user enters their opinion and rating in the feedback form and taps the submit button.

[0124] Step 17:

[0125] The terminal transmits the user's feedback information to the server.

[0126] Step 18:

[0127] The server stores the received feedback information in a database and performs statistical analysis to update the database.

[0128] Step 19:

[0129] The server generates a discount coupon that can be used next time for the user who provided the feedback, and transmits the information to the terminal.

[0130] Step 20:

[0131] The terminal displays discount coupon information on the application screen and provides the user with a coupon code.

[0132] Example 1

[0133] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0134] For foreign visitors to Japan and those who do not regularly use public transportation, it is difficult to provide information on tourist attractions, restaurants, accommodations, and detailed transfer directions in the appropriate language, and to improve the quality of the service by incorporating user feedback. It is also not easy to predict tourism demand or propose new business opportunities. To solve these challenges, a system that uses generative AI models to achieve multilingual support and provide users with optimal information and services in real time is needed.

[0135] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0136] In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for searching a database of tourist facilities, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for collecting feedback from users and updating the database, means for translating languages ​​using a generative AI model, means for forecasting demand and proposing new business opportunities, and means for providing real-time information based on the user's current location. This makes it possible to provide detailed tourist information and transfer information in multiple languages ​​to foreign visitors to Japan and people who do not regularly use public transportation, and to improve the quality of the service based on feedback.

[0137] "User location information" means data indicating the current location of a person holding a device, and is information obtained using GPS or other location identification technology.

[0138] "Tourist facilities" refer to tourist spots and tourist destinations that users would like to visit, and include historical buildings, parks, museums, etc.

[0139] "Food and beverage establishment" means a place where customers can enjoy food and beverages, and includes restaurants, cafes, bars, etc.

[0140] "Accommodation" refers to a place where users can stay overnight, and includes hotels, guesthouses, and other accommodations.

[0141] "Means of providing information in multiple languages" refers to a system that provides information on tourist facilities, restaurants, and accommodations in the language selected by the user, and refers to the use of machine translation and multilingual databases.

[0142] "Means for providing detailed transfer information for public transport" refers to a system that provides detailed information on the optimal transfer routes and methods for users to reach their destination, including train and bus operation information.

[0143] "Means for collecting feedback and updating the database" refers to a system for collecting user ratings and comments, updating the database information based on them, and improving the quality of the service.

[0144] "Means for translating languages ​​using generative AI models" refers to a system that uses generative artificial intelligence models to translate text into different languages, utilizing natural language processing technology.

[0145] "Means of forecasting demand and proposing new business opportunities" refers to a system that analyzes trends in tourism demand based on collected data and finds new business opportunities.

[0146] "Means for providing real-time information" refers to a mechanism that instantly provides the latest information based on the user's current location, including dynamic searches and information updates.

[0147] This invention is a system that provides detailed transfer information, tourist facilities, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation. This system provides optimal information based on the user's location information and achieves multilingual support using a generative AI model. A specific method for implementing this system is described below.

[0148] Server-side processing

[0149] The server uses the following hardware and software to collect, translate, and analyze various information.

[0150] 1. Building and updating the database

[0151] The server collects information on tourist facilities, restaurants, and accommodations from external data sources and stores it in a database. It periodically retrieves and updates the latest information using a web crawler or API. Specifically, MySQL or PostgreSQL is used as the database.

[0152] 2. Multilingual support

[0153] A generative AI model is used to translate the collected information into multiple languages. Specifically, OpenAI's GPT model and Google Translate API are used, and the translation results are stored in a database. An example prompt is as follows:

[0154] Prompt: "Translate the following information about the tourist spot 'Sensoji Temple' into English, Chinese, and Korean: [More information about the tourist spot]"

[0155] 3. Demand forecasting

[0156] The server analyzes the collected data using data science tools (e.g., TensorFlow and Scikit-learn) to predict tourism demand, which will then suggest new business opportunities.

[0157] Terminal side processing

[0158] The device (smartphone, etc.) operates with the following configuration.

[0159] 1. Providing a user interface

[0160] The device uses React Native and Flutter to provide a user-friendly interface, including a search bar, settings menu, history display, and more.

[0161] 2. Obtaining location information

[0162] The device uses the smartphone's GPS to obtain the user's current location, using Google Maps API or Apple's Core Location.

[0163] 3. Displaying Data

[0164] The terminal displays the multilingual information received from the server, including detailed information on tourist attractions and transfer information, so that the user can check it.

[0165] User operation

[0166] The user performs the following operations using the app.

[0167] 1. Installing the app and initial settings

[0168] Users download and install the app from the App Store or Google Play Store, and set the language when they first launch it.

[0169] 2. Search and provide feedback

[0170] Users search for destinations within the app and check the displayed information. After visiting a tourist spot, they can use the in-app feedback function to enter and submit their ratings and comments. This allows the server to update the database and issue discount coupons.

[0171] Example: A visit to Sensoji Temple

[0172] For example, when a user visits Sensoji Temple, information and guidance are provided in the following manner.

[0173] 1. The server collects information about Sensoji Temple from external data sources and translates it into multiple languages ​​using a generative AI model.

[0174] 2. The user launches the app and changes the language setting to English.

[0175] 3. The user enters "Sensoji Temple" and searches.

[0176] 4. The device obtains its current location using GPS and sends it to the server.

[0177] 5. The server sends the corresponding multilingual information to the terminal, which then displays the information in English.

[0178] 6. The user requests route guidance, and the device again obtains the current location and sends it to the server.

[0179] 7. The server calculates the optimal transfer route and sends it to the device, which then displays detailed transfer information.

[0180] 8. After visiting a tourist attraction, users can provide feedback and receive discount coupons.

[0181] In this way, the present invention is a system that provides users with appropriate tourist information and transportation guidance in multiple languages ​​and utilizes feedback to improve services.

[0182] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0183] System program processing flow:

[0184] Step 1: Collecting data and storing it in a database

[0185] The server collects information about tourist facilities, restaurants, and accommodations from external data sources (such as APIs and web crawlers). The server organizes this data and stores it in a database such as MySQL or PostgreSQL. The input includes information from external data sources, and the output is stored in the database.

[0186] Step 2: Multilingual Translation

[0187] The server translates the information stored in the database into multiple languages ​​using a generative AI model (e.g., GPT-4). The input includes the original sentence retrieved from the database, and the output is the translated data into multiple languages. The following is used as an example of a prompt sentence:

[0188] Prompt: "Translate the following information about the tourist spot 'Sensoji Temple' into English, Chinese, and Korean: [More information about the tourist spot]"

[0189] The server stores the translated data back in the database.

[0190] Step 3: Obtaining user location information

[0191] The device obtains the user's current location using the smartphone's GPS function. The input includes the device's GPS data, and the output is the latitude and longitude of the user's current location. Specifically, Google Maps API or Apple's Core Location is used.

[0192] Step 4: Providing a User Interface

[0193] The device uses React Native and Flutter to provide an interface that makes it easy for users to search and configure settings. Input includes user actions (search keywords and setting changes), and the appropriate screen is displayed as output.

[0194] Step 5: Providing multilingual information

[0195] The device sends a request to the server based on the user's current location and the language selected. The server receives the request, retrieves the corresponding language translation data from the database, and sends it to the device. The input includes the user's location information and selected language, and the output is multilingually translated tourist information.

[0196] Step 6: Provide detailed transit directions

[0197] The server calculates the optimal transit route based on the user's current location and destination information. The input includes the user's current location and destination, and the output is a detailed transit route. The optimal route is obtained using Google Maps API and other public transport APIs.

[0198] Step 7: Gather feedback and update the database

[0199] The terminal sends the user-provided feedback to the server, which receives the feedback and updates the database. The input contains the user's feedback data, and the output is an updated database.

[0200] Step 8: Issue a discount coupon

[0201] The server generates a discount coupon after the user provides the feedback and sends it to the terminal. The input includes the feedback status, and the generated coupon is obtained as an output. The terminal displays the coupon information to the user.

[0202] Specific flow of processing steps

[0203] Step 1: Collecting data and storing it in a database

[0204] The server retrieves information about tourist facilities, restaurants, and accommodations from tourism websites and related databases via API. The retrieved data is then properly organized and stored in a MySQL or PostgreSQL database.

[0205] Step 2: Multilingual Translation

[0206] The server takes each piece of data in the database and translates it into each language using a generative AI model. The translated data is then stored back in the database and made available in each language.

[0207] Step 3: Obtaining user location information

[0208] The device uses GPS to obtain the latitude and longitude of the current location, which is then sent to the server, where it is used to provide information appropriate to the current location.

[0209] Step 4: Providing a User Interface

[0210] The device uses React Native and Flutter to display an intuitive user interface, including a home screen with a search bar, settings menu, and history view, where users can search for destinations and change language settings.

[0211] Step 5: Providing multilingual information

[0212] The device requests the user's current location and preferred language setting from the server, which retrieves multilingual tourist information from a database and sends it in JSON format to the device, which then displays the data in the appropriate format.

[0213] Step 6: Provide detailed transit directions

[0214] When the user requests transit directions, the device again retrieves its current location and sends it to the server. The server uses the appropriate transport API to calculate detailed transit routes and sends them to the device. The device then displays specific transit directions, helping the user reach their destination easily.

[0215] Step 7: Gather feedback and update the database

[0216] When a user provides feedback on the tourist attractions they visited or the services they used, the device sends the feedback data to the server, which then incorporates the feedback into a database and updates the information, thereby providing other users with the latest information.

[0217] Step 8: Issue a discount coupon

[0218] After the user provides feedback, the server generates a discount coupon and sends it to the terminal, which displays the coupon to the user so that it can be applied the next time they use the service.

[0219] (Application example 1)

[0220] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0221] Although systems exist that allow tourists to access detailed tourist information in multiple languages ​​and receive optimal tourist guidance based on their location in real time, there is a lack of methods to link information with real-world scenery and multilingual audio guides. There is also a need for tourist route suggestions tailored to individual needs and for service improvements based on feedback.

[0222] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0223] In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for collecting user feedback and updating the database, means for translating languages ​​using generative AI, means for translating information about tourist attractions and restaurants into multiple languages ​​using a generative AI model, means for guiding tourist routes and places of interest based on the user's current location, means for displaying tourist attraction information overlaid on real-world scenery using augmented reality (AR) technology, and means for providing multilingual audio guides. This allows tourists to obtain detailed, multilingual tourist information in real time, improving their local tourist experience.

[0224] "User Location Information" means your current geographic location obtained using GPS or Wi-Fi.

[0225] A "server" refers to a computer system that provides databases and applications via a network.

[0226] "Tourist attractions" refer to places of interest or facilities that tourists visit in a particular area.

[0227] "Restaurant" refers to an establishment that serves food and drinks.

[0228] "Accommodation facilities" refer to hotels, inns, and other establishments where travelers can stay temporarily.

[0229] A "database" refers to a system for organizing and storing information.

[0230] "Means for providing information in multiple languages" refers to the function of providing users with necessary information in multiple languages.

[0231] "Public transportation" refers to transportation systems such as buses and trains used by the general public.

[0232] "Transfer information" refers to providing detailed route information when users transfer between different public transportation modes.

[0233] "Feedback" refers to the evaluations and opinions provided by users after using a system or service.

[0234] "Generative AI" refers to technology that uses artificial intelligence to generate new language data and translations.

[0235] "Multilingual audio guide" refers to a system that provides audio guidance in multiple languages.

[0236] "Augmented reality (AR) technology" refers to the technology of overlaying digital information onto real-world scenery.

[0237] This invention relates to a tourism information provision system that allows tourists to receive detailed tourist information in multiple languages. This system utilizes user location information, a database of tourist spots, restaurants, and accommodations, multilingual translation, public transportation transfer information, feedback collection, generative AI, multilingual audio guides, and augmented reality (AR).

[0238] Hardware and software used

[0239] 1. Hardware

[0240] Smartphone (GPS, camera)

[0241] Smart glasses or head-mounted display (AR display, GPS)

[0242] 2. Software

[0243] Mobile app (compatible with iOS / Android)

[0244] Generative AI models (e.g., GPT-4)

[0245] Location services (Google Maps API)

[0246] Text-to-speech service (e.g., Amazon Polly)

[0247] AR Kit (ARCore, ARKit)

[0248] System Overview

[0249] Server Features

[0250] The server builds a database of information on tourist attractions, restaurants, and accommodations, and updates it regularly. The collected information is translated into multiple languages ​​using a generative AI model and provided based on the language selected by the user. The server also collects user feedback and updates the database.

[0251] Generative AI models are used to translate information into multiple languages, such as displaying detailed information about tourist attractions from Japanese to English.

[0252] Device Features

[0253] The device uses GPS to acquire the user's current location and sends it to the server. It also provides a user interface and displays information about tourist spots searched for by the user in multiple languages. It also provides public transport transfer information to support the user's travels.

[0254] When using the augmented reality (AR) function, information is superimposed on the real world via smart glasses or a head-mounted display. For example, when a user arrives at a tourist spot, the area's history and attractions can be displayed in AR. A multilingual audio guide function is also provided, providing detailed audio guidance about tourist spots.

[0255] User operations

[0256] Users download the mobile app and set their language preferences. When they launch the app, the app acquires their location information and provides optimal tourist information based on their current location. Users can search for tourist attractions and check detailed information and public transportation transfer information. When they arrive at a tourist attraction, they can effectively obtain tourist information using AR displays and multilingual audio guides.

[0257] Examples of concrete examples and prompts

[0258] Specific examples

[0259] When you visit an attraction:

[0260] Launch the app and get your location using GPS.

[0261] Multilingual tourist attraction information is obtained from the server.

[0262] Smart glasses display AR information about tourist attractions in real-world scenes.

[0263] Audio guides explain the history of the attractions.

[0264] Prompt example

[0265] "Translate the Japanese information about tourist attractions into English and list five must-see spots."

[0266] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0267] Step 1:

[0268] A user launches an app on their smartphone and sets the language. The input is the user's selected language setting information, and the output is the initial settings screen of the app. The app then sends the language setting information to the server.

[0269] Step 2:

[0270] The device uses GPS to obtain the user's current location. The input is GPS data, and the output is the current location information. This location information is converted into a digital format and sent to the server.

[0271] Step 3:

[0272] The server searches the database for information on tourist attractions, restaurants, and accommodations based on the received location information. The input is location information and language setting information, and the output is tourist information related to the user's current location. This information is translated into multiple languages ​​using a generative AI model.

[0273] Step 4:

[0274] The server sends multilingual tourist information translated by a generative AI model to the terminal. The input is the translated tourist information, and the output is multilingual tourist information. The terminal displays this information in a user interface.

[0275] Step 5:

[0276] Users wear smart glasses or a head-mounted display and view tourist spot information in real time. The input is real-world scenery and tourist information, and the output is information overlaid using AR technology. The device uses a camera to capture the real-world scenery and overlays tourist information on it.

[0277] Step 6:

[0278] When a user arrives at a tourist spot, the device uses a speech synthesis service to provide a multilingual audio guide. The input is detailed information about the tourist spot and the user's preferred language, and the output is a speech synthesis guide voice. The speech synthesis service is used to generate and play the audio guide on the spot.

[0279] Step 7:

[0280] After viewing or using the tourist attraction information, the user provides feedback through the app. The input is the feedback information from the user, and the output is the feedback data. This feedback data is sent to the server and updates the database.

[0281] Step 8:

[0282] The server updates the database based on the feedback and issues discount coupons as necessary. The input is the feedback data and the user's usage history, and the output is the discount coupon information. The coupon is sent in digital format to the terminal and displayed to the user.

[0283] Step 9:

[0284] The server predicts inbound demand based on data on tourist spots, restaurants, and accommodations. The input is past usage data and tourist information, and the output is the demand forecast results. The server analyzes this data and predicts tourist demand.

[0285] Step 10:

[0286] The server creates and proposes new tourist spot and event information based on prompts generated using a generative AI model. The input is the prompt and current tourist information, and the output is new tourist suggestion information. An example prompt is, "Translate the Japanese information about tourist spots into English and list five highlights."

[0287] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0288] This invention is a system that provides detailed transfer information, tourist attractions, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation, and also combines it with an emotion engine that recognizes the user's emotions. It provides more appropriate information based on the user's location information and emotion data, and achieves multilingual support using generative AI.

[0289] System configuration

[0290] 1. Server-side processing

[0291] The server is responsible for the following processes:

[0292] 1. Building and updating the database

[0293] The server collects information on tourist spots, restaurants, and accommodations and stores it in a database.

[0294] Periodically retrieve the latest information from multiple external data sources and update the database.

[0295] 2. Multilingual support

[0296] Generative AI is used to translate collected information into multiple languages.

[0297] Select the appropriate translation data based on the language set by the user.

[0298] 3. Demand forecasting

[0299] The collected data will be analyzed and tourism demand will be predicted.

[0300] Based on the results of demand forecasting, we propose new business opportunities.

[0301] 4. Emotion recognition

[0302] The emotion engine is used to analyze the user's emotion data sent from the terminal.

[0303] Personalize the information you provide based on emotional data.

[0304] 2. Terminal processing

[0305] The device (such as a smartphone) is responsible for the following processes:

[0306] 1. Providing a user interface

[0307] Provide an interface designed for user ease of use.

[0308] You can change the language on the settings screen.

[0309] 2. Obtaining location information

[0310] The device uses GPS to obtain the user's current location.

[0311] 3. Emotion recognition

[0312] Emotional data is obtained from the user's facial expressions and voice using the device's built-in camera and microphone.

[0313] Emotion data is sent to the server.

[0314] 4. Displaying Data

[0315] Display multilingual and personalized information received from the server.

[0316] Displays detailed information about tourist spots and specific transfer information.

[0317] 3. User-side processing

[0318] The user performs the following operations using the app.

[0319] 1. Installing the app and initial settings

[0320] The user downloads and installs the app.

[0321] After launching the app, set the language.

[0322] 2. Search and provide feedback

[0323] Users can search for their destination and check detailed information and transfer information.

[0324] After using the service, provide feedback and receive discount coupons.

[0325] 3. Providing Emotion Data

[0326] Users express their emotions when using the app.

[0327] The device recognizes the user's emotions and sends the data to the server.

[0328] Specific examples of program processing

[0329] Example: A visit to Sensoji Temple

[0330] 1. Information collection and multilingual translation

[0331] The server collects tourist spot information and translates it into multiple languages ​​using generative AI.

[0332] Information about Sensoji Temple will be stored in a database and made available in multiple languages, including English and Chinese.

[0333] 2. User Search

[0334] The user launches the app and sets the language to English.

[0335] Enter "Sensoji Temple" in the app's search bar and tap the search button.

[0336] 3. Displaying Information

[0337] The device acquires the user's current location information and emotion data and sends them to the server.

[0338] The server sends multilingual information about Sensoji Temple and personalized information based on the user's emotional data to the device, which then displays the information in English.

[0339] 4. Route guidance

[0340] When the user requests route guidance, the device again obtains the current location and sends it to the server.

[0341] The server calculates the optimal transfer route and generates detailed transfer information.

[0342] The route guidance information received by the terminal is displayed on the screen, providing the user with details such as the means of transport to be used, the line to be taken, transfer stations, and travel time.

[0343] 5. Feedback and Discount Coupons

[0344] After visiting a tourist attraction, users provide feedback within the app.

[0345] The server receives the feedback and updates the database.

[0346] The server generates a discount coupon that can be used next time for the user who provided the feedback and sends that information to the terminal, which then provides the coupon code to the user.

[0347] 6. Utilizing Emotional Data

[0348] Users express their emotions through their smartphone's camera and microphone while using the app.

[0349] The terminal sends the emotion data to the emotion engine and transmits it to the server.

[0350] The server analyzes the emotional data and prioritizes suggestions for specific tourist spots and services based on the user's current emotional state.

[0351] In this way, the present invention is a system that provides users with appropriate tourist information and transportation guidance in multiple languages, and further personalizes the information by utilizing emotion recognition.

[0352] The processing flow will be explained below.

[0353] Step 1:

[0354] The user downloads and installs the app.

[0355] Step 2:

[0356] The user launches the app and selects the language they want to use (e.g., English) on the language settings screen.

[0357] Step 3:

[0358] The device will save the user's language preference and change all displays to the selected language.

[0359] Step 4:

[0360] The user enters the location they want to go to (for example, "Sensoji Temple") in the app's search bar and taps the search button.

[0361] Step 5:

[0362] The device obtains the user's current location information using its GPS function and sends that information and the search query ("Sensoji Temple") to the server.

[0363] Step 6:

[0364] Based on the current location information and search query received by the server, the server searches the database for relevant tourist spot information.

[0365] Step 7:

[0366] The tourist spot information acquired by the server is translated into the language set by the user (English) using generative AI.

[0367] Step 8:

[0368] The server transmits the translated tourist spot information to the terminal.

[0369] Step 9:

[0370] The information received by the device is displayed on the app screen, providing the user with detailed information about the tourist spot (location, opening hours, photos, etc.).

[0371] Step 10:

[0372] While the user is using the app, the device's camera and microphone capture emotional data through the user's facial expressions and voice.

[0373] Step 11:

[0374] The emotion data acquired by the device is analyzed by an emotion engine to identify the user's emotional state.

[0375] Step 12:

[0376] The device transmits the user's current location and emotion data to the server.

[0377] Step 13:

[0378] The server re-searches tourist spots, restaurants, accommodations, etc. suitable for the user from the database based on the user's emotional state.

[0379] Step 14:

[0380] The server generates personalized information and translates it into the user's preferred language.

[0381] Step 15:

[0382] The server transmits the translated personalized information to the terminal.

[0383] Step 16:

[0384] The personalized information received by the device is displayed on the app screen.

[0385] Step 17:

[0386] The user taps the "Directions" button to request directions to their destination.

[0387] Step 18:

[0388] The device again acquires the user's current location and sends it to the server.

[0389] Step 19:

[0390] The server calculates the optimal route from the current location to the destination and generates detailed transfer information.

[0391] Step 20:

[0392] The server transmits the calculated route guidance information to the terminal.

[0393] Step 21:

[0394] The terminal displays the route guidance information it receives on the screen, providing the user with details such as the means of transport to be used, the line to be taken, transfer stations, and travel time.

[0395] Step 22:

[0396] After a user visits a tourist attraction, they tap the "Feedback" button in the app to open a feedback form.

[0397] Step 23:

[0398] The user enters their opinion and rating in the feedback form and taps the submit button.

[0399] Step 24:

[0400] The terminal transmits the user's feedback information to the server.

[0401] Step 25:

[0402] The server stores the received feedback information in a database and performs statistical analysis to update the database.

[0403] Step 26:

[0404] The server generates a discount coupon that can be used next time for the user who provided the feedback, and transmits the information to the terminal.

[0405] Step 27:

[0406] The terminal displays discount coupon information on the application screen and provides the user with a coupon code.

[0407] Example 2

[0408] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0409] In today's world, it is extremely difficult to quickly provide appropriate tourist information, transit directions, and restaurant and accommodation information in multiple languages ​​to foreign visitors to Japan and people who do not regularly use public transportation. Furthermore, while this information needs to be tailored to the user's emotions and specific needs, there is a lack of technological means to achieve this. To solve this problem, flexible and personalized information provision that combines multilingual support and emotion recognition is required.

[0410] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0411] In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for acquiring user emotion data and transmitting it to the server, means for the server to analyze the emotion data using an emotion engine and personalize the information, means for collecting feedback from users and updating the database, means for translating languages ​​using a generative AI model, and means for translating tourist attraction information into multiple languages ​​using a generative AI model. This makes it possible to provide users with accurate and personalized tourist information and transportation guidance in multiple languages, regardless of language barriers.

[0412] "Means for obtaining user location information and transmitting it to a server" refers to a series of processes by which a device obtains the latitude and longitude information of the current location using GPS or other location identification technology and transmits that information to a server.

[0413] "Means of searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user, and providing information in multiple languages" refers to a function that enables the server to search for relevant information from a database in accordance with the language setting selected by the user, and provide that information to the user in multiple languages.

[0414] "Means for providing detailed transfer information for public transport" is a system function that allows the server to calculate the optimal public transport route and detailed transfer information based on the user's current location and destination, and provide the results to the user.

[0415] "Means for acquiring user emotion data and transmitting it to the server" refers to the process by which the device uses a camera or microphone to recognize emotions from the user's facial expressions and voice, and transmits that data to the server.

[0416] "Means for the server to analyze emotional data using an emotion engine and personalize information" refers to a function in which the server analyzes the received emotional data using an emotion engine (e.g., emotion recognition software) and, based on the results, customizes the information provided to the user in an optimal manner.

[0417] "Means for collecting feedback from users and updating the database" refers to the process by which the server collects feedback information provided by users and updates the contents of the database based on that information.

[0418] A "language translation method using a generative AI model" is a technique or process that uses generative AI (e.g., a natural language processing model) to translate a given text into multiple languages.

[0419] "Means for translating tourist destination information into multiple languages ​​using a generative AI model" is a function that uses a generative AI model to collect information about tourist destinations, translate that information into multiple languages, and provide it.

[0420] This invention is a system that provides detailed transfer information, tourist attractions, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more appropriate information based on the user's location information and emotional data, and achieves multilingual support using generative AI. A specific embodiment of this system is described below.

[0421] The server is responsible for many information processing tasks. Specifically, it collects information about tourist attractions, restaurants, and accommodations from external data sources and stores it in a database. For example, it uses the Google Places API to obtain information about Sensoji Temple and stores it in a database. The server also uses a generative AI model (e.g., OpenAI GPT-3) to translate the collected information into multiple languages. For example, it uses prompts such as "Translate the following Japanese text into English: 'Sensoji Temple is one of the most famous temples in Tokyo.'" It also provides appropriate translation data based on the language selected by the user.

[0422] The device is responsible for providing the user interface, obtaining location information, recognizing emotions, and displaying data. Specifically, the device is built using a platform such as React Native and provides a settings screen that allows the user to change language settings. The device uses its GPS function to obtain the user's current location (for example, the latitude and longitude around Sensoji Temple) and sends this to the server. It also uses the device's built-in camera and microphone to obtain emotional data from the user's facial expressions and voice, and sends this to the server. The server uses an emotion engine (for example, Microsoft Azure's Emotion API) to analyze the emotional data and personalize the information it provides based on the results.

[0423] Users download the app and set the language after installation. When launched for the first time, a language setting screen appears, allowing users to select their preferred language (e.g., English). Users can enter their destination (e.g., "Asakusa Temple") in the app's search bar to check detailed information and transit directions. After visiting, they can enter and submit feedback within the app and receive discount coupons. Additionally, when users express their emotions (e.g., smiling) while using the app, the device captures that data and sends it to the server.

[0424] Using a generative AI model makes it possible to smoothly support multiple languages. For example, by sending a prompt such as "Translate the following Japanese text into English: 'Sensoji Temple is one of the most famous temples in Tokyo,'" to the AI ​​model, it is possible to obtain information translated into multiple languages, including English and Chinese.

[0425] This makes it possible to provide users with accurate and personalized tourist information and transportation guidance in multiple languages, overcoming language barriers. This system is extremely useful for tourists and people who do not regularly use public transportation, and offers a significant improvement in convenience compared to conventional information provision systems.

[0426] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0427] Step 1:

[0428] The device provides the user interface

[0429] The device builds an application using a platform such as React Native and provides the user with a UI that includes a settings screen for language settings, a search bar, a feedback form, etc. The input is the user's operation (e.g., language selection, search term input), and the output is the dynamic display of the interface in response to the user's operation.

[0430] Step 2:

[0431] The device acquires location information and sends it to the server.

[0432] The device uses its GPS function to obtain the user's current location (e.g., latitude and longitude information). The specific operation is to use navigator.geolocation.getCurrentPosition. The input is the location information obtained from the device's GPS function, and the output is the location information data sent to the server.

[0433] Step 3:

[0434] The device acquires emotion data and sends it to the server.

[0435] The device uses a camera and microphone to acquire the user's emotional data. Specifically, it analyzes the user's facial expressions captured by the camera and the tone of voice picked up by the microphone. The input is the user's facial expressions and voice, and the output is emotional data analyzed by an emotion recognition engine (e.g., Microsoft Azure's Emotion API). This data is also sent to the server.

[0436] Step 4:

[0437] The server collects tourist spot information and stores it in a database

[0438] The server calls an external API (e.g., Google Places API) to obtain tourist spot information and saves it in a database. Specifically, it periodically sends API requests to obtain the latest information. The input is tourist spot information obtained from the API, and the output is the latest information saved in the database.

[0439] Example: GET https: / / maps.googleapis.com / maps / api / place / details / json?placeid=xxxxxx&key=API_KEY

[0440] Step 5:

[0441] The information collected by the server is translated into multiple languages ​​using a generative AI model.

[0442] The server uses a generative AI model (e.g., OpenAI GPT-3) to translate the collected information into various languages. For example, a prompt such as "Translate the following Japanese text into English: Sensoji Temple is one of the most famous temples in Tokyo" is input to the generative AI. The input is the collected tourist spot information in Japanese, and the output is the translated information in each language.

[0443] Step 6:

[0444] The server provides information based on the user's language preference.

[0445] The server searches for translation data based on the language information set by the user and sends it to the device. The input is information in the set language, and the output is tourist spot information translated into the selected language.

[0446] Step 7:

[0447] The server analyzes the emotional data and personalizes the information.

[0448] The server analyzes the emotional data received by the emotion engine and personalizes the information provided based on the user's emotional state. The input is emotional data, and the output is personalized recommendations for tourist spots. For example, if the user is having fun, it will suggest spots with a lot of activity.

[0449] Step 8:

[0450] The server provides detailed public transport route information.

[0451] The server calculates the optimal route based on the user's current location and destination, and generates detailed transfer information. The input is the user's current location and destination information, and the output is detailed information such as the transportation to be used, the line to take, transfer stations, and travel time.

[0452] Example: route = calculate_optimal_route(current_location, destination)

[0453] Step 9:

[0454] Display the data the device receives from the server

[0455] The device displays the multilingual and personalized information received from the server on the screen. Specifically, it updates the search results screen and route guidance screen appropriately. The input is the translated and personalized information sent from the server, and the output is the specific information displayed on the screen.

[0456] Example: Display information about Sensoji Temple as a tourist spot in English.

[0457] Step 10:

[0458] Users provide feedback and receive discount coupons

[0459] After a user visits the app, they input and submit feedback. The server receives this feedback, updates the database, generates a discount coupon for the next visit, and sends it to the device. The input is the user's feedback information, and the output is updating the database and issuing a discount coupon.

[0460] Example: Enter "I enjoyed my visit to Sensoji Temple" in the feedback form and submit.

[0461] Through the above processing steps, the system provides users with personalized tourist information and transportation guidance in multiple languages, enabling a more meaningful tourist experience.

[0462] (Application example 2)

[0463] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0464] When foreign visitors to Japan and tourists from other regions visit shopping centers and commercial facilities, they often face problems such as language barriers, unclear destinations, and a lack of personalized information. In addition, the lack of location information and store information within the shopping center is also a problem. Additionally, there is a lack of personalized services based on emotional data, which can lead to lower user satisfaction.

[0465] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for recognizing the user's emotions and transmitting emotion data to the server, means for searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for collecting user feedback and updating the database, means for translating languages ​​using generative AI, means for personalizing store information and sales information based on the user's emotion data, and means for providing information on the user's current location and nearby stores within a shopping center. This enables smooth provision of information within the shopping center or commercial facility visited and personalized guidance services based on the user's emotions.

[0466] "User" refers to an individual who uses this system.

[0467] "Location information" is information that indicates the user's current location and is obtained using technology such as GPS.

[0468] A "server" is a central computing device that collects, processes, and provides information to users.

[0469] "Emotion data" is information about the user's emotional state that can be inferred from facial expressions, voice, etc.

[0470] "Means of providing information in multiple languages" refers to the process of using generative AI to translate information into various languages ​​and provide it to users.

[0471] "Transfer guide" is a guide that includes detailed route information for reaching a destination using public transportation.

[0472] "Feedback" refers to opinions and evaluations regarding systems and services provided by users.

[0473] A "database" is a repository of information on tourist spots, restaurants, accommodations, and more.

[0474] "Generative AI" is a type of artificial intelligence, a technology that generates natural language and other information from given data.

[0475] "Means of personalization" refers to methods of optimizing and providing information based on each user's emotional data and behavioral history.

[0476] A "shopping center" is a commercial facility with multiple stores, where customers can shop and enjoy services.

[0477] This invention is a system that enables foreign visitors to Japan and tourists from other regions to receive appropriate information and personalized guidance when they visit shopping centers and commercial facilities. This system consists of a server and a terminal, and performs the following processes.

[0478] First, the device uses its GPS function to obtain the user's location information and sends it to the server. Based on this location information, the server obtains information about the shopping center the user is visiting and nearby stores. The device also uses its built-in camera and microphone to obtain emotional data from the user's facial expressions and voice, which it then sends to the server. The emotional data is analyzed using an emotion engine to personalize the information provided.

[0479] Based on the language selected by the user, the server uses generative AI to perform multilingual translation. It searches a database of tourist attractions, restaurants, and accommodations and provides information in multiple languages. The use of generative AI enables appropriate translation in real time. The server also provides detailed public transport transfer information to help users travel smoothly to their destination.

[0480] The server has a feedback function that collects feedback from users and updates the database. It also has a function to issue discount coupons in response to feedback, making the next shopping experience even more fulfilling. It also has a means to personalize store information and sales information based on emotional data. It provides information on the user's current location within the shopping center and nearby stores, and provides appropriate guidance.

[0481] The hardware used is a smartphone, and the software includes expo-location (location information acquisition), expo-camera (camera function), and axios (API communication function). An example of an emotion detection API is Microsoft Azure Face API, and an example of a translation API is Google Translate API.

[0482] As a specific example, to suggest relaxation spaces and cafes based on emotion data, a camera can detect the user's "I want to relax" emotion while shopping, and the server receives the data. The server then provides information on relaxation cafes and spaces in multiple languages ​​and guides users to the nearest location based on their location information. Furthermore, if a shopping center is attracting tourists from multiple countries, it can provide information on events at the center in multiple languages.

[0483] Examples of prompts using generative AI models include:

[0484] "Analyze my current emotions and suggest the best relaxation spot based on that."

[0485] "Please provide store information in English."

[0486] This will enable tourists and users to receive appropriate information and guidance, resulting in a more satisfying shopping experience.

[0487] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0488] Step 1:

[0489] The device uses its GPS to obtain the user's current location. This location information is obtained by the device's location service and is expressed as latitude and longitude coordinates. This location information is then sent to the server.

[0490] Step 2:

[0491] The server searches its database for store information in and around the shopping center based on the received location information. The search results provide a list of stores and services related to the user's current location. This list is obtained by matching the location information with store information that matches.

[0492] Step 3:

[0493] The device uses the smartphone's camera and microphone to acquire emotional data from the user's facial expressions and voice. The emotional data is obtained by analyzing the images and audio data acquired from the camera and microphone. This data is then sent to a server.

[0494] Step 4:

[0495] The server uses an emotion engine to analyze the received emotion data. The analysis results in the user's current emotional state (e.g., wanting to relax, wanting to have fun, etc.). This analysis is performed using a combination of image recognition and voice analysis technologies.

[0496] Step 5:

[0497] The server uses generative AI to perform multilingual translation based on the language selected by the user. Specifically, it translates store information retrieved from the database into the language selected by the user in real time. The translation results in store information available in multiple languages.

[0498] Step 6:

[0499] The server then personalizes store and sale information based on the user's emotional data and location information. Specifically, it prioritizes suggestions for relaxing cafes and specific stores based on the analyzed emotional data. These suggestions are made based on the user's current location.

[0500] Step 7:

[0501] The server sends the multilingually translated personalized information to the device, which then receives it and displays it to the user. The displayed content includes information about stores near the user's current location, sales information, and recommended stores based on user sentiment.

[0502] Step 8:

[0503] Users can then act on the information provided and enjoy their shopping experience. During this process, they can also use their devices to check detailed public transport route information. The route information is based on real-time traffic information provided by the server.

[0504] Step 9:

[0505] Users provide feedback within the app. The device sends this feedback to the server, which then stores and updates the received feedback in a database. The feedback includes store ratings and impressions.

[0506] Step 10:

[0507] The server generates a discount coupon that can be used for the user's next purchase based on the provided feedback and sends it to the terminal, which then displays the coupon to the user, enhancing their next shopping experience.

[0508] This allows the system as a whole to provide smooth information provision and personalized guidance to users, thereby improving user satisfaction.

[0509] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0510] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0511] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0512] [Second embodiment]

[0513] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0514] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0515] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0516] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0517] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0518] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0519] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0520] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0521] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0522] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0523] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0524] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0525] This invention is a system that provides detailed transfer information, tourist attractions, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation. The system provides optimal information based on the user's location information and achieves multilingual support using generative AI.

[0526] System configuration

[0527] 1. Server-side processing

[0528] The server is responsible for the following processes:

[0529] 1. Building and updating the database

[0530] The server collects information on tourist spots, restaurants, and accommodations and stores it in a database.

[0531] Periodically retrieve the latest information from multiple external data sources and update the database.

[0532] 2. Multilingual support

[0533] Generative AI is used to translate collected information into multiple languages.

[0534] Select the appropriate translation data based on the language set by the user.

[0535] 3. Demand forecasting

[0536] The collected data will be analyzed and tourism demand will be predicted.

[0537] Based on the results of demand forecasting, we propose new business opportunities.

[0538] 2. Terminal processing

[0539] The device (such as a smartphone) is responsible for the following processes:

[0540] 1. Providing a user interface

[0541] Provide an interface designed for user ease of use.

[0542] You can change the language in the settings screen and search for destinations and points of interest.

[0543] 2. Obtaining location information

[0544] The device uses GPS to obtain the user's current location.

[0545] 3. Displaying Data

[0546] Display multilingual information received from the server.

[0547] Displays detailed information about tourist spots and specific transfer information.

[0548] 3. User-side processing

[0549] The user performs the following operations using the app.

[0550] 1. Installing the app and initial settings

[0551] The user downloads and installs the app.

[0552] After launching the app, set the language.

[0553] 2. Search and provide feedback

[0554] Users search for their destination and check detailed information and transfer information.

[0555] After using the service, provide feedback and receive discount coupons.

[0556] Specific examples of program processing

[0557] Example: A visit to Sensoji Temple

[0558] 1. Information collection and multilingual translation

[0559] The server collects tourist spot information and translates it into multiple languages ​​using generative AI.

[0560] Information about Sensoji Temple will be stored in a database and made available in multiple languages, including English and Chinese.

[0561] 2. User Search

[0562] The user launches the app and sets the language to English.

[0563] Enter "Sensoji Temple" in the app's search bar and tap the search button.

[0564] 3. Displaying Information

[0565] The device obtains the user's current location and sends it to the server.

[0566] The server sends multilingual information about Sensoji Temple to the terminal, which then displays the information in English.

[0567] 4. Route guidance

[0568] When the user requests route guidance, the device again obtains the current location and sends it to the server.

[0569] The server calculates the optimal transfer route and sends it to the terminal.

[0570] The device displays detailed transfer information, allowing the user to check train and bus transfer information.

[0571] 5. Feedback and Discount Coupons

[0572] After visiting a tourist attraction, users provide feedback within the app.

[0573] The server receives the feedback and updates the database.

[0574] The server issues a discount coupon and sends it to the terminal, which then displays the coupon to the user.

[0575] In this way, the present invention is a system that provides users with appropriate tourist information and transportation guidance in multiple languages ​​and utilizes feedback to improve services.

[0576] The processing flow will be explained below.

[0577] Step 1:

[0578] The user downloads and installs the app.

[0579] Step 2:

[0580] The user launches the app and selects the language they want to use (e.g., English) on the language settings screen.

[0581] Step 3:

[0582] The device will save the user's language preference and change all displays to the selected language.

[0583] Step 4:

[0584] The user enters the location they want to go to (for example, "Sensoji Temple") in the app's search bar and taps the search button.

[0585] Step 5:

[0586] The device obtains the user's current location information using its GPS function and sends that information and the search query ("Sensoji Temple") to the server.

[0587] Step 6:

[0588] Based on the current location information and search query received by the server, the server searches the database for relevant tourist spot information.

[0589] Step 7:

[0590] The tourist spot information acquired by the server is translated into the language set by the user (English) using generative AI.

[0591] Step 8:

[0592] The server transmits the translated tourist spot information to the terminal.

[0593] Step 9:

[0594] The information received by the device is displayed on the app screen, providing the user with detailed information about the tourist spot (location, opening hours, photos, etc.).

[0595] Step 10:

[0596] The user taps the "Directions" button to request directions to their destination.

[0597] Step 11:

[0598] The device again acquires the user's current location and sends it to the server.

[0599] Step 12:

[0600] The server calculates the optimal route from the current location to the destination and generates detailed transfer information.

[0601] Step 13:

[0602] The server transmits the calculated route guidance information to the terminal.

[0603] Step 14:

[0604] The terminal displays the route guidance information it receives on the screen, providing the user with details such as the means of transport to be used, the line to be taken, transfer stations, and travel time.

[0605] Step 15:

[0606] After a user visits a tourist attraction, they tap the "Feedback" button in the app to open a feedback form.

[0607] Step 16:

[0608] The user enters their opinion and rating in the feedback form and taps the submit button.

[0609] Step 17:

[0610] The terminal transmits the user's feedback information to the server.

[0611] Step 18:

[0612] The server stores the received feedback information in a database and performs statistical analysis to update the database.

[0613] Step 19:

[0614] The server generates a discount coupon that can be used next time for the user who provided the feedback, and transmits the information to the terminal.

[0615] Step 20:

[0616] The terminal displays discount coupon information on the application screen and provides the user with a coupon code.

[0617] Example 1

[0618] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0619] For foreign visitors to Japan and those who do not regularly use public transportation, it is difficult to provide information on tourist attractions, restaurants, accommodations, and detailed transfer directions in the appropriate language, and to improve the quality of the service by incorporating user feedback. It is also not easy to predict tourism demand or propose new business opportunities. To solve these challenges, a system that uses generative AI models to achieve multilingual support and provide users with optimal information and services in real time is needed.

[0620] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0621] In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for searching a database of tourist facilities, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for collecting feedback from users and updating the database, means for translating languages ​​using a generative AI model, means for forecasting demand and proposing new business opportunities, and means for providing real-time information based on the user's current location. This makes it possible to provide detailed tourist information and transfer information in multiple languages ​​to foreign visitors to Japan and people who do not regularly use public transportation, and to improve the quality of the service based on feedback.

[0622] "User location information" means data indicating the current location of a person holding a device, and is information obtained using GPS or other location identification technology.

[0623] "Tourist facilities" refer to tourist spots and tourist destinations that users would like to visit, and include historical buildings, parks, museums, etc.

[0624] "Food and beverage establishment" means a place where customers can enjoy food and beverages, and includes restaurants, cafes, bars, etc.

[0625] "Accommodation" refers to a place where users can stay overnight, and includes hotels, private lodgings, guest houses, etc.

[0626] "Means of providing information in multiple languages" refers to a system that provides information on tourist facilities, restaurants, and accommodations in the language selected by the user, and refers to the use of machine translation and multilingual databases.

[0627] "Means for providing detailed transfer information for public transport" refers to a system that provides detailed information on the optimal transfer routes and methods for users to reach their destination, including train and bus operation information.

[0628] "Means for collecting feedback and updating the database" refers to a system for collecting user ratings and comments, updating the database information based on that, and improving the quality of the service.

[0629] "Means for translating languages ​​using generative AI models" refers to a system that uses generative artificial intelligence models to translate text into different languages, and utilizes natural language processing technology.

[0630] "Means of forecasting demand and proposing new business opportunities" refers to a system that analyzes trends in tourism demand based on collected data and finds new business opportunities.

[0631] "Means for providing real-time information" refers to a mechanism that instantly provides the latest information based on the user's current location, including dynamic searches and information updates.

[0632] This invention is a system that provides detailed transfer information, tourist facilities, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation. This system provides optimal information based on the user's location information and achieves multilingual support using a generative AI model. A specific method for implementing this system is described below.

[0633] Server-side processing

[0634] The server uses the following hardware and software to collect, translate, and analyze various information.

[0635] 1. Building and updating the database

[0636] The server collects information on tourist facilities, restaurants, and accommodations from external data sources and stores it in a database. It periodically retrieves and updates the latest information using a web crawler or API. Specifically, MySQL or PostgreSQL is used as the database.

[0637] 2. Multilingual support

[0638] A generative AI model is used to translate the collected information into multiple languages. Specifically, OpenAI's GPT model and Google Translate API are used, and the translation results are stored in a database. An example prompt is as follows:

[0639] Prompt: "Translate the following information about the tourist spot 'Sensoji Temple' into English, Chinese, and Korean: [More information about the tourist spot]"

[0640] 3. Demand forecasting

[0641] The server uses data science tools (e.g., TensorFlow and Scikit-learn) to analyze the collected data and predict tourism demand, which will then suggest new business opportunities.

[0642] Terminal side processing

[0643] The device (smartphone, etc.) operates with the following configuration.

[0644] 1. Providing a user interface

[0645] The device uses React Native and Flutter to provide a user-friendly interface, including a search bar, settings menu, history display, and more.

[0646] 2. Obtaining location information

[0647] The device uses the smartphone's GPS to obtain the user's current location, using Google Maps API or Apple's Core Location.

[0648] 3. Displaying Data

[0649] The terminal displays the multilingual information received from the server, including detailed information on tourist attractions and transfer information, so that the user can check it.

[0650] User operation

[0651] The user performs the following operations using the app.

[0652] 1. Installing the app and initial settings

[0653] Users download and install the app from the App Store or Google Play Store, and set the language when they first launch it.

[0654] 2. Search and provide feedback

[0655] Users search for destinations within the app and check the displayed information. After visiting a tourist spot, they can use the app's feedback function to enter and submit their ratings and comments. This allows the server to update the database and issue discount coupons.

[0656] Example: A visit to Sensoji Temple

[0657] For example, when a user visits Sensoji Temple, information and guidance are provided in the following manner.

[0658] 1. The server collects information about Sensoji Temple from external data sources and translates it into multiple languages ​​using a generative AI model.

[0659] 2. The user launches the app and changes the language setting to English.

[0660] 3. The user enters "Sensoji Temple" and searches.

[0661] 4. The device obtains its current location using GPS and sends it to the server.

[0662] 5. The server sends the corresponding multilingual information to the terminal, which then displays the information in English.

[0663] 6. The user requests route guidance, and the device again obtains the current location and sends it to the server.

[0664] 7. The server calculates the optimal transfer route and sends it to the device, which then displays detailed transfer information.

[0665] 8. After visiting a tourist attraction, users can provide feedback and receive discount coupons.

[0666] In this way, the present invention is a system that provides users with appropriate tourist information and transportation guidance in multiple languages ​​and utilizes feedback to improve services.

[0667] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0668] System program processing flow:

[0669] Step 1: Collecting data and storing it in a database

[0670] The server collects information about tourist facilities, restaurants, and accommodations from external data sources (such as APIs and web crawlers). The server organizes this data and stores it in a database such as MySQL or PostgreSQL. The input includes information from external data sources, and the output is stored in the database.

[0671] Step 2: Multilingual Translation

[0672] The server translates the information stored in the database into multiple languages ​​using a generative AI model (e.g., GPT-4). The input includes the original sentence retrieved from the database, and the output is the translated data into multiple languages. The following is used as an example of a prompt sentence:

[0673] Prompt: "Translate the following information about the tourist spot 'Sensoji Temple' into English, Chinese, and Korean: [More information about the tourist spot]"

[0674] The server stores the translated data back in the database.

[0675] Step 3: Obtaining user location information

[0676] The device obtains the user's current location using the smartphone's GPS function. The input includes the device's GPS data, and the output is the latitude and longitude of the user's current location. Specifically, Google Maps API or Apple's Core Location is used.

[0677] Step 4: Providing a User Interface

[0678] The device uses React Native and Flutter to provide an interface that makes it easy for users to search and configure settings. Input includes user actions (search keywords and setting changes), and the appropriate screen is displayed as output.

[0679] Step 5: Providing multilingual information

[0680] The device sends a request to the server based on the user's current location and the language selected. The server receives the request, retrieves the corresponding language translation data from the database, and sends it to the device. The input includes the user's location information and selected language, and the output is multilingually translated tourist information.

[0681] Step 6: Provide detailed transit directions

[0682] The server calculates the optimal transit route based on the user's current location and destination information. The input includes the user's current location and destination, and the output is a detailed transit route. The optimal route is obtained using Google Maps API and other public transport APIs.

[0683] Step 7: Gather feedback and update the database

[0684] The terminal sends the user-provided feedback to the server, which receives the feedback and updates the database. The input contains the user's feedback data, and the output is an updated database.

[0685] Step 8: Issue a discount coupon

[0686] The server generates a discount coupon after the user provides the feedback and sends it to the terminal. The input includes the feedback status, and the generated coupon is obtained as the output. The terminal displays the coupon information to the user.

[0687] Specific flow of processing steps

[0688] Step 1: Collecting data and storing it in a database

[0689] The server retrieves information about tourist facilities, restaurants, and accommodations from tourism websites and related databases via API. The retrieved data is then properly organized and stored in a MySQL or PostgreSQL database.

[0690] Step 2: Multilingual Translation

[0691] The server takes each piece of data in the database and translates it into each language using a generative AI model. The translated data is then stored back in the database and made available in each language.

[0692] Step 3: Obtaining user location information

[0693] The device uses GPS to obtain the latitude and longitude of the current location, which is then sent to the server, where it is used to provide information appropriate to the current location.

[0694] Step 4: Providing a User Interface

[0695] The device uses React Native and Flutter to display an intuitive user interface, including a home screen with a search bar, settings menu, and history view, where users can search for destinations and change language settings.

[0696] Step 5: Providing multilingual information

[0697] The device requests the user's current location and preferred language setting from the server, which retrieves multilingual tourist information from a database and sends it to the device in JSON format, where it is displayed in the appropriate format.

[0698] Step 6: Provide detailed transit directions

[0699] When the user requests transit directions, the device again retrieves its current location and sends it to the server. The server uses the appropriate transport API to calculate detailed transit routes and sends them to the device. The device then displays specific transit directions, helping the user reach their destination easily.

[0700] Step 7: Gather feedback and update the database

[0701] When a user provides feedback on the tourist attractions they visited or the services they used, the device sends the feedback data to the server, which then incorporates the feedback into a database and updates the information, thereby providing other users with the latest information.

[0702] Step 8: Issue a discount coupon

[0703] After the user provides feedback, the server generates a discount coupon and sends it to the terminal, which displays the coupon to the user so that it can be applied the next time they use the service.

[0704] (Application example 1)

[0705] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0706] Although systems exist that allow tourists to access detailed tourist information in multiple languages ​​and receive optimal tourist guidance based on their location in real time, there is a lack of methods to link information with real-world scenery and multilingual audio guides. There is also a need for tourist route suggestions tailored to individual needs and for service improvements based on feedback.

[0707] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0708] In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for collecting user feedback and updating the database, means for translating languages ​​using generative AI, means for translating information about tourist attractions and restaurants into multiple languages ​​using a generative AI model, means for guiding tourist routes and places of interest based on the user's current location, means for displaying tourist attraction information overlaid on real-world scenery using augmented reality (AR) technology, and means for providing multilingual audio guides. This allows tourists to obtain detailed, multilingual tourist information in real time, improving their local tourist experience.

[0709] "User Location Information" means your current geographic location obtained using GPS or Wi-Fi.

[0710] A "server" refers to a computer system that provides databases and applications via a network.

[0711] "Tourist attractions" refer to places of interest or facilities that tourists visit in a particular area.

[0712] "Restaurant" refers to an establishment that serves food and drinks.

[0713] "Accommodation facilities" refer to hotels, inns, and other establishments where travelers can stay temporarily.

[0714] A "database" refers to a system for organizing and storing information.

[0715] "Means for providing information in multiple languages" refers to the function of providing users with necessary information in multiple languages.

[0716] "Public transportation" refers to transportation systems such as buses and trains used by the general public.

[0717] "Transfer information" refers to providing detailed route information when users transfer between different public transportation modes.

[0718] "Feedback" refers to the evaluations and opinions provided by users after using a system or service.

[0719] "Generative AI" refers to technology that uses artificial intelligence to generate new language data and translations.

[0720] "Multilingual audio guide" refers to a system that provides audio guidance in multiple languages.

[0721] "Augmented reality (AR) technology" refers to the technology of overlaying digital information onto real-world scenery.

[0722] This invention relates to a tourism information provision system that allows tourists to receive detailed tourist information in multiple languages. This system utilizes user location information, a database of tourist spots, restaurants, and accommodations, multilingual translation, public transportation transfer information, feedback collection, generative AI, multilingual audio guides, and augmented reality (AR).

[0723] Hardware and software used

[0724] 1. Hardware

[0725] Smartphone (GPS, camera)

[0726] Smart glasses or head-mounted display (AR display, GPS)

[0727] 2. Software

[0728] Mobile app (compatible with iOS / Android)

[0729] Generative AI models (e.g., GPT-4)

[0730] Location services (Google Maps API)

[0731] Text-to-speech service (e.g., Amazon Polly)

[0732] AR Kit (ARCore, ARKit)

[0733] System Overview

[0734] Server Features

[0735] The server builds a database of information on tourist attractions, restaurants, and accommodations, and updates it regularly. The collected information is translated into multiple languages ​​using a generative AI model and provided based on the language selected by the user. The server also collects user feedback and updates the database.

[0736] Generative AI models are used to translate information into multiple languages, such as displaying detailed information about tourist attractions from Japanese to English.

[0737] Device Features

[0738] The device uses GPS to acquire the user's current location and sends it to the server. It also provides a user interface and displays information about tourist spots searched for by the user in multiple languages. It also provides public transport transfer information to support the user's travels.

[0739] When using the augmented reality (AR) function, information is superimposed on the real world via smart glasses or a head-mounted display. For example, when a user arrives at a tourist spot, the area's history and attractions can be displayed in AR. A multilingual audio guide function is also provided, providing detailed audio guidance about tourist spots.

[0740] User operations

[0741] Users download the mobile app and set their language preferences. When they launch the app, the app acquires their location information and provides optimal tourist information based on their current location. Users can search for tourist attractions and check detailed information and public transportation transfer information. When they arrive at a tourist attraction, they can effectively obtain tourist information using AR displays and multilingual audio guides.

[0742] Examples of concrete examples and prompts

[0743] Specific examples

[0744] When you visit an attraction:

[0745] Launch the app and get your location using GPS.

[0746] Multilingual tourist attraction information is obtained from the server.

[0747] Smart glasses display AR information about tourist attractions in real-world scenes.

[0748] Audio guides explain the history of the attractions.

[0749] Example prompts

[0750] "Translate the Japanese information about tourist attractions into English and list five must-see spots."

[0751] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0752] Step 1:

[0753] A user launches an app on their smartphone and sets the language. The input is the user's selected language setting information, and the output is the initial settings screen of the app. The app then sends the language setting information to the server.

[0754] Step 2:

[0755] The device uses GPS to obtain the user's current location. The input is GPS data, and the output is the current location information. This location information is converted into a digital format and sent to the server.

[0756] Step 3:

[0757] The server searches the database for information on tourist attractions, restaurants, and accommodations based on the received location information. The input is location information and language setting information, and the output is tourist information related to the user's current location. This information is translated into multiple languages ​​using a generative AI model.

[0758] Step 4:

[0759] The server sends multilingual tourist information translated by a generative AI model to the terminal. The input is the translated tourist information, and the output is multilingual tourist information. The terminal displays this information in a user interface.

[0760] Step 5:

[0761] Users wear smart glasses or a head-mounted display and view tourist spot information in real time. The input is real-world scenery and tourist information, and the output is information overlaid using AR technology. The device uses a camera to capture the real-world scenery and overlays tourist information on it.

[0762] Step 6:

[0763] When a user arrives at a tourist spot, the device uses a speech synthesis service to provide a multilingual audio guide. The input is detailed information about the tourist spot and the user's preferred language, and the output is a speech synthesis guide voice. The speech synthesis service is used to generate and play the audio guide on the spot.

[0764] Step 7:

[0765] After viewing or using the tourist attraction information, the user provides feedback through the app. The input is the feedback information from the user, and the output is the feedback data. This feedback data is sent to the server and updates the database.

[0766] Step 8:

[0767] The server updates the database based on the feedback and issues discount coupons as necessary. The input is the feedback data and the user's usage history, and the output is the discount coupon information. The coupon is sent in digital format to the terminal and displayed to the user.

[0768] Step 9:

[0769] The server predicts inbound demand based on data on tourist spots, restaurants, and accommodations. The input is past usage data and tourist information, and the output is the demand forecast results. The server analyzes this data and predicts tourist demand.

[0770] Step 10:

[0771] The server creates and proposes new tourist spot and event information based on prompts generated using a generative AI model. The input is the prompt and current tourist information, and the output is new tourist suggestion information. An example prompt is, "Translate the Japanese information about tourist spots into English and list five highlights."

[0772] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0773] This invention is a system that provides detailed transfer information, tourist attractions, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation, and also combines it with an emotion engine that recognizes the user's emotions. It provides more appropriate information based on the user's location information and emotion data, and achieves multilingual support using generative AI.

[0774] System configuration

[0775] 1. Server-side processing

[0776] The server is responsible for the following processes:

[0777] 1. Building and updating the database

[0778] The server collects information on tourist spots, restaurants, and accommodations and stores it in a database.

[0779] Periodically retrieve the latest information from multiple external data sources and update the database.

[0780] 2. Multilingual support

[0781] Generative AI is used to translate collected information into multiple languages.

[0782] Select the appropriate translation data based on the language set by the user.

[0783] 3. Demand forecasting

[0784] The collected data will be analyzed and tourism demand will be predicted.

[0785] Based on the results of demand forecasting, we propose new business opportunities.

[0786] 4. Emotion recognition

[0787] The emotion engine is used to analyze the user's emotion data sent from the terminal.

[0788] Personalize the information you provide based on emotional data.

[0789] 2. Terminal processing

[0790] The device (such as a smartphone) is responsible for the following processes:

[0791] 1. Providing a user interface

[0792] Provide an interface designed for user ease of use.

[0793] You can change the language on the settings screen.

[0794] 2. Obtaining location information

[0795] The device uses GPS to obtain the user's current location.

[0796] 3. Emotion recognition

[0797] Emotional data is obtained from the user's facial expressions and voice using the device's built-in camera and microphone.

[0798] Emotion data is sent to the server.

[0799] 4. Displaying Data

[0800] Display multilingual and personalized information received from the server.

[0801] Displays detailed information about tourist spots and specific transfer information.

[0802] 3. User-side processing

[0803] The user performs the following operations using the app.

[0804] 1. Installing the app and initial settings

[0805] The user downloads and installs the app.

[0806] After launching the app, set the language.

[0807] 2. Search and provide feedback

[0808] Users search for their destination and check detailed information and transfer information.

[0809] After using the service, provide feedback and receive discount coupons.

[0810] 3. Providing Emotion Data

[0811] Users express their emotions when using the app.

[0812] The device recognizes the user's emotions and sends the data to the server.

[0813] Specific examples of program processing

[0814] Example: A visit to Sensoji Temple

[0815] 1. Information collection and multilingual translation

[0816] The server collects tourist spot information and translates it into multiple languages ​​using generative AI.

[0817] Information about Sensoji Temple will be stored in a database and made available in multiple languages, including English and Chinese.

[0818] 2. User Search

[0819] The user launches the app and sets the language to English.

[0820] Enter "Sensoji Temple" in the app's search bar and tap the search button.

[0821] 3. Displaying Information

[0822] The device acquires the user's current location information and emotion data and sends them to the server.

[0823] The server sends multilingual information about Sensoji Temple and personalized information based on the user's emotional data to the device, which then displays the information in English.

[0824] 4. Route guidance

[0825] When the user requests route guidance, the device again obtains the current location and sends it to the server.

[0826] The server calculates the optimal transfer route and generates detailed transfer information.

[0827] The route guidance information received by the terminal is displayed on the screen, providing the user with details such as the means of transport to be used, the line to be taken, transfer stations, and travel time.

[0828] 5. Feedback and Discount Coupons

[0829] After visiting a tourist attraction, users provide feedback within the app.

[0830] The server receives the feedback and updates the database.

[0831] The server generates a discount coupon that can be used next time for the user who provided the feedback and sends that information to the terminal, which then provides the coupon code to the user.

[0832] 6. Utilizing Emotional Data

[0833] Users express their emotions through their smartphone's camera and microphone while using the app.

[0834] The terminal sends the emotion data to the emotion engine and transmits it to the server.

[0835] The server analyzes the emotional data and prioritizes suggestions for specific tourist spots and services based on the user's current emotional state.

[0836] In this way, the present invention is a system that provides users with appropriate tourist information and transportation guidance in multiple languages, and further personalizes the information by utilizing emotion recognition.

[0837] The processing flow will be explained below.

[0838] Step 1:

[0839] The user downloads and installs the app.

[0840] Step 2:

[0841] The user launches the app and selects the language they want to use (e.g., English) on the language settings screen.

[0842] Step 3:

[0843] The device will save the user's language preference and change all displays to the selected language.

[0844] Step 4:

[0845] The user enters the location they want to go to (for example, "Sensoji Temple") in the app's search bar and taps the search button.

[0846] Step 5:

[0847] The device obtains the user's current location information using its GPS function and sends that information and the search query ("Sensoji Temple") to the server.

[0848] Step 6:

[0849] Based on the current location information and search query received by the server, the server searches the database for relevant tourist spot information.

[0850] Step 7:

[0851] The tourist spot information acquired by the server is translated into the language set by the user (English) using generative AI.

[0852] Step 8:

[0853] The server transmits the translated tourist spot information to the terminal.

[0854] Step 9:

[0855] The information received by the device is displayed on the app screen, providing the user with detailed information about the tourist spot (location, opening hours, photos, etc.).

[0856] Step 10:

[0857] While the user is using the app, the device's camera and microphone capture emotional data through the user's facial expressions and voice.

[0858] Step 11:

[0859] The emotion data acquired by the device is analyzed by an emotion engine to identify the user's emotional state.

[0860] Step 12:

[0861] The device transmits the user's current location and emotion data to the server.

[0862] Step 13:

[0863] The server re-searches tourist spots, restaurants, accommodations, etc. suitable for the user from the database based on the user's emotional state.

[0864] Step 14:

[0865] The server generates personalized information and translates it into the user's preferred language.

[0866] Step 15:

[0867] The server transmits the translated personalized information to the terminal.

[0868] Step 16:

[0869] The personalized information received by the device is displayed on the app screen.

[0870] Step 17:

[0871] The user taps the "Directions" button to request directions to their destination.

[0872] Step 18:

[0873] The device again acquires the user's current location and sends it to the server.

[0874] Step 19:

[0875] The server calculates the optimal route from the current location to the destination and generates detailed transfer information.

[0876] Step 20:

[0877] The server transmits the calculated route guidance information to the terminal.

[0878] Step 21:

[0879] The terminal displays the route guidance information it receives on the screen, providing the user with details such as the means of transport to be used, the line to be taken, transfer stations, and travel time.

[0880] Step 22:

[0881] After a user visits a tourist attraction, they tap the "Feedback" button in the app to open a feedback form.

[0882] Step 23:

[0883] The user enters their opinion and rating in the feedback form and taps the submit button.

[0884] Step 24:

[0885] The terminal transmits the user's feedback information to the server.

[0886] Step 25:

[0887] The server stores the received feedback information in a database and performs statistical analysis to update the database.

[0888] Step 26:

[0889] The server generates a discount coupon that can be used next time for the user who provided the feedback, and transmits the information to the terminal.

[0890] Step 27:

[0891] The terminal displays discount coupon information on the application screen and provides the user with a coupon code.

[0892] Example 2

[0893] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0894] In today's world, it is extremely difficult to quickly provide appropriate tourist information, transit directions, and restaurant and accommodation information in multiple languages ​​to foreign visitors to Japan and people who do not regularly use public transportation. Furthermore, while this information needs to be tailored to the user's emotions and specific needs, there is a lack of technological means to achieve this. To solve this problem, flexible and personalized information provision that combines multilingual support and emotion recognition is required.

[0895] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0896] In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for acquiring user emotion data and transmitting it to the server, means for the server to analyze the emotion data using an emotion engine and personalize the information, means for collecting feedback from users and updating the database, means for translating languages ​​using a generative AI model, and means for translating tourist attraction information into multiple languages ​​using a generative AI model. This makes it possible to provide users with accurate and personalized tourist information and transportation guidance in multiple languages, regardless of language barriers.

[0897] "Means for obtaining user location information and transmitting it to a server" refers to a series of processes by which a device obtains the latitude and longitude information of the current location using GPS or other location identification technology and transmits that information to a server.

[0898] "Means of searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user, and providing information in multiple languages" refers to a function that enables the server to search for relevant information from a database in accordance with the language setting selected by the user, and provide that information to the user in multiple languages.

[0899] "Means for providing detailed transfer information for public transport" is a system function that allows the server to calculate the optimal public transport route and detailed transfer information based on the user's current location and destination, and provide the results to the user.

[0900] "Means for acquiring user emotion data and transmitting it to the server" refers to the process by which the device uses a camera or microphone to recognize emotions from the user's facial expressions and voice, and transmits that data to the server.

[0901] "Means for the server to analyze emotional data using an emotion engine and personalize information" refers to a function in which the server analyzes the received emotional data using an emotion engine (e.g., emotion recognition software) and, based on the results, customizes the information provided to the user in an optimal manner.

[0902] "Means for collecting feedback from users and updating the database" refers to the process by which the server collects feedback information provided by users and updates the contents of the database based on that information.

[0903] A "language translation method using a generative AI model" is a technique or process that uses generative AI (e.g., a natural language processing model) to translate a given text into multiple languages.

[0904] "Means for translating tourist destination information into multiple languages ​​using a generative AI model" is a function that utilizes a generative AI model to collect information about tourist destinations, translate that information into multiple languages, and provide it.

[0905] This invention is a system that provides detailed transfer information, tourist attractions, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more appropriate information based on the user's location information and emotional data, and achieves multilingual support using generative AI. A specific embodiment of this system is described below.

[0906] The server is responsible for many information processing tasks. Specifically, it collects information about tourist attractions, restaurants, and accommodations from external data sources and stores it in a database. For example, it uses the Google Places API to obtain information about Sensoji Temple and stores it in a database. The server also uses a generative AI model (e.g., OpenAI GPT-3) to translate the collected information into multiple languages. For example, it uses prompts such as "Translate the following Japanese text into English: 'Sensoji Temple is one of the most famous temples in Tokyo.'" It also provides appropriate translation data based on the language selected by the user.

[0907] The device is responsible for providing the user interface, obtaining location information, recognizing emotions, and displaying data. Specifically, the device is built using a platform such as React Native and provides a settings screen that allows the user to change language settings. The device uses its GPS function to obtain the user's current location (for example, the latitude and longitude around Sensoji Temple) and sends this to the server. It also uses the device's built-in camera and microphone to obtain emotional data from the user's facial expressions and voice, and sends this to the server. The server uses an emotion engine (for example, Microsoft Azure's Emotion API) to analyze the emotional data and personalize the information it provides based on the results.

[0908] Users download the app and set the language after installation. When launched for the first time, a language setting screen appears, allowing users to select their preferred language (e.g., English). Users can enter their destination (e.g., "Asakusa Temple") in the app's search bar to check detailed information and transit directions. After visiting, they can enter and submit feedback within the app and receive discount coupons. Additionally, when users express their emotions (e.g., smiling) while using the app, the device captures that data and sends it to the server.

[0909] Using a generative AI model makes it possible to smoothly support multiple languages. For example, by sending a prompt such as "Translate the following Japanese text into English: 'Sensoji Temple is one of the most famous temples in Tokyo,'" to the AI ​​model, it is possible to obtain information translated into multiple languages, including English and Chinese.

[0910] This makes it possible to provide users with accurate and personalized tourist information and transportation guidance in multiple languages, overcoming language barriers. This system is extremely useful for tourists and people who do not regularly use public transportation, and offers a significant improvement in convenience compared to conventional information provision systems.

[0911] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0912] Step 1:

[0913] The device provides the user interface

[0914] The device builds an application using a platform such as React Native and provides the user with a UI that includes a settings screen for language settings, a search bar, a feedback form, etc. The input is the user's operation (e.g., language selection, search term input), and the output is the dynamic display of the interface in response to the user's operation.

[0915] Step 2:

[0916] The device acquires location information and sends it to the server.

[0917] The device uses its GPS function to obtain the user's current location (e.g., latitude and longitude information). The specific operation is to use navigator.geolocation.getCurrentPosition. The input is the location information obtained from the device's GPS function, and the output is the location information data sent to the server.

[0918] Step 3:

[0919] The device acquires emotion data and sends it to the server.

[0920] The device uses a camera and microphone to acquire the user's emotional data. Specifically, it analyzes the user's facial expressions captured by the camera and the tone of voice picked up by the microphone. The input is the user's facial expressions and voice, and the output is emotional data analyzed by an emotion recognition engine (e.g., Microsoft Azure's Emotion API). This data is also sent to the server.

[0921] Step 4:

[0922] The server collects tourist spot information and stores it in a database

[0923] The server calls an external API (e.g., Google Places API) to obtain tourist spot information and saves it in a database. Specifically, it periodically sends API requests to obtain the latest information. The input is tourist spot information obtained from the API, and the output is the latest information saved in the database.

[0924] Example: GET https: / / maps.googleapis.com / maps / api / place / details / json?placeid=xxxxxx&key=API_KEY

[0925] Step 5:

[0926] The information collected by the server is translated into multiple languages ​​using a generative AI model.

[0927] The server uses a generative AI model (e.g., OpenAI GPT-3) to translate the collected information into various languages. For example, a prompt such as "Translate the following Japanese text into English: Sensoji Temple is one of the most famous temples in Tokyo" is input to the generative AI. The input is the collected tourist spot information in Japanese, and the output is the translated information in each language.

[0928] Step 6:

[0929] The server provides information based on the user's language preference.

[0930] The server searches for translation data based on the language information set by the user and sends it to the device. The input is information in the set language, and the output is tourist spot information translated into the selected language.

[0931] Step 7:

[0932] The server analyzes the emotional data and personalizes the information.

[0933] The server analyzes the emotional data received by the emotion engine and personalizes the information provided based on the user's emotional state. The input is emotional data, and the output is personalized recommendations for tourist spots. For example, if the user is having fun, it will suggest spots with a lot of activity.

[0934] Step 8:

[0935] The server provides detailed public transport route information.

[0936] The server calculates the optimal route based on the user's current location and destination, and generates detailed transfer information. The input is the user's current location and destination information, and the output is detailed information such as the transportation to be used, the line to take, transfer stations, and travel time.

[0937] Example: route = calculate_optimal_route(current_location, destination)

[0938] Step 9:

[0939] Display the data the device receives from the server

[0940] The device displays the multilingual and personalized information received from the server on the screen. Specifically, it updates the search results screen and route guidance screen appropriately. The input is the translated and personalized information sent from the server, and the output is the specific information displayed on the screen.

[0941] Example: Display information about Sensoji Temple as a tourist spot in English.

[0942] Step 10:

[0943] Users provide feedback and receive discount coupons

[0944] After a user visits the app, they input and submit feedback. The server receives this feedback, updates the database, generates a discount coupon for the next visit, and sends it to the device. The input is the user's feedback information, and the output is updating the database and issuing a discount coupon.

[0945] Example: Enter "I enjoyed my visit to Sensoji Temple" in the feedback form and submit.

[0946] Through the above processing steps, the system provides users with personalized tourist information and transportation guidance in multiple languages, enabling a more meaningful tourist experience.

[0947] (Application example 2)

[0948] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0949] When foreign visitors to Japan and tourists from other regions visit shopping centers and commercial facilities, they often face problems such as language barriers, unclear destinations, and a lack of personalized information. In addition, the lack of location information and store information within the shopping center is also a problem. Additionally, there is a lack of personalized services based on emotional data, which can lead to lower user satisfaction.

[0950] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for recognizing the user's emotions and transmitting emotion data to the server, means for searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for collecting user feedback and updating the database, means for translating languages ​​using generative AI, means for personalizing store information and sales information based on the user's emotion data, and means for providing information on the user's current location and nearby stores within a shopping center. This enables smooth provision of information within the shopping center or commercial facility visited and personalized guidance services based on the user's emotions.

[0951] "User" refers to an individual who uses this system.

[0952] "Location information" is information that indicates the user's current location and is obtained using technology such as GPS.

[0953] A "server" is a central computing device that collects, processes, and provides information to users.

[0954] "Emotion data" is information about the user's emotional state that can be inferred from facial expressions, voice, etc.

[0955] "Means of providing information in multiple languages" refers to the process of using generative AI to translate information into various languages ​​and provide it to users.

[0956] "Transfer guide" is a guide that includes detailed route information for reaching a destination using public transportation.

[0957] "Feedback" refers to opinions and evaluations regarding systems and services provided by users.

[0958] A "database" is a repository of information on tourist spots, restaurants, accommodations, and more.

[0959] "Generative AI" is a type of artificial intelligence, a technology that generates natural language and other information from given data.

[0960] "Means of personalization" refers to methods of optimizing and providing information based on each user's emotional data and behavioral history.

[0961] A "shopping center" is a commercial facility with multiple stores, where customers can shop and enjoy services.

[0962] This invention is a system that enables foreign visitors to Japan and tourists from other regions to receive appropriate information and personalized guidance when they visit shopping centers and commercial facilities. This system consists of a server and a terminal, and performs the following processes.

[0963] First, the device uses its GPS function to obtain the user's location information and sends it to the server. Based on this location information, the server obtains information about the shopping center the user is visiting and nearby stores. The device also uses its built-in camera and microphone to obtain emotional data from the user's facial expressions and voice, which it then sends to the server. The emotional data is analyzed using an emotion engine to personalize the information provided.

[0964] Based on the language selected by the user, the server uses generative AI to perform multilingual translation. It searches a database of tourist attractions, restaurants, and accommodations and provides information in multiple languages. The use of generative AI enables appropriate translation in real time. The server also provides detailed public transport transfer information to help users travel smoothly to their destination.

[0965] The server has a feedback function that collects feedback from users and updates the database. It also has a function to issue discount coupons in response to feedback, making the next shopping experience even more fulfilling. It also has a means to personalize store information and sales information based on emotional data. It provides information on the user's current location within the shopping center and nearby stores, and provides appropriate guidance.

[0966] The hardware used is a smartphone, and the software includes expo-location (location information acquisition), expo-camera (camera function), and axios (API communication function). An example of an emotion detection API is Microsoft Azure Face API, and an example of a translation API is Google Translate API.

[0967] As a specific example, to suggest relaxation spaces and cafes based on emotion data, a camera can detect the user's "I want to relax" emotion while shopping, and the server receives the data. The server then provides information on relaxation cafes and spaces in multiple languages ​​and guides users to the nearest location based on their location information. Furthermore, if a shopping center is attracting tourists from multiple countries, it can provide information on events at the center in multiple languages.

[0968] Examples of prompts using generative AI models include:

[0969] "Analyze my current emotions and suggest the best relaxation spot based on that."

[0970] "Please provide store information in English."

[0971] This will enable tourists and users to receive appropriate information and guidance, resulting in a more satisfying shopping experience.

[0972] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0973] Step 1:

[0974] The device uses its GPS to obtain the user's current location. This location information is obtained by the device's location service and is expressed as latitude and longitude coordinates. This location information is then sent to the server.

[0975] Step 2:

[0976] The server searches its database for store information in and around the shopping center based on the received location information. The search results provide a list of stores and services related to the user's current location. This list is obtained by matching the location information with store information that matches.

[0977] Step 3:

[0978] The device uses the smartphone's camera and microphone to acquire emotional data from the user's facial expressions and voice. The emotional data is obtained by analyzing the images and audio data acquired from the camera and microphone. This data is then sent to a server.

[0979] Step 4:

[0980] The server uses an emotion engine to analyze the received emotion data. The analysis results in the user's current emotional state (e.g., wanting to relax, wanting to have fun, etc.). This analysis is performed using a combination of image recognition and voice analysis technologies.

[0981] Step 5:

[0982] The server uses generative AI to perform multilingual translation based on the language selected by the user. Specifically, it translates store information retrieved from the database into the language selected by the user in real time. The translation results in store information available in multiple languages.

[0983] Step 6:

[0984] The server then personalizes store and sale information based on the user's emotional data and location information. Specifically, it prioritizes suggestions for relaxing cafes and specific stores based on the analyzed emotional data. These suggestions are made based on the user's current location.

[0985] Step 7:

[0986] The server sends the multilingually translated personalized information to the device, which then receives it and displays it to the user. The displayed content includes information about stores near the user's current location, sales information, and recommended stores based on user sentiment.

[0987] Step 8:

[0988] Users can then act on the information provided and enjoy their shopping experience. During this process, they can also use their devices to check detailed public transport route information. The route information is based on real-time traffic information provided by the server.

[0989] Step 9:

[0990] Users provide feedback within the app. The device sends this feedback to the server, which then stores and updates the received feedback in a database. The feedback includes store ratings and impressions.

[0991] Step 10:

[0992] The server generates a discount coupon that can be used for the user's next purchase based on the provided feedback and sends it to the terminal, which then displays the coupon to the user, enhancing their next shopping experience.

[0993] This allows the system as a whole to provide smooth information provision and personalized guidance to users, thereby improving user satisfaction.

[0994] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0995] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0996] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0997] [Third embodiment]

[0998] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0999] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1000] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1001] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1002] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1003] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1004] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1005] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1006] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1007] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1008] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1009] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1010] This invention is a system that provides detailed transfer information, tourist attractions, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation. The system provides optimal information based on the user's location information and achieves multilingual support using generative AI.

[1011] System configuration

[1012] 1. Server-side processing

[1013] The server is responsible for the following processes:

[1014] 1. Building and updating the database

[1015] The server collects information on tourist spots, restaurants, and accommodations and stores it in a database.

[1016] Periodically retrieve the latest information from multiple external data sources and update the database.

[1017] 2. Multilingual support

[1018] Generative AI is used to translate collected information into multiple languages.

[1019] Select the appropriate translation data based on the language set by the user.

[1020] 3. Demand forecasting

[1021] The collected data will be analyzed and tourism demand will be predicted.

[1022] Based on the results of demand forecasting, we propose new business opportunities.

[1023] 2. Terminal processing

[1024] The device (such as a smartphone) is responsible for the following processes:

[1025] 1. Providing a user interface

[1026] Provide an interface designed for user ease of use.

[1027] You can change the language in the settings screen and search for destinations and points of interest.

[1028] 2. Obtaining location information

[1029] The device uses GPS to obtain the user's current location.

[1030] 3. Displaying Data

[1031] Display multilingual information received from the server.

[1032] Displays detailed information about tourist spots and specific transfer information.

[1033] 3. User-side processing

[1034] The user performs the following operations using the app.

[1035] 1. Installing the app and initial settings

[1036] The user downloads and installs the app.

[1037] After launching the app, set the language.

[1038] 2. Search and provide feedback

[1039] Users search for their destination and check detailed information and transfer information.

[1040] After using the service, provide feedback and receive discount coupons.

[1041] Specific examples of program processing

[1042] Example: A visit to Sensoji Temple

[1043] 1. Information collection and multilingual translation

[1044] The server collects tourist spot information and translates it into multiple languages ​​using generative AI.

[1045] Information about Sensoji Temple will be stored in a database and made available in multiple languages, including English and Chinese.

[1046] 2. User Search

[1047] The user launches the app and sets the language to English.

[1048] Enter "Sensoji Temple" in the app's search bar and tap the search button.

[1049] 3. Displaying Information

[1050] The device obtains the user's current location and sends it to the server.

[1051] The server sends multilingual information about Sensoji Temple to the terminal, which then displays the information in English.

[1052] 4. Route guidance

[1053] When the user requests route guidance, the device again obtains the current location and sends it to the server.

[1054] The server calculates the optimal transfer route and sends it to the terminal.

[1055] The device displays detailed transfer information, allowing the user to check train and bus transfer information.

[1056] 5. Feedback and Discount Coupons

[1057] After visiting a tourist attraction, users provide feedback within the app.

[1058] The server receives the feedback and updates the database.

[1059] The server issues a discount coupon and sends it to the terminal, which then displays the coupon to the user.

[1060] In this way, the present invention is a system that provides users with appropriate tourist information and transportation guidance in multiple languages ​​and utilizes feedback to improve services.

[1061] The processing flow will be explained below.

[1062] Step 1:

[1063] The user downloads and installs the app.

[1064] Step 2:

[1065] The user launches the app and selects the language they want to use (e.g., English) on the language settings screen.

[1066] Step 3:

[1067] The device will save the user's language preference and change all displays to the selected language.

[1068] Step 4:

[1069] The user enters the location they want to go to (for example, "Sensoji Temple") in the app's search bar and taps the search button.

[1070] Step 5:

[1071] The device obtains the user's current location information using its GPS function and sends that information and the search query ("Sensoji Temple") to the server.

[1072] Step 6:

[1073] Based on the current location information and search query received by the server, the server searches the database for relevant tourist spot information.

[1074] Step 7:

[1075] The tourist spot information acquired by the server is translated into the language set by the user (English) using generative AI.

[1076] Step 8:

[1077] The server transmits the translated tourist spot information to the terminal.

[1078] Step 9:

[1079] The information received by the device is displayed on the app screen, providing the user with detailed information about the tourist spot (location, opening hours, photos, etc.).

[1080] Step 10:

[1081] The user taps the "Directions" button to request directions to their destination.

[1082] Step 11:

[1083] The device again acquires the user's current location and sends it to the server.

[1084] Step 12:

[1085] The server calculates the optimal route from the current location to the destination and generates detailed transfer information.

[1086] Step 13:

[1087] The server transmits the calculated route guidance information to the terminal.

[1088] Step 14:

[1089] The terminal displays the route guidance information it receives on the screen, providing the user with details such as the means of transport to be used, the line to be taken, transfer stations, and travel time.

[1090] Step 15:

[1091] After a user visits a tourist attraction, they tap the "Feedback" button in the app to open a feedback form.

[1092] Step 16:

[1093] The user enters their opinion and rating in the feedback form and taps the submit button.

[1094] Step 17:

[1095] The terminal transmits the user's feedback information to the server.

[1096] Step 18:

[1097] The server stores the received feedback information in a database and performs statistical analysis to update the database.

[1098] Step 19:

[1099] The server generates a discount coupon that can be used next time for the user who provided the feedback, and transmits the information to the terminal.

[1100] Step 20:

[1101] The terminal displays discount coupon information on the application screen and provides the user with a coupon code.

[1102] Example 1

[1103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1104] For foreign visitors to Japan and those who do not regularly use public transportation, it is difficult to provide information on tourist attractions, restaurants, accommodations, and detailed transfer directions in the appropriate language, and to improve the quality of the service by incorporating user feedback. It is also not easy to predict tourism demand or propose new business opportunities. To solve these challenges, a system that uses generative AI models to achieve multilingual support and provide users with optimal information and services in real time is needed.

[1105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1106] In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for searching a database of tourist facilities, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for collecting feedback from users and updating the database, means for translating languages ​​using a generative AI model, means for forecasting demand and proposing new business opportunities, and means for providing real-time information based on the user's current location. This makes it possible to provide detailed tourist information and transfer information in multiple languages ​​to foreign visitors to Japan and people who do not regularly use public transportation, and to improve the quality of the service based on feedback.

[1107] "User location information" means data indicating the current location of a person holding a device, and is information obtained using GPS or other location identification technology.

[1108] "Tourist facilities" refer to tourist spots and tourist destinations that users would like to visit, and include historical buildings, parks, museums, etc.

[1109] "Food and beverage establishment" means a place where customers can enjoy food and beverages, and includes restaurants, cafes, bars, etc.

[1110] "Accommodation" refers to a place where users can stay overnight, and includes hotels, private lodgings, guest houses, etc.

[1111] "Means of providing information in multiple languages" refers to a system that provides information on tourist facilities, restaurants, and accommodations in the language selected by the user, and refers to the use of machine translation and multilingual databases.

[1112] "Means for providing detailed transfer information for public transport" refers to a system that provides detailed information on the optimal transfer routes and methods for users to reach their destination, including train and bus operation information.

[1113] "Means for collecting feedback and updating the database" refers to a system for collecting user ratings and comments, updating the database information based on that, and improving the quality of the service.

[1114] "Means for translating languages ​​using generative AI models" refers to a system that uses generative artificial intelligence models to translate text into different languages, and utilizes natural language processing technology.

[1115] "Means of forecasting demand and proposing new business opportunities" refers to a system that analyzes trends in tourism demand based on collected data and finds new business opportunities.

[1116] "Means for providing real-time information" refers to a mechanism that instantly provides the latest information based on the user's current location, including dynamic searches and information updates.

[1117] This invention is a system that provides detailed transfer information, tourist facilities, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation. This system provides optimal information based on the user's location information and achieves multilingual support using a generative AI model. A specific method for implementing this system is described below.

[1118] Server-side processing

[1119] The server uses the following hardware and software to collect, translate, and analyze various information.

[1120] 1. Building and updating the database

[1121] The server collects information on tourist facilities, restaurants, and accommodations from external data sources and stores it in a database. It periodically retrieves and updates the latest information using a web crawler or API. Specifically, MySQL or PostgreSQL is used as the database.

[1122] 2. Multilingual support

[1123] A generative AI model is used to translate the collected information into multiple languages. Specifically, OpenAI's GPT model and Google Translate API are used, and the translation results are stored in a database. An example prompt is as follows:

[1124] Prompt: "Translate the following information about the tourist spot 'Sensoji Temple' into English, Chinese, and Korean: [More information about the tourist spot]"

[1125] 3. Demand forecasting

[1126] The server uses data science tools (e.g., TensorFlow and Scikit-learn) to analyze the collected data and predict tourism demand, which will then suggest new business opportunities.

[1127] Terminal side processing

[1128] The device (smartphone, etc.) operates with the following configuration.

[1129] 1. Providing a user interface

[1130] The device uses React Native and Flutter to provide a user-friendly interface, including a search bar, settings menu, history display, and more.

[1131] 2. Obtaining location information

[1132] The device uses the smartphone's GPS to obtain the user's current location, using Google Maps API or Apple's Core Location.

[1133] 3. Displaying Data

[1134] The terminal displays the multilingual information received from the server, including detailed information on tourist attractions and transfer information, so that the user can check it.

[1135] User operation

[1136] The user performs the following operations using the app.

[1137] 1. Installing the app and initial settings

[1138] Users download and install the app from the App Store or Google Play Store, and set the language when they first launch it.

[1139] 2. Search and provide feedback

[1140] Users search for destinations within the app and check the displayed information. After visiting a tourist spot, they can use the app's feedback function to enter and submit their ratings and comments. This allows the server to update the database and issue discount coupons.

[1141] Example: A visit to Sensoji Temple

[1142] For example, when a user visits Sensoji Temple, information and guidance are provided in the following manner.

[1143] 1. The server collects information about Sensoji Temple from external data sources and translates it into multiple languages ​​using a generative AI model.

[1144] 2. The user launches the app and changes the language setting to English.

[1145] 3. The user enters "Sensoji Temple" and searches.

[1146] 4. The device obtains its current location using GPS and sends it to the server.

[1147] 5. The server sends the corresponding multilingual information to the terminal, which then displays the information in English.

[1148] 6. The user requests route guidance, and the device again obtains the current location and sends it to the server.

[1149] 7. The server calculates the optimal transfer route and sends it to the device, which then displays detailed transfer information.

[1150] 8. After visiting a tourist attraction, users can provide feedback and receive discount coupons.

[1151] In this way, the present invention is a system that provides users with appropriate tourist information and transportation guidance in multiple languages ​​and utilizes feedback to improve services.

[1152] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1153] System program processing flow:

[1154] Step 1: Collecting data and storing it in a database

[1155] The server collects information about tourist facilities, restaurants, and accommodations from external data sources (such as APIs and web crawlers). The server organizes this data and stores it in a database such as MySQL or PostgreSQL. The input includes information from external data sources, and the output is stored in the database.

[1156] Step 2: Multilingual Translation

[1157] The server translates the information stored in the database into multiple languages ​​using a generative AI model (e.g., GPT-4). The input includes the original sentence retrieved from the database, and the output is the translated data into multiple languages. The following is used as an example of a prompt sentence:

[1158] Prompt: "Translate the following information about the tourist spot 'Sensoji Temple' into English, Chinese, and Korean: [More information about the tourist spot]"

[1159] The server stores the translated data back in the database.

[1160] Step 3: Obtaining user location information

[1161] The device obtains the user's current location using the smartphone's GPS function. The input includes the device's GPS data, and the output is the latitude and longitude of the user's current location. Specifically, Google Maps API or Apple's Core Location is used.

[1162] Step 4: Providing a User Interface

[1163] The device uses React Native and Flutter to provide an interface that makes it easy for users to search and configure settings. Input includes user actions (search keywords and setting changes), and the appropriate screen is displayed as output.

[1164] Step 5: Providing multilingual information

[1165] The device sends a request to the server based on the user's current location and the language selected. The server receives the request, retrieves the corresponding language translation data from the database, and sends it to the device. The input includes the user's location information and selected language, and the output is multilingually translated tourist information.

[1166] Step 6: Provide detailed transit directions

[1167] The server calculates the optimal transit route based on the user's current location and destination information. The input includes the user's current location and destination, and the output is a detailed transit route. The optimal route is obtained using Google Maps API and other public transport APIs.

[1168] Step 7: Gather feedback and update the database

[1169] The terminal sends the user-provided feedback to the server, which receives the feedback and updates the database. The input contains the user's feedback data, and the output is an updated database.

[1170] Step 8: Issue a discount coupon

[1171] The server generates a discount coupon after the user provides the feedback and sends it to the terminal. The input includes the feedback status, and the generated coupon is obtained as the output. The terminal displays the coupon information to the user.

[1172] Specific flow of processing steps

[1173] Step 1: Collecting data and storing it in a database

[1174] The server retrieves information about tourist facilities, restaurants, and accommodations from tourism websites and related databases via API. The retrieved data is then properly organized and stored in a MySQL or PostgreSQL database.

[1175] Step 2: Multilingual Translation

[1176] The server takes each piece of data in the database and translates it into each language using a generative AI model. The translated data is then stored back in the database and made available in each language.

[1177] Step 3: Obtaining user location information

[1178] The device uses GPS to obtain the latitude and longitude of the current location, which is then sent to the server, where it is used to provide information appropriate to the current location.

[1179] Step 4: Providing a User Interface

[1180] The device uses React Native and Flutter to display an intuitive user interface, including a home screen with a search bar, settings menu, and history view, where users can search for destinations and change language settings.

[1181] Step 5: Providing multilingual information

[1182] The device requests the user's current location and preferred language setting from the server, which retrieves multilingual tourist information from a database and sends it to the device in JSON format, where it is displayed in the appropriate format.

[1183] Step 6: Provide detailed transit directions

[1184] When the user requests transit directions, the device again retrieves its current location and sends it to the server. The server uses the appropriate transport API to calculate detailed transit routes and sends them to the device. The device then displays specific transit directions, helping the user reach their destination easily.

[1185] Step 7: Gather feedback and update the database

[1186] When a user provides feedback on the tourist attractions they visited or the services they used, the device sends the feedback data to the server, which then incorporates the feedback into a database and updates the information, thereby providing other users with the latest information.

[1187] Step 8: Issue a discount coupon

[1188] After the user provides feedback, the server generates a discount coupon and sends it to the terminal, which displays the coupon to the user so that it can be applied the next time they use the service.

[1189] (Application example 1)

[1190] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1191] Although systems exist that allow tourists to access detailed tourist information in multiple languages ​​and receive optimal tourist guidance based on their location in real time, there is a lack of methods to link information with real-world scenery and multilingual audio guides. There is also a need for tourist route suggestions tailored to individual needs and for service improvements based on feedback.

[1192] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1193] In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for collecting user feedback and updating the database, means for translating languages ​​using generative AI, means for translating information about tourist attractions and restaurants into multiple languages ​​using a generative AI model, means for guiding tourist routes and places of interest based on the user's current location, means for displaying tourist attraction information overlaid on real-world scenery using augmented reality (AR) technology, and means for providing multilingual audio guides. This allows tourists to obtain detailed, multilingual tourist information in real time, improving their local tourist experience.

[1194] "User Location Information" means your current geographic location obtained using GPS or Wi-Fi.

[1195] A "server" refers to a computer system that provides databases and applications via a network.

[1196] "Tourist attractions" refer to places of interest or facilities that tourists visit in a particular area.

[1197] "Restaurant" refers to an establishment that serves food and drinks.

[1198] "Accommodation facilities" refer to hotels, inns, and other establishments where travelers can stay temporarily.

[1199] A "database" refers to a system for organizing and storing information.

[1200] "Means for providing information in multiple languages" refers to the function of providing users with necessary information in multiple languages.

[1201] "Public transportation" refers to transportation systems such as buses and trains used by the general public.

[1202] "Transfer information" refers to providing detailed route information when users transfer between different public transportation modes.

[1203] "Feedback" refers to the evaluations and opinions provided by users after using a system or service.

[1204] "Generative AI" refers to technology that uses artificial intelligence to generate new language data and translations.

[1205] "Multilingual audio guide" refers to a system that provides audio guidance in multiple languages.

[1206] "Augmented reality (AR) technology" refers to the technology of overlaying digital information onto real-world scenery.

[1207] This invention relates to a tourism information provision system that allows tourists to receive detailed tourist information in multiple languages. This system utilizes user location information, a database of tourist spots, restaurants, and accommodations, multilingual translation, public transportation transfer information, feedback collection, generative AI, multilingual audio guides, and augmented reality (AR).

[1208] Hardware and software used

[1209] 1. Hardware

[1210] Smartphone (GPS, camera)

[1211] Smart glasses or head-mounted display (AR display, GPS)

[1212] 2. Software

[1213] Mobile app (compatible with iOS / Android)

[1214] Generative AI models (e.g., GPT-4)

[1215] Location services (Google Maps API)

[1216] Text-to-speech service (e.g., Amazon Polly)

[1217] AR Kit (ARCore, ARKit)

[1218] System Overview

[1219] Server Features

[1220] The server builds a database of information on tourist attractions, restaurants, and accommodations, and updates it regularly. The collected information is translated into multiple languages ​​using a generative AI model and provided based on the language selected by the user. The server also collects user feedback and updates the database.

[1221] Generative AI models are used to translate information into multiple languages, such as displaying detailed information about tourist attractions from Japanese to English.

[1222] Device Features

[1223] The device uses GPS to acquire the user's current location and sends it to the server. It also provides a user interface and displays information about tourist spots searched for by the user in multiple languages. It also provides public transport transfer information to support the user's travels.

[1224] When using the augmented reality (AR) function, information is superimposed on the real world via smart glasses or a head-mounted display. For example, when a user arrives at a tourist spot, the area's history and attractions can be displayed in AR. A multilingual audio guide function is also provided, providing detailed audio guidance about tourist spots.

[1225] User operations

[1226] Users download the mobile app and set their language preferences. When they launch the app, the app acquires their location information and provides optimal tourist information based on their current location. Users can search for tourist attractions and check detailed information and public transportation transfer information. When they arrive at a tourist attraction, they can effectively obtain tourist information using AR displays and multilingual audio guides.

[1227] Examples of concrete examples and prompts

[1228] Specific examples

[1229] When you visit an attraction:

[1230] Launch the app and get your location using GPS.

[1231] Multilingual tourist attraction information is obtained from the server.

[1232] Smart glasses display AR information about tourist attractions in real-world scenes.

[1233] Audio guides explain the history of the attractions.

[1234] Example prompts

[1235] "Translate the Japanese information about tourist attractions into English and list five must-see spots."

[1236] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1237] Step 1:

[1238] A user launches an app on their smartphone and sets the language. The input is the user's selected language setting information, and the output is the initial settings screen of the app. The app then sends the language setting information to the server.

[1239] Step 2:

[1240] The device uses GPS to obtain the user's current location. The input is GPS data, and the output is the current location information. This location information is converted into a digital format and sent to the server.

[1241] Step 3:

[1242] The server searches the database for information on tourist attractions, restaurants, and accommodations based on the received location information. The input is location information and language setting information, and the output is tourist information related to the user's current location. This information is translated into multiple languages ​​using a generative AI model.

[1243] Step 4:

[1244] The server sends multilingual tourist information translated by a generative AI model to the terminal. The input is the translated tourist information, and the output is multilingual tourist information. The terminal displays this information in a user interface.

[1245] Step 5:

[1246] Users wear smart glasses or a head-mounted display and view tourist spot information in real time. The input is real-world scenery and tourist information, and the output is information overlaid using AR technology. The device uses a camera to capture the real-world scenery and overlays tourist information on it.

[1247] Step 6:

[1248] When a user arrives at a tourist spot, the device uses a speech synthesis service to provide a multilingual audio guide. The input is detailed information about the tourist spot and the user's preferred language, and the output is a speech synthesis guide voice. The speech synthesis service is used to generate and play the audio guide on the spot.

[1249] Step 7:

[1250] After viewing or using the tourist attraction information, the user provides feedback through the app. The input is the feedback information from the user, and the output is the feedback data. This feedback data is sent to the server and updates the database.

[1251] Step 8:

[1252] The server updates the database based on the feedback and issues discount coupons as necessary. The input is the feedback data and the user's usage history, and the output is the discount coupon information. The coupon is sent in digital format to the terminal and displayed to the user.

[1253] Step 9:

[1254] The server predicts inbound demand based on data on tourist spots, restaurants, and accommodations. The input is past usage data and tourist information, and the output is the demand forecast results. The server analyzes this data and predicts tourist demand.

[1255] Step 10:

[1256] The server creates and proposes new tourist spot and event information based on prompts generated using a generative AI model. The input is the prompt and current tourist information, and the output is new tourist suggestion information. An example prompt is, "Translate the Japanese information about tourist spots into English and list five highlights."

[1257] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1258] This invention is a system that provides detailed transfer information, tourist attractions, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation, and also combines it with an emotion engine that recognizes the user's emotions. It provides more appropriate information based on the user's location information and emotion data, and achieves multilingual support using generative AI.

[1259] System configuration

[1260] 1. Server-side processing

[1261] The server is responsible for the following processes:

[1262] 1. Building and updating the database

[1263] The server collects information on tourist spots, restaurants, and accommodations and stores it in a database.

[1264] Periodically retrieve the latest information from multiple external data sources and update the database.

[1265] 2. Multilingual support

[1266] Generative AI is used to translate collected information into multiple languages.

[1267] Select the appropriate translation data based on the language set by the user.

[1268] 3. Demand forecasting

[1269] The collected data will be analyzed and tourism demand will be predicted.

[1270] Based on the results of demand forecasting, we propose new business opportunities.

[1271] 4. Emotion recognition

[1272] The emotion engine is used to analyze the user's emotion data sent from the terminal.

[1273] Personalize the information you provide based on emotional data.

[1274] 2. Terminal processing

[1275] The device (such as a smartphone) is responsible for the following processes:

[1276] 1. Providing a user interface

[1277] Provide an interface designed for user ease of use.

[1278] You can change the language on the settings screen.

[1279] 2. Obtaining location information

[1280] The device uses GPS to obtain the user's current location.

[1281] 3. Emotion recognition

[1282] Emotional data is obtained from the user's facial expressions and voice using the device's built-in camera and microphone.

[1283] Emotion data is sent to the server.

[1284] 4. Displaying Data

[1285] Display multilingual and personalized information received from the server.

[1286] Displays detailed information about tourist spots and specific transfer information.

[1287] 3. User-side processing

[1288] The user performs the following operations using the app.

[1289] 1. Installing the app and initial settings

[1290] The user downloads and installs the app.

[1291] After launching the app, set the language.

[1292] 2. Search and provide feedback

[1293] Users search for their destination and check detailed information and transfer information.

[1294] After using the service, provide feedback and receive discount coupons.

[1295] 3. Providing Emotion Data

[1296] Users express their emotions when using the app.

[1297] The device recognizes the user's emotions and sends the data to the server.

[1298] Specific examples of program processing

[1299] Example: A visit to Sensoji Temple

[1300] 1. Information collection and multilingual translation

[1301] The server collects tourist spot information and translates it into multiple languages ​​using generative AI.

[1302] Information about Sensoji Temple will be stored in a database and made available in multiple languages, including English and Chinese.

[1303] 2. User Search

[1304] The user launches the app and sets the language to English.

[1305] Enter "Sensoji Temple" in the app's search bar and tap the search button.

[1306] 3. Displaying Information

[1307] The device acquires the user's current location information and emotion data and sends them to the server.

[1308] The server sends multilingual information about Sensoji Temple and personalized information based on the user's emotional data to the device, which then displays the information in English.

[1309] 4. Route guidance

[1310] When the user requests route guidance, the device again obtains the current location and sends it to the server.

[1311] The server calculates the optimal transfer route and generates detailed transfer information.

[1312] The route guidance information received by the terminal is displayed on the screen, providing the user with details such as the means of transport to be used, the line to be taken, transfer stations, and travel time.

[1313] 5. Feedback and Discount Coupons

[1314] After visiting a tourist attraction, users provide feedback within the app.

[1315] The server receives the feedback and updates the database.

[1316] The server generates a discount coupon that can be used next time for the user who provided the feedback and sends that information to the terminal, which then provides the coupon code to the user.

[1317] 6. Utilizing Emotional Data

[1318] Users express their emotions through their smartphone's camera and microphone while using the app.

[1319] The terminal sends the emotion data to the emotion engine and transmits it to the server.

[1320] The server analyzes the emotional data and prioritizes suggestions for specific tourist spots and services based on the user's current emotional state.

[1321] In this way, the present invention is a system that provides users with appropriate tourist information and transportation guidance in multiple languages, and further personalizes the information by utilizing emotion recognition.

[1322] The processing flow will be explained below.

[1323] Step 1:

[1324] The user downloads and installs the app.

[1325] Step 2:

[1326] The user launches the app and selects the language they want to use (e.g., English) on the language settings screen.

[1327] Step 3:

[1328] The device will save the user's language preference and change all displays to the selected language.

[1329] Step 4:

[1330] The user enters the location they want to go to (for example, "Sensoji Temple") in the app's search bar and taps the search button.

[1331] Step 5:

[1332] The device obtains the user's current location information using its GPS function and sends that information and the search query ("Sensoji Temple") to the server.

[1333] Step 6:

[1334] Based on the current location information and search query received by the server, the server searches the database for relevant tourist spot information.

[1335] Step 7:

[1336] The tourist spot information acquired by the server is translated into the language set by the user (English) using generative AI.

[1337] Step 8:

[1338] The server transmits the translated tourist spot information to the terminal.

[1339] Step 9:

[1340] The information received by the device is displayed on the app screen, providing the user with detailed information about the tourist spot (location, opening hours, photos, etc.).

[1341] Step 10:

[1342] While the user is using the app, the device's camera and microphone capture emotional data through the user's facial expressions and voice.

[1343] Step 11:

[1344] The emotion data acquired by the device is analyzed by an emotion engine to identify the user's emotional state.

[1345] Step 12:

[1346] The device transmits the user's current location and emotion data to the server.

[1347] Step 13:

[1348] The server re-searches tourist spots, restaurants, accommodations, etc. suitable for the user from the database based on the user's emotional state.

[1349] Step 14:

[1350] The server generates personalized information and translates it into the user's preferred language.

[1351] Step 15:

[1352] The server transmits the translated personalized information to the terminal.

[1353] Step 16:

[1354] The personalized information received by the device is displayed on the app screen.

[1355] Step 17:

[1356] The user taps the "Directions" button to request directions to their destination.

[1357] Step 18:

[1358] The device again acquires the user's current location and sends it to the server.

[1359] Step 19:

[1360] The server calculates the optimal route from the current location to the destination and generates detailed transfer information.

[1361] Step 20:

[1362] The server transmits the calculated route guidance information to the terminal.

[1363] Step 21:

[1364] The terminal displays the route guidance information it receives on the screen, providing the user with details such as the means of transport to be used, the line to be taken, transfer stations, and travel time.

[1365] Step 22:

[1366] After a user visits a tourist attraction, they tap the "Feedback" button in the app to open a feedback form.

[1367] Step 23:

[1368] The user enters their opinion and rating in the feedback form and taps the submit button.

[1369] Step 24:

[1370] The terminal transmits the user's feedback information to the server.

[1371] Step 25:

[1372] The server stores the received feedback information in a database and performs statistical analysis to update the database.

[1373] Step 26:

[1374] The server generates a discount coupon that can be used next time for the user who provided the feedback, and transmits the information to the terminal.

[1375] Step 27:

[1376] The terminal displays discount coupon information on the application screen and provides the user with a coupon code.

[1377] Example 2

[1378] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1379] In today's world, it is extremely difficult to quickly provide appropriate tourist information, transit directions, and restaurant and accommodation information in multiple languages ​​to foreign visitors to Japan and people who do not regularly use public transportation. Furthermore, while this information needs to be tailored to the user's emotions and specific needs, there is a lack of technological means to achieve this. To solve this problem, flexible and personalized information provision that combines multilingual support and emotion recognition is required.

[1380] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1381] In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for acquiring user emotion data and transmitting it to the server, means for the server to analyze the emotion data using an emotion engine and personalize the information, means for collecting feedback from users and updating the database, means for translating languages ​​using a generative AI model, and means for translating tourist attraction information into multiple languages ​​using a generative AI model. This makes it possible to provide users with accurate and personalized tourist information and transportation guidance in multiple languages, regardless of language barriers.

[1382] "Means for obtaining user location information and transmitting it to a server" refers to a series of processes by which a device obtains the latitude and longitude information of the current location using GPS or other location identification technology and transmits that information to a server.

[1383] "Means of searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user, and providing information in multiple languages" refers to a function that enables the server to search for relevant information from a database in accordance with the language setting selected by the user, and provide that information to the user in multiple languages.

[1384] "Means for providing detailed transfer information for public transport" is a system function that allows the server to calculate the optimal public transport route and detailed transfer information based on the user's current location and destination, and provide the results to the user.

[1385] "Means for acquiring user emotion data and transmitting it to the server" refers to the process by which the device uses a camera or microphone to recognize emotions from the user's facial expressions and voice, and transmits that data to the server.

[1386] "Means for the server to analyze emotional data using an emotion engine and personalize information" refers to a function in which the server analyzes the received emotional data using an emotion engine (e.g., emotion recognition software) and, based on the results, customizes the information provided to the user in an optimal manner.

[1387] "Means for collecting feedback from users and updating the database" refers to the process by which the server collects feedback information provided by users and updates the contents of the database based on that information.

[1388] A "language translation method using a generative AI model" is a technique or process that uses generative AI (e.g., a natural language processing model) to translate a given text into multiple languages.

[1389] "Means for translating tourist destination information into multiple languages ​​using a generative AI model" is a function that utilizes a generative AI model to collect information about tourist destinations, translate that information into multiple languages, and provide it.

[1390] This invention is a system that provides detailed transfer information, tourist attractions, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more appropriate information based on the user's location information and emotional data, and achieves multilingual support using generative AI. A specific embodiment of this system is described below.

[1391] The server is responsible for many information processing tasks. Specifically, it collects information about tourist attractions, restaurants, and accommodations from external data sources and stores it in a database. For example, it uses the Google Places API to obtain information about Sensoji Temple and stores it in a database. The server also uses a generative AI model (e.g., OpenAI GPT-3) to translate the collected information into multiple languages. For example, it uses prompts such as "Translate the following Japanese text into English: 'Sensoji Temple is one of the most famous temples in Tokyo.'" It also provides appropriate translation data based on the language selected by the user.

[1392] The device is responsible for providing the user interface, obtaining location information, recognizing emotions, and displaying data. Specifically, the device is built using a platform such as React Native and provides a settings screen that allows the user to change language settings. The device uses its GPS function to obtain the user's current location (for example, the latitude and longitude around Sensoji Temple) and sends this to the server. It also uses the device's built-in camera and microphone to obtain emotional data from the user's facial expressions and voice, and sends this to the server. The server uses an emotion engine (for example, Microsoft Azure's Emotion API) to analyze the emotional data and personalize the information it provides based on the results.

[1393] Users download the app and set the language after installation. When launched for the first time, a language setting screen appears, allowing users to select their preferred language (e.g., English). Users can enter their destination (e.g., "Asakusa Temple") in the app's search bar to check detailed information and transit directions. After visiting, they can enter and submit feedback within the app and receive discount coupons. Additionally, when users express their emotions (e.g., smiling) while using the app, the device captures that data and sends it to the server.

[1394] Using a generative AI model makes it possible to smoothly support multiple languages. For example, by sending a prompt such as "Translate the following Japanese text into English: 'Sensoji Temple is one of the most famous temples in Tokyo,'" to the AI ​​model, it is possible to obtain information translated into multiple languages, including English and Chinese.

[1395] This makes it possible to provide users with accurate and personalized tourist information and transportation guidance in multiple languages, overcoming language barriers. This system is extremely useful for tourists and people who do not regularly use public transportation, and offers a significant improvement in convenience compared to conventional information provision systems.

[1396] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1397] Step 1:

[1398] The device provides the user interface

[1399] The device builds an application using a platform such as React Native and provides the user with a UI that includes a settings screen for language settings, a search bar, a feedback form, etc. The input is the user's operation (e.g., language selection, search term input), and the output is the dynamic display of the interface in response to the user's operation.

[1400] Step 2:

[1401] The device acquires location information and sends it to the server.

[1402] The device uses its GPS function to obtain the user's current location (e.g., latitude and longitude information). The specific operation is to use navigator.geolocation.getCurrentPosition. The input is the location information obtained from the device's GPS function, and the output is the location information data sent to the server.

[1403] Step 3:

[1404] The device acquires emotion data and sends it to the server.

[1405] The device uses a camera and microphone to acquire the user's emotional data. Specifically, it analyzes the user's facial expressions captured by the camera and the tone of voice picked up by the microphone. The input is the user's facial expressions and voice, and the output is emotional data analyzed by an emotion recognition engine (e.g., Microsoft Azure's Emotion API). This data is also sent to the server.

[1406] Step 4:

[1407] The server collects tourist spot information and stores it in a database

[1408] The server calls an external API (e.g., Google Places API) to obtain tourist spot information and saves it in a database. Specifically, it periodically sends API requests to obtain the latest information. The input is tourist spot information obtained from the API, and the output is the latest information saved in the database.

[1409] Example: GET https: / / maps.googleapis.com / maps / api / place / details / json?placeid=xxxxxx&key=API_KEY

[1410] Step 5:

[1411] The information collected by the server is translated into multiple languages ​​using a generative AI model.

[1412] The server uses a generative AI model (e.g., OpenAI GPT-3) to translate the collected information into various languages. For example, a prompt such as "Translate the following Japanese text into English: Sensoji Temple is one of the most famous temples in Tokyo" is input to the generative AI. The input is the collected tourist spot information in Japanese, and the output is the translated information in each language.

[1413] Step 6:

[1414] The server provides information based on the user's language preference.

[1415] The server searches for translation data based on the language information set by the user and sends it to the device. The input is information in the set language, and the output is tourist spot information translated into the selected language.

[1416] Step 7:

[1417] The server analyzes the emotional data and personalizes the information.

[1418] The server analyzes the emotional data received by the emotion engine and personalizes the information provided based on the user's emotional state. The input is emotional data, and the output is personalized recommendations for tourist spots. For example, if the user is having fun, it will suggest spots with a lot of activity.

[1419] Step 8:

[1420] The server provides detailed public transport route information.

[1421] The server calculates the optimal route based on the user's current location and destination, and generates detailed transfer information. The input is the user's current location and destination information, and the output is detailed information such as the transportation to be used, the line to take, transfer stations, and travel time.

[1422] Example: route = calculate_optimal_route(current_location, destination)

[1423] Step 9:

[1424] Display the data the device receives from the server

[1425] The device displays the multilingual and personalized information received from the server on the screen. Specifically, it updates the search results screen and route guidance screen appropriately. The input is the translated and personalized information sent from the server, and the output is the specific information displayed on the screen.

[1426] Example: Display information about Sensoji Temple as a tourist spot in English.

[1427] Step 10:

[1428] Users provide feedback and receive discount coupons

[1429] After a user visits the app, they input and submit feedback. The server receives this feedback, updates the database, generates a discount coupon for the next visit, and sends it to the device. The input is the user's feedback information, and the output is updating the database and issuing a discount coupon.

[1430] Example: Enter "I enjoyed my visit to Sensoji Temple" in the feedback form and submit.

[1431] Through the above processing steps, the system provides users with personalized tourist information and transportation guidance in multiple languages, enabling a more meaningful tourist experience.

[1432] (Application example 2)

[1433] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1434] When foreign visitors to Japan and tourists from other regions visit shopping centers and commercial facilities, they often face problems such as language barriers, unclear destinations, and a lack of personalized information. In addition, the lack of location information and store information within the shopping center is also a problem. Additionally, there is a lack of personalized services based on emotional data, which can lead to lower user satisfaction.

[1435] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for recognizing the user's emotions and transmitting emotion data to the server, means for searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for collecting user feedback and updating the database, means for translating languages ​​using generative AI, means for personalizing store information and sales information based on the user's emotion data, and means for providing information on the user's current location and nearby stores within a shopping center. This enables smooth provision of information within the shopping center or commercial facility visited and personalized guidance services based on the user's emotions.

[1436] "User" refers to an individual who uses this system.

[1437] "Location information" refers to information that indicates the user's current location and is obtained using technology such as GPS.

[1438] A "server" is a central computing device that collects, processes, and provides information to users.

[1439] "Emotion data" is information about the emotional state of the user that can be inferred from facial expressions, voice, etc.

[1440] "Means of providing information in multiple languages" refers to the process of using generative AI to translate information into various languages ​​and provide it to users.

[1441] "Transfer guide" is a guide that includes detailed route information for reaching a destination using public transportation.

[1442] "Feedback" refers to opinions and evaluations regarding systems and services provided by users.

[1443] A "database" is a repository of information on tourist spots, restaurants, accommodations, and more.

[1444] "Generative AI" is a type of artificial intelligence, a technology that generates natural language and other information from given data.

[1445] "Means of personalization" refers to methods of optimizing and providing information based on each user's emotional data and behavioral history.

[1446] A "shopping center" is a commercial facility with multiple stores, where customers can shop and enjoy services.

[1447] This invention is a system that enables foreign visitors to Japan and tourists from other regions to receive appropriate information and personalized guidance when they visit shopping centers and commercial facilities. This system consists of a server and a terminal, and performs the following processes.

[1448] First, the device uses its GPS function to obtain the user's location information and sends it to the server. Based on this location information, the server obtains information about the shopping center the user is visiting and nearby stores. The device also uses its built-in camera and microphone to obtain emotional data from the user's facial expressions and voice, which it then sends to the server. The emotional data is analyzed using an emotion engine to personalize the information provided.

[1449] Based on the language selected by the user, the server uses generative AI to perform multilingual translation. It searches a database of tourist attractions, restaurants, and accommodations and provides information in multiple languages. The use of generative AI enables appropriate translation in real time. The server also provides detailed public transport transfer information to help users travel smoothly to their destination.

[1450] The server has a feedback function that collects feedback from users and updates the database. It also has a function to issue discount coupons in response to feedback, making the next shopping experience even more fulfilling. It also has a means to personalize store information and sales information based on emotional data. It provides information on the user's current location within the shopping center and nearby stores, and provides appropriate guidance.

[1451] The hardware used is a smartphone, and the software includes expo-location (location information acquisition), expo-camera (camera function), and axios (API communication function). An example of an emotion detection API is Microsoft Azure Face API, and an example of a translation API is Google Translate API.

[1452] As a specific example, to suggest relaxation spaces and cafes based on emotion data, a camera can detect the user's "I want to relax" emotion while shopping, and the server receives the data. The server then provides information on relaxation cafes and spaces in multiple languages ​​and guides users to the nearest location based on their location information. Furthermore, if a shopping center is attracting tourists from multiple countries, it can provide information on events at the center in multiple languages.

[1453] Examples of prompts using generative AI models include:

[1454] "Analyze my current emotions and suggest the best relaxation spot based on that."

[1455] "Please provide store information in English."

[1456] This will enable tourists and users to receive appropriate information and guidance, resulting in a more satisfying shopping experience.

[1457] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1458] Step 1:

[1459] The device uses its GPS to obtain the user's current location. This location information is obtained by the device's location service and is expressed as latitude and longitude coordinates. This location information is then sent to the server.

[1460] Step 2:

[1461] The server searches its database for store information in and around the shopping center based on the received location information. The search results provide a list of stores and services related to the user's current location. This list is obtained by matching the location information with store information that matches.

[1462] Step 3:

[1463] The device uses the smartphone's camera and microphone to acquire emotional data from the user's facial expressions and voice. The emotional data is obtained by analyzing the images and audio data acquired from the camera and microphone. This data is then sent to a server.

[1464] Step 4:

[1465] The server uses an emotion engine to analyze the received emotion data. The analysis results in the user's current emotional state (e.g., wanting to relax, wanting to have fun, etc.). This analysis is performed using a combination of image recognition and voice analysis technologies.

[1466] Step 5:

[1467] The server uses generative AI to perform multilingual translation based on the language selected by the user. Specifically, it translates store information retrieved from the database into the language selected by the user in real time. The translation results in store information available in multiple languages.

[1468] Step 6:

[1469] The server then personalizes store and sale information based on the user's emotional data and location information. Specifically, it prioritizes suggestions for relaxing cafes and specific stores based on the analyzed emotional data. These suggestions are made based on the user's current location.

[1470] Step 7:

[1471] The server sends the multilingually translated personalized information to the device, which then receives it and displays it to the user. The displayed content includes information about stores near the user's current location, sales information, and recommended stores based on user sentiment.

[1472] Step 8:

[1473] Users can act based on the information provided and enjoy their shopping experience. During this process, they can also use their devices to check detailed public transport route information. The route information is based on real-time traffic information provided by the server.

[1474] Step 9:

[1475] Users provide feedback within the app. The device sends this feedback to the server, which then stores and updates the received feedback in a database. The feedback includes store ratings and impressions.

[1476] Step 10:

[1477] The server generates a discount coupon for the user's next purchase based on the provided feedback and sends it to the terminal, which then displays the coupon to the user, enhancing their next shopping experience.

[1478] This allows the system as a whole to provide smooth information provision and personalized guidance to users, thereby improving user satisfaction.

[1479] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1480] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1481] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1482] [Fourth embodiment]

[1483] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1484] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1485] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1486] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1487] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1488] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1489] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1490] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1491] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1492] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1493] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1494] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1495] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1496] This invention is a system that provides detailed transfer information, tourist attractions, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation. The system provides optimal information based on the user's location information and achieves multilingual support using generative AI.

[1497] System configuration

[1498] 1. Server-side processing

[1499] The server is responsible for the following processes:

[1500] 1. Building and updating the database

[1501] The server collects information on tourist spots, restaurants, and accommodations and stores it in a database.

[1502] Periodically retrieve the latest information from multiple external data sources and update the database.

[1503] 2. Multilingual support

[1504] Generative AI is used to translate collected information into multiple languages.

[1505] Select the appropriate translation data based on the language set by the user.

[1506] 3. Demand forecasting

[1507] The collected data will be analyzed and tourism demand will be predicted.

[1508] Based on the results of demand forecasting, we propose new business opportunities.

[1509] 2. Terminal processing

[1510] The device (such as a smartphone) is responsible for the following processes:

[1511] 1. Providing a user interface

[1512] Provide an interface designed for user ease of use.

[1513] You can change the language in the settings screen and search for destinations and points of interest.

[1514] 2. Obtaining location information

[1515] The device uses GPS to obtain the user's current location.

[1516] 3. Displaying Data

[1517] Display multilingual information received from the server.

[1518] Displays detailed information about tourist spots and specific transfer information.

[1519] 3. User-side processing

[1520] The user performs the following operations using the app.

[1521] 1. Installing the app and initial settings

[1522] The user downloads and installs the app.

[1523] After launching the app, set the language.

[1524] 2. Search and provide feedback

[1525] Users search for their destination and check detailed information and transfer information.

[1526] After using the service, provide feedback and receive discount coupons.

[1527] Specific examples of program processing

[1528] Example: A visit to Sensoji Temple

[1529] 1. Information collection and multilingual translation

[1530] The server collects tourist spot information and translates it into multiple languages ​​using generative AI.

[1531] Information about Sensoji Temple will be stored in a database and made available in multiple languages, including English and Chinese.

[1532] 2. User Search

[1533] The user launches the app and sets the language to English.

[1534] Enter "Sensoji Temple" in the app's search bar and tap the search button.

[1535] 3. Displaying Information

[1536] The device obtains the user's current location and sends it to the server.

[1537] The server sends multilingual information about Sensoji Temple to the terminal, which then displays the information in English.

[1538] 4. Route guidance

[1539] When the user requests route guidance, the device again obtains the current location and sends it to the server.

[1540] The server calculates the optimal transfer route and sends it to the terminal.

[1541] The device displays detailed transfer information, allowing the user to check train and bus transfer information.

[1542] 5. Feedback and Discount Coupons

[1543] After visiting a tourist attraction, users provide feedback within the app.

[1544] The server receives the feedback and updates the database.

[1545] The server issues a discount coupon and sends it to the terminal, which then displays the coupon to the user.

[1546] In this way, the present invention is a system that provides users with appropriate tourist information and transportation guidance in multiple languages ​​and utilizes feedback to improve services.

[1547] The processing flow will be explained below.

[1548] Step 1:

[1549] The user downloads and installs the app.

[1550] Step 2:

[1551] The user launches the app and selects the language they want to use (e.g., English) on the language settings screen.

[1552] Step 3:

[1553] The device will save the user's language preference and change all displays to the selected language.

[1554] Step 4:

[1555] The user enters the location they want to go to (for example, "Sensoji Temple") in the app's search bar and taps the search button.

[1556] Step 5:

[1557] The device obtains the user's current location information using its GPS function and sends that information and the search query ("Sensoji Temple") to the server.

[1558] Step 6:

[1559] Based on the current location information and search query received by the server, the server searches the database for relevant tourist spot information.

[1560] Step 7:

[1561] The tourist spot information acquired by the server is translated into the language set by the user (English) using generative AI.

[1562] Step 8:

[1563] The server transmits the translated tourist spot information to the terminal.

[1564] Step 9:

[1565] The information received by the device is displayed on the app screen, providing the user with detailed information about the tourist spot (location, opening hours, photos, etc.).

[1566] Step 10:

[1567] The user taps the "Directions" button to request directions to their destination.

[1568] Step 11:

[1569] The device again acquires the user's current location and sends it to the server.

[1570] Step 12:

[1571] The server calculates the optimal route from the current location to the destination and generates detailed transfer information.

[1572] Step 13:

[1573] The server transmits the calculated route guidance information to the terminal.

[1574] Step 14:

[1575] The terminal displays the route guidance information it receives on the screen, providing the user with details such as the means of transport to be used, the line to be taken, transfer stations, and travel time.

[1576] Step 15:

[1577] After a user visits a tourist attraction, they tap the "Feedback" button in the app to open a feedback form.

[1578] Step 16:

[1579] The user enters their opinion and rating in the feedback form and taps the submit button.

[1580] Step 17:

[1581] The terminal transmits the user's feedback information to the server.

[1582] Step 18:

[1583] The server stores the received feedback information in a database and performs statistical analysis to update the database.

[1584] Step 19:

[1585] The server generates a discount coupon that can be used next time for the user who provided the feedback, and transmits the information to the terminal.

[1586] Step 20:

[1587] The terminal displays discount coupon information on the application screen and provides the user with a coupon code.

[1588] Example 1

[1589] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1590] For foreign visitors to Japan and those who do not regularly use public transportation, it is difficult to provide information on tourist attractions, restaurants, accommodations, and detailed transfer directions in the appropriate language, and to improve the quality of the service by incorporating user feedback. It is also not easy to predict tourism demand or propose new business opportunities. To solve these challenges, a system that uses generative AI models to achieve multilingual support and provide users with optimal information and services in real time is needed.

[1591] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1592] In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for searching a database of tourist facilities, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for collecting feedback from users and updating the database, means for translating languages ​​using a generative AI model, means for forecasting demand and proposing new business opportunities, and means for providing real-time information based on the user's current location. This makes it possible to provide detailed tourist information and transfer information in multiple languages ​​to foreign visitors to Japan and people who do not regularly use public transportation, and to improve the quality of the service based on feedback.

[1593] "User location information" means data indicating the current location of a person holding a device, and is information obtained using GPS or other location identification technology.

[1594] "Tourist facilities" refer to tourist spots and tourist destinations that users would like to visit, and include historical buildings, parks, museums, etc.

[1595] "Food and beverage establishment" means a place where customers can enjoy food and beverages, and includes restaurants, cafes, bars, etc.

[1596] "Accommodation" refers to a place where users can stay overnight, and includes hotels, private lodgings, guest houses, etc.

[1597] "Means of providing information in multiple languages" refers to a system that provides information on tourist facilities, restaurants, and accommodations in the language selected by the user, and refers to the use of machine translation and multilingual databases.

[1598] "Means for providing detailed transfer information for public transport" refers to a system that provides detailed information on the optimal transfer routes and methods for users to reach their destination, including train and bus operation information.

[1599] "Means for collecting feedback and updating the database" refers to a system for collecting user ratings and comments, updating the database information based on that, and improving the quality of the service.

[1600] "Means for translating languages ​​using generative AI models" refers to a system that uses generative artificial intelligence models to translate text into different languages, and utilizes natural language processing technology.

[1601] "Means of forecasting demand and proposing new business opportunities" refers to a system that analyzes trends in tourism demand based on collected data and finds new business opportunities.

[1602] "Means for providing real-time information" refers to a mechanism that instantly provides the latest information based on the user's current location, including dynamic searches and information updates.

[1603] This invention is a system that provides detailed transfer information, tourist facilities, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation. This system provides optimal information based on the user's location information and achieves multilingual support using a generative AI model. A specific method for implementing this system is described below.

[1604] Server-side processing

[1605] The server uses the following hardware and software to collect, translate, and analyze various information.

[1606] 1. Building and updating the database

[1607] The server collects information on tourist facilities, restaurants, and accommodations from external data sources and stores it in a database. It periodically retrieves and updates the latest information using a web crawler or API. Specifically, MySQL or PostgreSQL is used as the database.

[1608] 2. Multilingual support

[1609] A generative AI model is used to translate the collected information into multiple languages. Specifically, OpenAI's GPT model and Google Translate API are used, and the translation results are stored in a database. An example prompt is as follows:

[1610] Prompt: "Translate the following information about the tourist spot 'Sensoji Temple' into English, Chinese, and Korean: [More information about the tourist spot]"

[1611] 3. Demand forecasting

[1612] The server uses data science tools (e.g., TensorFlow and Scikit-learn) to analyze the collected data and predict tourism demand, which will then suggest new business opportunities.

[1613] Terminal side processing

[1614] The device (smartphone, etc.) operates with the following configuration.

[1615] 1. Providing a user interface

[1616] The device uses React Native and Flutter to provide a user-friendly interface, including a search bar, settings menu, history display, and more.

[1617] 2. Obtaining location information

[1618] The device uses the smartphone's GPS to obtain the user's current location, using Google Maps API or Apple's Core Location.

[1619] 3. Displaying Data

[1620] The terminal displays the multilingual information received from the server, including detailed information on tourist attractions and transfer information, so that the user can check it.

[1621] User operation

[1622] The user performs the following operations using the app.

[1623] 1. Installing the app and initial settings

[1624] Users download and install the app from the App Store or Google Play Store, and set the language when they first launch it.

[1625] 2. Search and provide feedback

[1626] Users search for destinations within the app and check the displayed information. After visiting a tourist spot, they can use the app's feedback function to enter and submit their ratings and comments. This allows the server to update the database and issue discount coupons.

[1627] Example: A visit to Sensoji Temple

[1628] For example, when a user visits Sensoji Temple, information and guidance are provided in the following manner.

[1629] 1. The server collects information about Sensoji Temple from external data sources and translates it into multiple languages ​​using a generative AI model.

[1630] 2. The user launches the app and changes the language setting to English.

[1631] 3. The user enters "Sensoji Temple" and searches.

[1632] 4. The device obtains its current location using GPS and sends it to the server.

[1633] 5. The server sends the corresponding multilingual information to the terminal, which then displays the information in English.

[1634] 6. The user requests route guidance, and the device again obtains the current location and sends it to the server.

[1635] 7. The server calculates the optimal transfer route and sends it to the device, which then displays detailed transfer information.

[1636] 8. After visiting a tourist attraction, users can provide feedback and receive discount coupons.

[1637] In this way, the present invention is a system that provides users with appropriate tourist information and transportation guidance in multiple languages ​​and utilizes feedback to improve services.

[1638] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1639] System program processing flow:

[1640] Step 1: Collecting data and storing it in a database

[1641] The server collects information about tourist facilities, restaurants, and accommodations from external data sources (such as APIs and web crawlers). The server organizes this data and stores it in a database such as MySQL or PostgreSQL. The input includes information from external data sources, and the output is stored in the database.

[1642] Step 2: Multilingual Translation

[1643] The server translates the information stored in the database into multiple languages ​​using a generative AI model (e.g., GPT-4). The input includes the original sentence retrieved from the database, and the output is the translated data into multiple languages. The following is used as an example of a prompt sentence:

[1644] Prompt: "Translate the following information about the tourist spot 'Sensoji Temple' into English, Chinese, and Korean: [More information about the tourist spot]"

[1645] The server stores the translated data back in the database.

[1646] Step 3: Obtaining user location information

[1647] The device obtains the user's current location using the smartphone's GPS function. The input includes the device's GPS data, and the output is the latitude and longitude of the user's current location. Specifically, Google Maps API or Apple's Core Location is used.

[1648] Step 4: Providing a User Interface

[1649] The device uses React Native and Flutter to provide an interface that makes it easy for users to search and configure settings. Input includes user actions (search keywords and setting changes), and the appropriate screen is displayed as output.

[1650] Step 5: Providing multilingual information

[1651] The device sends a request to the server based on the user's current location and the language selected. The server receives the request, retrieves the corresponding language translation data from the database, and sends it to the device. The input includes the user's location information and selected language, and the output is multilingually translated tourist information.

[1652] Step 6: Provide detailed transit directions

[1653] The server calculates the optimal transit route based on the user's current location and destination information. The input includes the user's current location and destination, and the output is a detailed transit route. The optimal route is obtained using Google Maps API and other public transport APIs.

[1654] Step 7: Gather feedback and update the database

[1655] The terminal sends the user-provided feedback to the server, which receives the feedback and updates the database. The input contains the user's feedback data, and the output is an updated database.

[1656] Step 8: Issue a discount coupon

[1657] The server generates a discount coupon after the user provides the feedback and sends it to the terminal. The input includes the feedback status, and the generated coupon is obtained as the output. The terminal displays the coupon information to the user.

[1658] Specific flow of processing steps

[1659] Step 1: Collecting data and storing it in a database

[1660] The server retrieves information about tourist facilities, restaurants, and accommodations from tourism websites and related databases via API. The retrieved data is then properly organized and stored in a MySQL or PostgreSQL database.

[1661] Step 2: Multilingual Translation

[1662] The server takes each piece of data in the database and translates it into each language using a generative AI model. The translated data is then stored back in the database and made available in each language.

[1663] Step 3: Obtaining user location information

[1664] The device uses GPS to obtain the latitude and longitude of the current location, which is then sent to the server, where it is used to provide information appropriate to the current location.

[1665] Step 4: Providing a User Interface

[1666] The device uses React Native and Flutter to display an intuitive user interface, including a home screen with a search bar, settings menu, and history view, where users can search for destinations and change language settings.

[1667] Step 5: Providing multilingual information

[1668] The device requests the user's current location and preferred language setting from the server, which retrieves multilingual tourist information from a database and sends it to the device in JSON format, where it is displayed in the appropriate format.

[1669] Step 6: Provide detailed transit directions

[1670] When the user requests transit directions, the device again retrieves its current location and sends it to the server. The server uses the appropriate transport API to calculate detailed transit routes and sends them to the device. The device then displays specific transit directions, helping the user reach their destination easily.

[1671] Step 7: Gather feedback and update the database

[1672] When a user provides feedback on the tourist attractions they visited or the services they used, the device sends the feedback data to the server, which then incorporates the feedback into a database and updates the information, thereby providing other users with the latest information.

[1673] Step 8: Issue a discount coupon

[1674] After the user provides feedback, the server generates a discount coupon and sends it to the terminal, which displays the coupon to the user so that it can be applied the next time they use the service.

[1675] (Application example 1)

[1676] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1677] Although systems exist that allow tourists to access detailed tourist information in multiple languages ​​and receive optimal tourist guidance based on their location in real time, there is a lack of methods to link information with real-world scenery and multilingual audio guides. There is also a need for tourist route suggestions tailored to individual needs and for service improvements based on feedback.

[1678] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1679] In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for collecting user feedback and updating the database, means for translating languages ​​using generative AI, means for translating information about tourist attractions and restaurants into multiple languages ​​using a generative AI model, means for guiding tourist routes and places of interest based on the user's current location, means for displaying tourist attraction information overlaid on real-world scenery using augmented reality (AR) technology, and means for providing multilingual audio guides. This allows tourists to obtain detailed, multilingual tourist information in real time, improving their local tourist experience.

[1680] "User Location Information" means your current geographic location obtained using GPS or Wi-Fi.

[1681] A "server" refers to a computer system that provides databases and applications via a network.

[1682] "Tourist attractions" refer to places of interest or facilities that tourists visit in a particular area.

[1683] "Restaurant" refers to an establishment that serves food and drinks.

[1684] "Accommodation facilities" refer to hotels, inns, and other establishments where travelers can stay temporarily.

[1685] A "database" refers to a system for organizing and storing information.

[1686] "Means for providing information in multiple languages" refers to the function of providing users with necessary information in multiple languages.

[1687] "Public transportation" refers to transportation systems such as buses and trains used by the general public.

[1688] "Transfer information" refers to providing detailed route information when users transfer between different public transportation modes.

[1689] "Feedback" refers to the evaluations and opinions provided by users after using a system or service.

[1690] "Generative AI" refers to technology that uses artificial intelligence to generate new language data and translations.

[1691] "Multilingual audio guide" refers to a system that provides audio guidance in multiple languages.

[1692] "Augmented reality (AR) technology" refers to the technology of overlaying digital information onto real-world scenery.

[1693] This invention relates to a tourism information provision system that allows tourists to receive detailed tourist information in multiple languages. This system utilizes user location information, a database of tourist spots, restaurants, and accommodations, multilingual translation, public transportation transfer information, feedback collection, generative AI, multilingual audio guides, and augmented reality (AR).

[1694] Hardware and software used

[1695] 1. Hardware

[1696] Smartphone (GPS, camera)

[1697] Smart glasses or head-mounted display (AR display, GPS)

[1698] 2. Software

[1699] Mobile app (compatible with iOS / Android)

[1700] Generative AI models (e.g., GPT-4)

[1701] Location services (Google Maps API)

[1702] Text-to-speech service (e.g., Amazon Polly)

[1703] AR Kit (ARCore, ARKit)

[1704] System Overview

[1705] Server Features

[1706] The server builds a database of information on tourist attractions, restaurants, and accommodations, and updates it regularly. The collected information is translated into multiple languages ​​using a generative AI model and provided based on the language selected by the user. The server also collects user feedback and updates the database.

[1707] Generative AI models are used to translate information into multiple languages, such as displaying detailed information about tourist attractions from Japanese to English.

[1708] Device Features

[1709] The device uses GPS to acquire the user's current location and sends it to the server. It also provides a user interface and displays information about tourist spots searched for by the user in multiple languages. It also provides public transport transfer information to support the user's travels.

[1710] When using the augmented reality (AR) function, information is superimposed on the real world via smart glasses or a head-mounted display. For example, when a user arrives at a tourist spot, the area's history and attractions can be displayed in AR. A multilingual audio guide function is also provided, providing detailed audio guidance about tourist spots.

[1711] User operations

[1712] Users download the mobile app and set their language preferences. When they launch the app, the app acquires their location information and provides optimal tourist information based on their current location. Users can search for tourist attractions and check detailed information and public transportation transfer information. When they arrive at a tourist attraction, they can effectively obtain tourist information using AR displays and multilingual audio guides.

[1713] Examples of concrete examples and prompts

[1714] Specific examples

[1715] When you visit an attraction:

[1716] Launch the app and get your location using GPS.

[1717] Multilingual tourist attraction information is obtained from the server.

[1718] Smart glasses display AR information about tourist attractions in real-world scenes.

[1719] Audio guides explain the history of the attractions.

[1720] Example prompts

[1721] "Translate the Japanese information about tourist attractions into English and list five must-see spots."

[1722] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1723] Step 1:

[1724] A user launches an app on their smartphone and sets the language. The input is the user's selected language setting information, and the output is the initial settings screen of the app. The app then sends the language setting information to the server.

[1725] Step 2:

[1726] The device uses GPS to obtain the user's current location. The input is GPS data, and the output is the current location information. This location information is converted into a digital format and sent to the server.

[1727] Step 3:

[1728] The server searches the database for information on tourist attractions, restaurants, and accommodations based on the received location information. The input is location information and language setting information, and the output is tourist information related to the user's current location. This information is translated into multiple languages ​​using a generative AI model.

[1729] Step 4:

[1730] The server sends multilingual tourist information translated by a generative AI model to the terminal. The input is the translated tourist information, and the output is multilingual tourist information. The terminal displays this information in a user interface.

[1731] Step 5:

[1732] Users wear smart glasses or a head-mounted display and view tourist spot information in real time. The input is real-world scenery and tourist information, and the output is information overlaid using AR technology. The device uses a camera to capture the real-world scenery and overlays tourist information on it.

[1733] Step 6:

[1734] When a user arrives at a tourist spot, the device uses a speech synthesis service to provide a multilingual audio guide. The input is detailed information about the tourist spot and the user's preferred language, and the output is a speech synthesis guide voice. The speech synthesis service is used to generate and play the audio guide on the spot.

[1735] Step 7:

[1736] After viewing or using the tourist attraction information, the user provides feedback through the app. The input is the feedback information from the user, and the output is the feedback data. This feedback data is sent to the server and updates the database.

[1737] Step 8:

[1738] The server updates the database based on the feedback and issues discount coupons as necessary. The input is the feedback data and the user's usage history, and the output is the discount coupon information. The coupon is sent in digital format to the terminal and displayed to the user.

[1739] Step 9:

[1740] The server predicts inbound demand based on data on tourist spots, restaurants, and accommodations. The input is past usage data and tourist information, and the output is the demand forecast results. The server analyzes this data and predicts tourist demand.

[1741] Step 10:

[1742] The server creates and proposes new tourist spot and event information based on prompts generated using a generative AI model. The input is the prompt and current tourist information, and the output is new tourist suggestion information. An example prompt is, "Translate the Japanese information about tourist spots into English and list five highlights."

[1743] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1744] This invention is a system that provides detailed transfer information, tourist attractions, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation, and also combines it with an emotion engine that recognizes the user's emotions. It provides more appropriate information based on the user's location information and emotion data, and achieves multilingual support using generative AI.

[1745] System configuration

[1746] 1. Server-side processing

[1747] The server is responsible for the following processes:

[1748] 1. Building and updating the database

[1749] The server collects information on tourist spots, restaurants, and accommodations and stores it in a database.

[1750] Periodically retrieve the latest information from multiple external data sources and update the database.

[1751] 2. Multilingual support

[1752] Generative AI is used to translate collected information into multiple languages.

[1753] Select the appropriate translation data based on the language set by the user.

[1754] 3. Demand forecasting

[1755] The collected data will be analyzed and tourism demand will be predicted.

[1756] Based on the results of demand forecasting, we propose new business opportunities.

[1757] 4. Emotion recognition

[1758] The emotion engine is used to analyze the user's emotion data sent from the terminal.

[1759] Personalize the information you provide based on emotional data.

[1760] 2. Terminal processing

[1761] The device (such as a smartphone) is responsible for the following processes:

[1762] 1. Providing a user interface

[1763] Provide an interface designed for user ease of use.

[1764] You can change the language on the settings screen.

[1765] 2. Obtaining location information

[1766] The device uses GPS to obtain the user's current location.

[1767] 3. Emotion recognition

[1768] Emotional data is obtained from the user's facial expressions and voice using the device's built-in camera and microphone.

[1769] Emotion data is sent to the server.

[1770] 4. Displaying Data

[1771] Display multilingual and personalized information received from the server.

[1772] Displays detailed information about tourist spots and specific transfer information.

[1773] 3. User-side processing

[1774] The user performs the following operations using the app.

[1775] 1. Installing the app and initial settings

[1776] The user downloads and installs the app.

[1777] After launching the app, set the language.

[1778] 2. Search and provide feedback

[1779] Users search for their destination and check detailed information and transfer information.

[1780] After using the service, provide feedback and receive discount coupons.

[1781] 3. Providing Emotion Data

[1782] Users express their emotions when using the app.

[1783] The device recognizes the user's emotions and sends the data to the server.

[1784] Specific examples of program processing

[1785] Example: A visit to Sensoji Temple

[1786] 1. Information collection and multilingual translation

[1787] The server collects tourist spot information and translates it into multiple languages ​​using generative AI.

[1788] Information about Sensoji Temple will be stored in a database and made available in multiple languages, including English and Chinese.

[1789] 2. User Search

[1790] The user launches the app and sets the language to English.

[1791] Enter "Sensoji Temple" in the app's search bar and tap the search button.

[1792] 3. Displaying Information

[1793] The device acquires the user's current location information and emotion data and sends them to the server.

[1794] The server sends multilingual information about Sensoji Temple and personalized information based on the user's emotional data to the device, which then displays the information in English.

[1795] 4. Route guidance

[1796] When the user requests route guidance, the device again obtains the current location and sends it to the server.

[1797] The server calculates the optimal transfer route and generates detailed transfer information.

[1798] The route guidance information received by the terminal is displayed on the screen, providing the user with details such as the means of transport to be used, the line to be taken, transfer stations, and travel time.

[1799] 5. Feedback and Discount Coupons

[1800] After visiting a tourist attraction, users provide feedback within the app.

[1801] The server receives the feedback and updates the database.

[1802] The server generates a discount coupon that can be used next time for the user who provided the feedback and sends that information to the terminal, which then provides the coupon code to the user.

[1803] 6. Utilizing Emotional Data

[1804] Users express their emotions through their smartphone's camera and microphone while using the app.

[1805] The terminal sends the emotion data to the emotion engine and transmits it to the server.

[1806] The server analyzes the emotional data and prioritizes suggestions for specific tourist spots and services based on the user's current emotional state.

[1807] In this way, the present invention is a system that provides users with appropriate tourist information and transportation guidance in multiple languages, and further personalizes the information by utilizing emotion recognition.

[1808] The processing flow will be explained below.

[1809] Step 1:

[1810] The user downloads and installs the app.

[1811] Step 2:

[1812] The user launches the app and selects the language they want to use (e.g., English) on the language settings screen.

[1813] Step 3:

[1814] The device will save the user's language preference and change all displays to the selected language.

[1815] Step 4:

[1816] The user enters the location they want to go to (for example, "Sensoji Temple") in the app's search bar and taps the search button.

[1817] Step 5:

[1818] The device obtains the user's current location information using its GPS function and sends that information and the search query ("Sensoji Temple") to the server.

[1819] Step 6:

[1820] Based on the current location information and search query received by the server, the server searches the database for relevant tourist spot information.

[1821] Step 7:

[1822] The tourist spot information acquired by the server is translated into the language set by the user (English) using generative AI.

[1823] Step 8:

[1824] The server transmits the translated tourist spot information to the terminal.

[1825] Step 9:

[1826] The information received by the device is displayed on the app screen, providing the user with detailed information about the tourist spot (location, opening hours, photos, etc.).

[1827] Step 10:

[1828] While the user is using the app, the device's camera and microphone capture emotional data through the user's facial expressions and voice.

[1829] Step 11:

[1830] The emotion data acquired by the device is analyzed by an emotion engine to identify the user's emotional state.

[1831] Step 12:

[1832] The device transmits the user's current location and emotion data to the server.

[1833] Step 13:

[1834] The server re-searches tourist spots, restaurants, accommodations, etc. suitable for the user from the database based on the user's emotional state.

[1835] Step 14:

[1836] The server generates personalized information and translates it into the user's preferred language.

[1837] Step 15:

[1838] The server transmits the translated personalized information to the terminal.

[1839] Step 16:

[1840] The personalized information received by the device is displayed on the app screen.

[1841] Step 17:

[1842] The user taps the "Directions" button to request directions to their destination.

[1843] Step 18:

[1844] The device again acquires the user's current location and sends it to the server.

[1845] Step 19:

[1846] The server calculates the optimal route from the current location to the destination and generates detailed transfer information.

[1847] Step 20:

[1848] The server transmits the calculated route guidance information to the terminal.

[1849] Step 21:

[1850] The terminal displays the route guidance information it receives on the screen, providing the user with details such as the means of transport to be used, the line to be taken, transfer stations, and travel time.

[1851] Step 22:

[1852] After a user visits a tourist attraction, they tap the "Feedback" button in the app to open a feedback form.

[1853] Step 23:

[1854] The user enters their opinion and rating in the feedback form and taps the submit button.

[1855] Step 24:

[1856] The terminal transmits the user's feedback information to the server.

[1857] Step 25:

[1858] The server stores the received feedback information in a database and performs statistical analysis to update the database.

[1859] Step 26:

[1860] The server generates a discount coupon that can be used next time for the user who provided the feedback, and transmits the information to the terminal.

[1861] Step 27:

[1862] The terminal displays discount coupon information on the application screen and provides the user with a coupon code.

[1863] Example 2

[1864] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1865] In today's world, it is extremely difficult to quickly provide appropriate tourist information, transit directions, and restaurant and accommodation information in multiple languages ​​to foreign visitors to Japan and people who do not regularly use public transportation. Furthermore, while this information needs to be tailored to the user's emotions and specific needs, there is a lack of technological means to achieve this. To solve this problem, flexible and personalized information provision that combines multilingual support and emotion recognition is required.

[1866] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1867] In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for acquiring user emotion data and transmitting it to the server, means for the server to analyze the emotion data using an emotion engine and personalize the information, means for collecting feedback from users and updating the database, means for translating languages ​​using a generative AI model, and means for translating tourist attraction information into multiple languages ​​using a generative AI model. This makes it possible to provide users with accurate and personalized tourist information and transportation guidance in multiple languages, regardless of language barriers.

[1868] "Means for obtaining user location information and transmitting it to a server" refers to a series of processes by which a device obtains the latitude and longitude information of the current location using GPS or other location identification technology and transmits that information to a server.

[1869] "Means of searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user, and providing information in multiple languages" refers to a function that enables the server to search for relevant information from a database in accordance with the language setting selected by the user, and provide that information to the user in multiple languages.

[1870] "Means for providing detailed transfer information for public transport" is a system function that allows the server to calculate the optimal public transport route and detailed transfer information based on the user's current location and destination, and provide the results to the user.

[1871] "Means for acquiring user emotion data and transmitting it to the server" refers to the process by which the device uses a camera or microphone to recognize emotions from the user's facial expressions and voice, and transmits that data to the server.

[1872] "Means for the server to analyze emotional data using an emotion engine and personalize information" refers to a function in which the server analyzes the received emotional data using an emotion engine (e.g., emotion recognition software) and, based on the results, customizes the information provided to the user in an optimal manner.

[1873] "Means for collecting feedback from users and updating the database" refers to the process by which the server collects feedback information provided by users and updates the contents of the database based on that information.

[1874] A "language translation method using a generative AI model" is a technique or process that uses generative AI (e.g., a natural language processing model) to translate a given text into multiple languages.

[1875] "Means for translating tourist destination information into multiple languages ​​using a generative AI model" is a function that utilizes a generative AI model to collect information about tourist destinations, translate that information into multiple languages, and provide it.

[1876] This invention is a system that provides detailed transfer information, tourist attractions, restaurants, and accommodation information and reservation services in multiple languages ​​for foreign visitors to Japan and people who do not regularly use public transportation. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more appropriate information based on the user's location information and emotional data, and achieves multilingual support using generative AI. A specific embodiment of this system is described below.

[1877] The server is responsible for many information processing tasks. Specifically, it collects information about tourist attractions, restaurants, and accommodations from external data sources and stores it in a database. For example, it uses the Google Places API to obtain information about Sensoji Temple and stores it in a database. The server also uses a generative AI model (e.g., OpenAI GPT-3) to translate the collected information into multiple languages. For example, it uses prompts such as "Translate the following Japanese text into English: 'Sensoji Temple is one of the most famous temples in Tokyo.'" It also provides appropriate translation data based on the language selected by the user.

[1878] The device is responsible for providing the user interface, obtaining location information, recognizing emotions, and displaying data. Specifically, the device is built using a platform such as React Native and provides a settings screen that allows the user to change language settings. The device uses its GPS function to obtain the user's current location (for example, the latitude and longitude around Sensoji Temple) and sends this to the server. It also uses the device's built-in camera and microphone to obtain emotional data from the user's facial expressions and voice, and sends this to the server. The server uses an emotion engine (for example, Microsoft Azure's Emotion API) to analyze the emotional data and personalize the information it provides based on the results.

[1879] Users download the app and set the language after installation. When launched for the first time, a language setting screen appears, allowing users to select their preferred language (e.g., English). Users can enter their destination (e.g., "Asakusa Temple") in the app's search bar to check detailed information and transit directions. After visiting, they can enter and submit feedback within the app and receive discount coupons. Additionally, when users express their emotions (e.g., smiling) while using the app, the device captures that data and sends it to the server.

[1880] Using a generative AI model makes it possible to smoothly support multiple languages. For example, by sending a prompt such as "Translate the following Japanese text into English: 'Sensoji Temple is one of the most famous temples in Tokyo,'" to the AI ​​model, it is possible to obtain information translated into multiple languages, including English and Chinese.

[1881] This makes it possible to provide users with accurate and personalized tourist information and transportation guidance in multiple languages, overcoming language barriers. This system is extremely useful for tourists and people who do not regularly use public transportation, and offers a significant improvement in convenience compared to conventional information provision systems.

[1882] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1883] Step 1:

[1884] The device provides the user interface

[1885] The device builds an application using a platform such as React Native and provides the user with a UI that includes a settings screen for language settings, a search bar, a feedback form, etc. The input is the user's operation (e.g., language selection, search term input), and the output is the dynamic display of the interface in response to the user's operation.

[1886] Step 2:

[1887] The device acquires location information and sends it to the server.

[1888] The device uses its GPS function to obtain the user's current location (e.g., latitude and longitude information). The specific operation is to use navigator.geolocation.getCurrentPosition. The input is the location information obtained from the device's GPS function, and the output is the location information data sent to the server.

[1889] Step 3:

[1890] The device acquires emotion data and sends it to the server.

[1891] The device uses a camera and microphone to acquire the user's emotional data. Specifically, it analyzes the user's facial expressions captured by the camera and the tone of voice picked up by the microphone. The input is the user's facial expressions and voice, and the output is emotional data analyzed by an emotion recognition engine (e.g., Microsoft Azure's Emotion API). This data is also sent to the server.

[1892] Step 4:

[1893] The server collects tourist spot information and stores it in a database

[1894] The server calls an external API (e.g., Google Places API) to obtain tourist spot information and saves it in a database. Specifically, it periodically sends API requests to obtain the latest information. The input is tourist spot information obtained from the API, and the output is the latest information saved in the database.

[1895] Example: GET https: / / maps.googleapis.com / maps / api / place / details / json?placeid=xxxxxx&key=API_KEY

[1896] Step 5:

[1897] The information collected by the server is translated into multiple languages ​​using a generative AI model.

[1898] The server uses a generative AI model (e.g., OpenAI GPT-3) to translate the collected information into various languages. For example, a prompt such as "Translate the following Japanese text into English: Sensoji Temple is one of the most famous temples in Tokyo" is input to the generative AI. The input is the collected tourist spot information in Japanese, and the output is the translated information in each language.

[1899] Step 6:

[1900] The server provides information based on the user's language preference.

[1901] The server searches for translation data based on the language information set by the user and sends it to the device. The input is information in the set language, and the output is tourist spot information translated into the selected language.

[1902] Step 7:

[1903] The server analyzes the emotional data and personalizes the information.

[1904] The server analyzes the emotional data received by the emotion engine and personalizes the information provided based on the user's emotional state. The input is emotional data, and the output is personalized recommendations for tourist spots. For example, if the user is having fun, it will suggest spots with a lot of activity.

[1905] Step 8:

[1906] The server provides detailed public transport route information.

[1907] The server calculates the optimal route based on the user's current location and destination, and generates detailed transfer information. The input is the user's current location and destination information, and the output is detailed information such as the transportation to be used, the line to take, transfer stations, and travel time.

[1908] Example: route = calculate_optimal_route(current_location, destination)

[1909] Step 9:

[1910] Display the data the device receives from the server

[1911] The device displays the multilingual and personalized information received from the server on the screen. Specifically, it updates the search results screen and route guidance screen appropriately. The input is the translated and personalized information sent from the server, and the output is the specific information displayed on the screen.

[1912] Example: Display information about Sensoji Temple as a tourist spot in English.

[1913] Step 10:

[1914] Users provide feedback and receive discount coupons

[1915] After a user visits the app, they input and submit feedback. The server receives this feedback, updates the database, generates a discount coupon for the next visit, and sends it to the device. The input is the user's feedback information, and the output is updating the database and issuing a discount coupon.

[1916] Example: Enter "I enjoyed my visit to Sensoji Temple" in the feedback form and submit.

[1917] Through the above processing steps, the system provides users with personalized tourist information and transportation guidance in multiple languages, enabling a more meaningful tourist experience.

[1918] (Application example 2)

[1919] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1920] When foreign visitors to Japan and tourists from other regions visit shopping centers and commercial facilities, they often face problems such as language barriers, unclear destinations, and a lack of personalized information. In addition, the lack of location information and store information within the shopping center is also a problem. Additionally, there is a lack of personalized services based on emotional data, which can lead to lower user satisfaction.

[1921] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user location information and transmitting it to the server, means for recognizing the user's emotions and transmitting emotion data to the server, means for searching a database of tourist attractions, restaurants, and accommodations based on the language set by the user and providing information in multiple languages, means for providing detailed public transportation transfer information, means for collecting user feedback and updating the database, means for translating languages ​​using generative AI, means for personalizing store information and sales information based on the user's emotion data, and means for providing information on the user's current location and nearby stores within a shopping center. This enables smooth provision of information within the shopping center or commercial facility visited and personalized guidance services based on the user's emotions.

[1922] "User" refers to an individual who uses this system.

[1923] "Location information" refers to information that indicates the user's current location and is obtained using technology such as GPS.

[1924] A "server" is a central computing device that collects, processes, and provides information to users.

[1925] "Emotion data" is information about the emotional state of the user that can be inferred from facial expressions, voice, etc.

[1926] "Means of providing information in multiple languages" refers to the process of using generative AI to translate information into various languages ​​and provide it to users.

[1927] "Transfer guide" is a guide that includes detailed route information for reaching a destination using public transportation.

[1928] "Feedback" refers to opinions and evaluations regarding systems and services provided by users.

[1929] A "database" is a repository of information on tourist spots, restaurants, accommodations, and more.

[1930] "Generative AI" is a type of artificial intelligence, a technology that generates natural language and other information from given data.

[1931] "Means of personalization" refers to methods of optimizing and providing information based on each user's emotional data and behavioral history.

[1932] A "shopping center" is a commercial facility with multiple stores, where customers can shop and enjoy services.

[1933] This invention is a system that enables foreign visitors to Japan and tourists from other regions to receive appropriate information and personalized guidance when they visit shopping centers and commercial facilities. This system consists of a server and a terminal, and performs the following processes.

[1934] First, the device uses its GPS function to obtain the user's location information and sends it to the server. Based on this location information, the server obtains information about the shopping center the user is visiting and nearby stores. The device also uses its built-in camera and microphone to obtain emotional data from the user's facial expressions and voice, which it then sends to the server. The emotional data is analyzed using an emotion engine to personalize the information provided.

[1935] Based on the language selected by the user, the server uses generative AI to perform multilingual translation. It searches a database of tourist attractions, restaurants, and accommodations and provides information in multiple languages. The use of generative AI enables appropriate translation in real time. The server also provides detailed public transport transfer information to help users travel smoothly to their destination.

[1936] The server has a feedback function that collects feedback from users and updates the database. It also has a function to issue discount coupons in response to feedback, making the next shopping experience even more fulfilling. It also has a means to personalize store information and sales information based on emotional data. It provides information on the user's current location within the shopping center and nearby stores, and provides appropriate guidance.

[1937] The hardware used is a smartphone, and the software includes expo-location (location information acquisition), expo-camera (camera function), and axios (API communication function). An example of an emotion detection API is Microsoft Azure Face API, and an example of a translation API is Google Translate API.

[1938] As a specific example, to suggest relaxation spaces and cafes based on emotion data, a camera can detect the user's "I want to relax" emotion while shopping, and the server receives the data. The server then provides information on relaxation cafes and spaces in multiple languages ​​and guides users to the nearest location based on their location information. Furthermore, if a shopping center is attracting tourists from multiple countries, it can provide information on events at the center in multiple languages.

[1939] Examples of prompts using generative AI models include:

[1940] "Analyze my current emotions and suggest the best relaxation spot based on that."

[1941] "Please provide store information in English."

[1942] This will enable tourists and users to receive appropriate information and guidance, resulting in a more satisfying shopping experience.

[1943] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1944] Step 1:

[1945] The device uses its GPS to obtain the user's current location. This location information is obtained by the device's location service and is expressed as latitude and longitude coordinates. This location information is then sent to the server.

[1946] Step 2:

[1947] The server searches its database for store information in and around the shopping center based on the received location information. The search results provide a list of stores and services related to the user's current location. This list is obtained by matching the location information with store information that matches.

[1948] Step 3:

[1949] The device uses the smartphone's camera and microphone to acquire emotional data from the user's facial expressions and voice. The emotional data is obtained by analyzing the images and audio data acquired from the camera and microphone. This data is then sent to a server.

[1950] Step 4:

[1951] The server uses an emotion engine to analyze the received emotion data. The analysis results in the user's current emotional state (e.g., wanting to relax, wanting to have fun, etc.). This analysis is performed using a combination of image recognition and voice analysis technologies.

[1952] Step 5:

[1953] The server uses generative AI to perform multilingual translation based on the language selected by the user. Specifically, it translates store information retrieved from the database into the language selected by the user in real time. The translation results in store information available in multiple languages.

[1954] Step 6:

[1955] The server then personalizes store and sale information based on the user's emotional data and location information. Specifically, it prioritizes suggestions for relaxing cafes and specific stores based on the analyzed emotional data. These suggestions are made based on the user's current location.

[1956] Step 7:

[1957] The server sends the multilingually translated personalized information to the device, which then receives it and displays it to the user. The displayed content includes information about stores near the user's current location, sales information, and recommended stores based on user sentiment.

[1958] Step 8:

[1959] Users can act based on the information provided and enjoy their shopping experience. During this process, they can also use their devices to check detailed public transport route information. The route information is based on real-time traffic information provided by the server.

[1960] Step 9:

[1961] Users provide feedback within the app. The device sends this feedback to the server, which then stores and updates the received feedback in a database. The feedback includes store ratings and impressions.

[1962] Step 10:

[1963] The server generates a discount coupon for the user's next purchase based on the provided feedback and sends it to the terminal, which then displays the coupon to the user, enhancing their next shopping experience.

[1964] This allows the system as a whole to provide smooth information provision and personalized guidance to users, thereby improving user satisfaction.

[1965] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1966] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1967] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1968] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1969] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1970] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1971] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1972] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1973] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1974] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1975] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1976] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1977] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in th...

Claims

1. A means for acquiring user location information and transmitting it to a server; A means to search a database of tourist attractions, restaurants, and accommodations based on the user's language preference and provide information in multiple languages; A means of providing detailed public transport route information; a means of collecting user feedback and updating the database; A means of translating languages ​​using generative AI, A system including:

2. 2. The system according to claim 1, further comprising means for issuing discount coupons in response to feedback from users.

3. 2. The system according to claim 1, further comprising means for forecasting inbound demand based on information obtained from databases of tourist spots, restaurants, and accommodations.

Images