system

Abstract

A system is provided.SOLUTION: A guide system for assisting a traveler to acquire spot information, comprising: means for automatically summarizing spot information to be presented to the traveler using a generative model; means for translating the summarized spot information into a local language; means for collecting word-of-mouth information corresponding to a plurality of languages and automatically translating and summarizing the word-of-mouth information; means for displaying specific spot information on a map in response to a user request; and QR code reading means for allowing the user to acquire detailed information of a spot.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With the increasing number of foreign tourists visiting Japan, they are facing travel inconveniences due to their inability to understand the local language. They also face the challenge of being unable to smoothly enjoy the shops frequented by locals. Furthermore, there is a lack of ways to effectively utilize the vast amount of information available about tourist spots. To solve these problems, it is necessary to make it easier for tourists to obtain local information. [Means for solving the problem]

[0005] The present invention relates to a guide system for assisting travelers in obtaining spot information. It includes a means for automatically summarizing spot information presented to travelers using a generative model, a means for translating the summarized spot information into a local language, a means for automatically translating and summarizing reviews in multiple languages, a means for displaying specific spot information on a map in response to a user request, and a means for reading QR codes that allow users to obtain detailed spot information. It also includes a function for calculating the optimal route based on the traveler's current location and displaying navigation to the spot, and a means for users to translate conversations with local people in real time via voice or text input. This allows travelers to smoothly obtain local information and further enjoy their trip in Japan.

[0006] A "generative model" is an artificial intelligence technology that automatically summarizes information from large amounts of collected data and organizes it into an appropriate form.

[0007] "Spot information" is information about specific places that travelers visit, including details of tourist attractions, restaurants, shops, and tourist attractions.

[0008] A "local language" is a language commonly spoken in the country or region the traveler is visiting.

[0009] "Word-of-mouth information" is information that includes visitor reviews and ratings of a particular spot.

[0010] The "means for displaying specific spot information on a map" is a technology for displaying specific spots on a map based on user input or conditions.

[0011] "QR code reading means" refers to a device or software for reading a QR code and analyzing the information to obtain specific data.

[0012] The "optimal route calculation function" refers to an algorithm or technology for calculating and displaying the optimal travel route from the user's current location to their destination.

[0013] The "navigation display function" is a function for visually displaying the calculated route and guiding the user to the destination.

[0014] "A means of translating conversations with local people in real time through voice or text input" is a technology that translates what a user inputs via voice or text into other languages ​​in real time to support conversations. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a guide system for assisting travelers in obtaining spot information. Specific examples will be described below.

[0037] Server Processing

[0038] Data collection and model generation

[0039] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides. This data is stored in a database and used as training data for the generative model. The generative model is trained using natural language processing techniques to automatically summarize the collected information about tourist spots and convert it into an easy-to-understand text format.

[0040] Processing user requests

[0041] When a request is received from a device, the server analyzes the request and retrieves the relevant spot information from the database, which is then summarized by a generative model and automatically translated into the local language.

[0042] Translating and summarizing reviews

[0043] When a user requests review information, the server retrieves the relevant reviews from the database, translates them into major international languages ​​using an automatic translation engine, and then summarizes them using a generative model and provides them to the user.

[0044] Information distribution

[0045] The server then sends the translated and summarized information about the spots and reviews to the device, allowing users to easily understand the local information.

[0046] Terminal handling

[0047] QR code reading

[0048] When a user scans a QR code at a specific tourist spot, the device analyzes the code information, extracts the spot ID, and sends a request to the server. Once the information is received from the server, the device displays it in the GUI.

[0049] Map display and navigation

[0050] The device displays spot information from the server on a map and uses GPS information to provide navigation from the user's current location to the spot. When the user selects a specific spot, the device calculates the optimal route and displays it on the map.

[0051] Real-time translation

[0052] The device displays spot information and reviews received from the server in the local language. When the user converses with a local person, the device accepts voice or text input and translates the content using a real-time translation function. The translation results are immediately provided to the user.

[0053] Specific examples

[0054] For example, when a user visits a tourist spot in Japan, they scan the QR code of a specific restaurant. The device extracts the spot ID and sends a request to the server. The server retrieves the restaurant's information from the database, summarizes it using a generative model, translates it into the local language, and sends it. The device displays this information, and the user can also view reviews from other tourists.

[0055] In addition, users can use the map app to search for nearby tourist attractions and restaurants and select points of interest on the map. After selection, the device will display the optimal route and begin navigation.

[0056] In this way, the guide system according to the present invention enables travelers to smoothly obtain information across language barriers and enrich their local experiences.

[0057] The processing flow will be explained below.

[0058] Server Processing

[0059] Step 1:

[0060] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, and stores the collected data in a database.

[0061] Step 2:

[0062] The server uses the information stored in the database to train a generative model, which then uses natural language processing techniques to extract important keywords and context and acquire the ability to summarize information.

[0063] Step 3:

[0064] When a request for spot information is received from a terminal, the server analyzes the request and retrieves the relevant spot information from the database based on the analysis results.

[0065] Step 4:

[0066] The server passes the acquired spot information to a generative model, which summarizes and translates it into the local language, and then sends the results to the device.

[0067] Step 5:

[0068] When a user requests review information, the server retrieves the relevant review data from the database, then translates it into major international languages ​​using an automatic translation engine and summarizes it using a generative model. The translated and summarized review information is then sent to the device.

[0069] Terminal handling

[0070] Step 1:

[0071] The user reads the QR code, and the device analyzes the data and extracts the spot ID.

[0072] Step 2:

[0073] The device sends a request for spot information to the server based on the analyzed spot ID.

[0074] Step 3:

[0075] The spot information received from the server is displayed on the GUI, allowing users to check the spot information summarized in the local language.

[0076] Step 4:

[0077] The user opens a map app and searches for a specific spot. The device acquires the user's current location via GPS and displays candidate spots received from the server on the map.

[0078] Step 5:

[0079] When the user selects a specific spot, the device calculates the optimal route and displays route guidance on the map.

[0080] User operation example

[0081] Step 1:

[0082] Users find a QR code at a tourist spot and when they scan the code with their device, a request for information about the spot is automatically sent to the server.

[0083] Step 2:

[0084] The spot information received from the server is displayed on the terminal, and the user can check detailed restaurant information in the local language.

[0085] Step 3:

[0086] When a user taps the review tab, the device sends a request for review information to the server, and the translated and summarized review information received from the server is displayed on the device.

[0087] Step 4:

[0088] When a user searches for nearby tourist spots using a map app, multiple options are displayed on the map. When the user selects a specific spot, the device displays the optimal route and begins navigation.

[0089] Step 5:

[0090] When a user speaks to a local person, they input the content of the conversation using the device's voice input function. The translation is done in real time and the results are displayed on the screen.

[0091] Example 1

[0092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] There is a need for a system that allows travelers to efficiently obtain information about places they are visiting and utilize local information without experiencing language barriers. However, existing systems lack sufficient timely summarization and translation of geographical feature information and review information, and are unable to fully support multiple languages. Furthermore, they are limited in terms of navigation functions that allow travelers to instantly obtain the optimal route from their current location to their destination, and functions that support real-time communication with local people. This can make it difficult for travelers to navigate the local area.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0095] In this invention, the server includes a means for automatically summarizing geographical feature information to be presented to travelers using a machine learning model, a means for translating the summarized geographical feature information into a local language, and a means for collecting review information corresponding to multiple languages ​​and automatically translating and summarizing the information. This allows travelers to quickly and appropriately obtain the necessary geographical feature information without experiencing language barriers. Furthermore, the server includes a code reading means for displaying specific geographical feature information on a map in response to a user's request and allowing the user to obtain detailed information about the geographical feature, thereby significantly improving user convenience.

[0096] A "traveler" is someone who travels to different locations for tourism, work, or other purposes and needs information and services related to those locations.

[0097] "Geographical feature information" refers to data that indicates the location and characteristics of tourist attractions, facilities, natural landscapes, etc., as well as related information.

[0098] A "guidance system" refers to a system that assists users in efficiently obtaining the information they desire.

[0099] A "machine learning model" refers to an algorithm that is trained using large amounts of data to automatically perform a specific task (e.g., summarizing or translating text).

[0100] "Automatic summarization" refers to technology that extracts the main points from long textual information and organizes them in a short, easy-to-understand format.

[0101] "Means of translation" refers to the technology that converts information written in one language into another language.

[0102] "Review information" refers to feedback data in which users describe their experiences and evaluations.

[0103] "Means for displaying geographic feature information" refers to a method for visually displaying the collected and processed geographic feature information on a user's terminal.

[0104] "Code reading means" refers to technology that reads coded information such as QR codes and barcodes.

[0105] "Location information" refers to data that indicates the latitude and longitude of a specific location using technology such as GPS.

[0106] "Route guidance display function" refers to technology that calculates the optimal route from the current location to the destination and visually shows that route to the user.

[0107] "Voice or text input" refers to a method in which a user speaks to a terminal or inputs text.

[0108] "Means capable of real-time translation" refers to technology that instantly converts input speech or text into another language and quickly provides it to the user.

[0109] The present invention relates to a guidance system that enables travelers to efficiently obtain information about places they are visiting and to utilize local information without feeling the language barrier.

[0110] Server Processing

[0111] The server periodically crawls geographic feature information and review information from multiple travel sites and tourist guides. The collected data is stored in a database such as MongoDB. The collected data is then analyzed using a generative AI model (e.g., GPT-4), extracting and summarizing the necessary information. The summarized information is formatted as text and presented to users in an easy-to-understand format. The generative model is periodically retrained with training data to improve its accuracy.

[0112] Examples:

[0113] For example, information is collected from "travel review site A" and "tourist guide site B." This data is stored in MongoDB, analyzed using Python, and summarized using GPT-4.

[0114] Terminal handling

[0115] The device accepts input from the user (e.g., reading a QR code or selecting a spot). When the user scans a QR code at a specific tourist attraction, the device analyzes the code information to extract the spot ID and sends a request to the server. The information received from the server is displayed on the device's GUI. Furthermore, the device uses GPS information to identify the user's current location and uses the Google Maps API to display geographical feature information on a map. Route guidance is also provided to the user in real time via voice and text.

[0116] Examples:

[0117] For example, a user visits a tourist spot in Japan and scans a QR code for a specific restaurant. The device extracts the spot ID and sends a request to the server. The server retrieves the restaurant's information from the database, summarizes it using a generative AI model, translates it into the local language, and sends it to the device. The device displays this information, and the user can also view reviews from other travelers.

[0118] Processing reviews

[0119] The server receives a user request for review information. The server retrieves the relevant review information from the database and translates it into major international languages ​​using an automatic translation engine (e.g., Google Translate API). The generative AI model then summarizes the review information and provides it to the user.

[0120] Examples:

[0121] When a user requests to view reviews for a specific restaurant, the server retrieves multiple reviews from the database, translates them using the Google Translate API, summarizes them using GPT-4, and sends them to the device.

[0122] Real-time translation

[0123] The device accepts voice or text input from the user and uses real-time translation to translate the input into the specified language, providing the translation results instantly to the user.

[0124] Examples:

[0125] When a user wants to converse with a local person, they speak into the device. The speech recognition function converts the speech into text, which is then translated by the real-time translation engine and displayed or spoken back to the user.

[0126] Prompt Sentence Examples

[0127] "Please summarize recent reviews of this restaurant in Japanese."

[0128] "Please summarize the tourist information about Tokyo Tower and translate it into English."

[0129] Using this system, travelers can easily obtain detailed local information and enjoy a rich experience, regardless of language barriers.

[0130] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0131] The flow of this system's program processing

[0132] Step 1: Data collection and storage

[0133] explanation:

[0134] The server periodically crawls geographical feature information and review information from multiple travel sites and tourist guides.

[0135] input:

[0136] URLs for travel sites and tourist guides.

[0137] Specific behavior:

[0138] The server uses a crawler to gather the required information from the specified URLs.

[0139] output:

[0140] Collected data (geographical feature information and review information).

[0141] Step 2: Save to the database

[0142] explanation:

[0143] The server stores the collected data in a database.

[0144] input:

[0145] Geographical feature information and review information collected through crawling.

[0146] Specific behavior:

[0147] The server performs operations to insert data into a database such as MongoDB.

[0148] output:

[0149] Geographic feature information and review information stored in a database.

[0150] Step 3: Analyze and summarize data using the model

[0151] explanation:

[0152] The server uses a generative AI model (e.g., GPT-4) to analyze and summarize the collected data.

[0153] input:

[0154] Geographic feature information and review information retrieved from databases.

[0155] Specific behavior:

[0156] The server uses a programming language such as Python to analyze the data using the GPT-4 model, extracting and summarizing key points.

[0157] output:

[0158] Summarized geographic feature and review information.

[0159] Step 4: Translation

[0160] explanation:

[0161] The server translates the summarized information into the local language.

[0162] input:

[0163] Summarized geographic feature and review information.

[0164] Specific behavior:

[0165] The server uses a translation API (eg, Google Translate API) to translate the summary information into other languages.

[0166] output:

[0167] Geographical feature information and review information translated into local languages.

[0168] Step 5: Receiving a user request

[0169] explanation:

[0170] The terminal receives a request from a user.

[0171] input:

[0172] Scan QR codes and select specific spots.

[0173] Specific behavior:

[0174] The device uses a QR code reader to extract the code information or obtain the user's selection.

[0175] output:

[0176] The analyzed spot ID or request content.

[0177] Step 6: Parse the request and get information

[0178] explanation:

[0179] The server receives the request from the terminal, analyzes it, and retrieves the relevant information from the database.

[0180] input:

[0181] The spot ID or request content received from the device.

[0182] Specific behavior:

[0183] The server analyzes the request and issues the appropriate query to the database.

[0184] output:

[0185] Geographic feature information and review information retrieved from databases.

[0186] Step 7: Summarize and translate the information

[0187] explanation:

[0188] The server summarizes the acquired information using a generative AI model and translates it into the local language.

[0189] input:

[0190] Obtained geographic feature information and review information.

[0191] Specific behavior:

[0192] The server summarizes the information using the GPT-4 model and translates it into the local language using a translation API.

[0193] output:

[0194] Summarized and translated geographic feature and review information.

[0195] Step 8: Distributing information

[0196] explanation:

[0197] The server transmits the translated and summarized information to the terminal.

[0198] input:

[0199] Summarized and translated geographic feature and review information.

[0200] Specific behavior:

[0201] The server uses HTTP requests to send information to the device.

[0202] output:

[0203] Information delivered to your device.

[0204] Step 9: View information

[0205] explanation:

[0206] The terminal displays the received information in a GUI.

[0207] input:

[0208] The geographic feature information and review information received from the server.

[0209] Specific behavior:

[0210] Use the device's UI components to visually display information.

[0211] output:

[0212] Information visually displayed to the user.

[0213] Step 10: Real-time translation

[0214] explanation:

[0215] The device accepts and translates the user's voice or text input.

[0216] input:

[0217] User voice or text input.

[0218] Specific behavior:

[0219] The device uses a voice recognition system to convert speech into text, which is then translated using a real-time translation engine.

[0220] output:

[0221] Translated text or audio output.

[0222] By following each processing step of this system, travelers can efficiently obtain local information and gain a rich experience beyond language barriers.

[0223] (Application example 1)

[0224] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0225] Modern travelers want to quickly and efficiently obtain information about tourist attractions and restaurants in their destinations. However, language barriers and the complexity of information present obstacles. Furthermore, with advances in virtual reality technology, more and more people are seeking realistic travel experiences in virtual spaces without actually visiting the destinations. However, there is a lack of systems that support such experiences. There is a need for a system that can resolve these issues and enable users to smoothly enjoy both real and virtual travel experiences.

[0226] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0227] In this invention, the server includes means for automatically summarizing spot information to be presented to travelers using a generative model, means for translating the summarized spot information into a local language, means for collecting word-of-mouth information corresponding to multiple languages ​​and automatically translating and summarizing it, means for displaying information about specific tourist spots in a virtual space on a virtual screen, and navigation means for virtually exploring the local area based on the displayed information. This allows users to quickly and efficiently obtain spot information across language barriers and experience exploring the local area in a virtual space.

[0228] A "generative model" is a machine learning model that automatically summarizes and translates based on collected data.

[0229] "Spot information" refers to detailed data about places to visit, such as tourist attractions and restaurants.

[0230] A "local language" is a language commonly spoken in a given region.

[0231] "Word-of-mouth information" refers to the evaluations and impressions provided by users about specific spots.

[0232] A QR code is a two-dimensional barcode with embedded information, which can be obtained by scanning it with a device.

[0233] "Virtual space" is a virtual environment or world created by a computer.

[0234] "Virtual screen" refers to digital information displayed on a head-mounted display or smart glasses.

[0235] "Navigation means" refers to technologies and functions that assist users in understanding directions and routes to their destinations.

[0236] To implement this invention, the server, the terminal (such as smart glasses or a head-mounted display (HMD)), and the user need to cooperate with each other.

[0237] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides. This data is stored in a database and used as training data for the generative model. The generative model uses natural language processing technology to automatically summarize the collected information about tourist spots and convert it into an easy-to-understand text format. The server also analyzes user requests, retrieves relevant information about tourist spots from the database, and uses the generative model to summarize and automatically translate it into the local language.

[0238] When a user scans a QR code at a specific tourist spot, the device analyzes the code information, extracts the spot ID, and sends a request to the server. Once the information is received from the server, the device displays it on the GUI. The device also uses GPS information to display spot information on a map and provides navigation functions. When a user selects a specific spot, the device calculates the optimal route and displays it on the map. In addition, the device displays spot information and reviews received from the server in the local language and uses a real-time translation function to translate conversations with local people.

[0239] In a virtual space implementation, a device reads a QR code in the virtual space and displays information about a specific tourist spot on a virtual screen. A navigation function is also provided to virtually explore the location based on the displayed information. This allows users to enjoy a realistic travel experience in the virtual space without actually visiting the location.

[0240] Hardware and software used

[0241] Hardware: Server, smart glasses, head-mounted display, GPS receiver

[0242] Software: Natural language processing technology, generative models, databases, real-time translation engines

[0243] The server is built using Python. The program retrieves information about spots and reviews via API and summarizes and translates them using a generative AI model. Qt is used to create a GUI on the device, allowing users to smoothly retrieve information and navigate.

[0244] Specific examples

[0245] For example, when a user scans a specific QR code in the virtual space, detailed information about that tourist spot is instantly displayed. The user can check information such as "You can see a traditional festival at this spot" on the virtual screen. Also, when the user inputs "Take a virtual trip to this spot," the device starts virtual navigation and shows the user the optimal route.

[0246] Prompt Sentence Examples

[0247] "Get information about specific tourist spots and display it as a virtual guide."

[0248] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0249] Step 1:

[0250] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides. The input is data obtained through web scraping and API calls, and the output is organized data stored in a database. This data is later used as training data for generative models. Specifically, it processes data collection from websites using Python's BeautifulSoup and Requests.

[0251] Step 2:

[0252] The server uses the collected data to train a generative AI model. The input is the data collected in step 1, and the output is a model that performs summarization and translation. Specifically, the generative AI model is trained using TensorFlow and PyTorch.

[0253] Step 3:

[0254] The user scans a specific QR code on their device. The input is the image data of the QR code, and the output is the analyzed spot ID. Specifically, the device's camera captures the QR code and decodes it using a library such as OpenCV.

[0255] Step 4:

[0256] After extracting the spot ID, the device sends the ID to the server. The input is the spot ID, and the output is a request to the server. Specifically, it uses the Python Requests library to send the HTTP request.

[0257] Step 5:

[0258] The server retrieves the corresponding spot information from the database based on the received spot ID. The input is the spot ID and the output is the spot information. Specifically, it executes an SQL query to retrieve the information from the database.

[0259] Step 6:

[0260] The server summarizes the acquired spot information using a generative AI model and automatically translates it into the local language. The input is the spot information, and the output is the summarized and translated information. Specifically, data is input into a pre-trained generative AI model, and summary and translation results are obtained.

[0261] Step 7:

[0262] The server sends summarized and translated spot information to the terminal. The input is the summarized and translated information, and the output is the response to the terminal. Specifically, the information is returned as an HTTP response.

[0263] Step 8:

[0264] The terminal displays the received information in a GUI. The input is summarized and translated spot information, and the output is a visual display for the user. Specifically, the information is displayed using a GUI framework such as Qt.

[0265] Step 9:

[0266] The device uses GPS information to display spot information on a map and provide navigation. The input is the current location's GPS data, and the output is a navigation route on the map. Specifically, it calculates the route using Google Maps API and displays it on the map.

[0267] Step 10:

[0268] When a user speaks or inputs text, the device translates it in real time. The input is the user's voice or text, and the output is the translated text or voice. Specifically, the device performs text translation using the Google Translate API or similar.

[0269] Step 11:

[0270] In the virtual space, the device reads the QR code and displays information about a specific tourist spot on a virtual screen. The input is the QR code data, and the output is the spot information displayed on the virtual screen. Specific operations utilize QR code scanning technology and a virtual reality framework.

[0271] Step 12:

[0272] The device supports users in navigating within the virtual space based on the displayed information. The input is spot information and user operations, and the output is a navigation route within the virtual space. Specifically, the navigation system within the virtual space is implemented using an engine such as Unity.

[0273] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0274] The present invention relates to a guide system that recognizes the emotions of travelers and combines an emotion engine to optimize the travel experience. Specific examples will be described below.

[0275] Server Processing

[0276] Data collection and model generation

[0277] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, stores this data in a database, and trains the generative model using natural language processing technology to automatically summarize the information about tourist spots and reviews.

[0278] Processing user requests

[0279] When a request for spot information is received from a device, the server analyzes the request, retrieves the relevant spot information from the database, and summarizes and translates it using a generative model.

[0280] Emotion engine processing

[0281] The emotion engine embedded in the server analyzes the voice and text data received from the user to recognize the user's emotion, which is then taken into account in other processing steps.

[0282] Translating and summarizing reviews

[0283] When a user requests review information, the server retrieves the review data from the database, translates it using an automatic translation engine, and then summarizes it using a generative model.The server then customizes the content by reflecting the user's emotional information and sends it to the device.

[0284] Information distribution and suggestions

[0285] The emotion engine recognizes the user's emotions and optimizes the suggested spots and activities. If the user is under stress, the system prioritizes the display of information about relaxing spots.

[0286] Terminal handling

[0287] QR code reading

[0288] When a user reads the QR code, the device analyzes the code data and extracts the spot ID, which is then sent to the server as a request.

[0289] Map display and navigation

[0290] The device displays the spot information received from the server on a map and displays the optimal route from the user's current location to the spot. When the user selects a specific spot, the device begins navigation based on the selection.

[0291] Real-time translation and emotion recognition

[0292] The device receives information about spots and reviews from the server and displays them in the local language, and uses an emotion engine to recognize emotions based on the user's voice and text input. The recognized emotion information is sent to the server and reflected in processing.

[0293] Specific examples

[0294] For example, when a user visits a tourist spot, they can scan the QR code of a specific restaurant. The device extracts the spot ID and sends a request to the server. The server summarizes and translates the acquired restaurant information and presents it to the user. If the server determines that the user's current emotional state is stressful, it will also suggest information about relaxing spots.

[0295] Furthermore, when users search for nearby tourist attractions in the map app and select a specific spot, the device will calculate the optimal route and display it on the map. If users want to converse with local people, real-time translation is provided through voice input. Voice feedback based on emotions is also provided, providing a better travel experience.

[0296] In this way, the present invention makes it possible to recognize the user's emotions, smoothly obtain information across language barriers, and optimize the travel experience.

[0297] The processing flow will be explained below.

[0298] Server Processing

[0299] Step 1:

[0300] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, and stores the collected data in a database.

[0301] Step 2:

[0302] The server uses the information stored in the database to train a generative model, which uses natural language processing techniques to automatically summarize information about spots and reviews.

[0303] Step 3:

[0304] When a request for spot information is received from a terminal, the server analyzes the request and retrieves the corresponding spot information from the database.

[0305] Step 4:

[0306] The server passes the acquired spot information to the generative model, which summarizes and translates it into the local language. The translated and summarized information is then sent to the device.

[0307] Step 5:

[0308] When a user submits a review request, the server retrieves the review data from the database and translates it into major international languages ​​using an automatic translation engine.Then, a generative model summarizes the review information and customizes it based on the user's sentiment information.

[0309] Step 6:

[0310] The emotion engine built into the server analyzes the voice and text data received from the user and recognizes the user's emotions. The recognized emotion information is reflected in information processing and presentation.

[0311] Step 7:

[0312] Based on the user's emotions recognized by the emotion engine, the server optimizes and suggests information on spots and activities suitable for the traveler. Also, if the user is feeling stressed, it will prioritize providing information on spots where they can relax.

[0313] Terminal handling

[0314] Step 1:

[0315] The user scans the QR code at a tourist spot, and the device analyzes the QR code data to extract the spot ID.

[0316] Step 2:

[0317] The terminal sends a request for spot information to the server based on the analyzed spot ID.

[0318] Step 3:

[0319] The spot information received from the server is displayed in the local language on the terminal's GUI, and the user can check the displayed spot information.

[0320] Step 4:

[0321] The user opens a map app and searches for a specific spot. The device acquires the user's current location via GPS and displays the spot information received from the server on a map.

[0322] Step 5:

[0323] When the user selects a specific spot, the device calculates the optimal route and displays route guidance on the map.

[0324] Step 6:

[0325] The terminal displays the reviews received from the server to the user in the local language, with the reviews automatically translated and summarized.

[0326] Step 7:

[0327] When the user expresses their emotions through voice or text input, the device sends this input data to the server.

[0328] Step 8:

[0329] Based on the emotional feedback received from the server in real time, the device will suggest spot information and activities that correspond to the user's emotions.

[0330] User operation example

[0331] Step 1:

[0332] The user scans the QR code at a tourist spot and sends a request to the server to display the information on the device.

[0333] Step 2:

[0334] The server summarizes the spot information using a generative model, translates it, and sends it to the device, where the user can view the spot information in their local language.

[0335] Step 3:

[0336] If the user wants to check the reviews, they tap the reviews tab to send a request. The server translates and summarizes the reviews and sends them to the device.

[0337] Step 4:

[0338] Users use a map app to search for nearby tourist spots and select a specific spot, and the device calculates the optimal route and displays navigation.

[0339] Step 5:

[0340] When users converse with local people, they use the voice input function. The device sends the input speech to the server and displays the translation results in real time. The emotion engine also analyzes the user's emotions and provides customized feedback based on the results.

[0341] Example 2

[0342] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0343] Conventional guide systems do not adequately consider traveler emotions or language barriers, resulting in suboptimal travel experiences. Travelers often find themselves in situations where they feel stressed or are unable to obtain sufficient information due to not understanding the local language, resulting in lower travel satisfaction.

[0344] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0345] In this invention, the server includes means for automatically summarizing spot information to be presented to travelers using a generative model, means for translating the summarized spot information into a local language, means for collecting word-of-mouth information corresponding to multiple languages ​​and automatically translating and summarizing it, emotion analysis means for recognizing emotions from a user's voice or text input, and suggestion means for optimizing the spot information and suggestions based on the emotion analysis results. This enables travelers to smoothly obtain information across language barriers and receive optimal suggestions according to their emotions, thereby improving satisfaction with their travel experience.

[0346] A "generative model" is a machine learning algorithm that automatically summarizes information based on collected data.

[0347] A "local language" is a language commonly spoken in the area the traveller is visiting.

[0348] "Word-of-mouth information" refers to data in which users write their ratings and opinions about certain spots or services.

[0349] "Machine translation" refers to the technology of automatically converting text between multiple languages.

[0350] "Emotion analysis" is a technology that analyzes a user's voice and text data and recognizes their emotions.

[0351] "Suggestion means" refers to a function that suggests optimal spot information and activities to users based on the results of emotion analysis.

[0352] "Code reading" refers to the technology of reading code information such as QR codes using a device's camera.

[0353] "Navigation" refers to a function that provides guidance on a route from the user's current location to a destination.

[0354] A "user request" is a request for information made by a user to a server via a terminal.

[0355] A "database" is a system that systematically stores and manages collected spot information and reviews.

[0356] The present invention relates to a guide system for recognizing a traveler's emotions and optimizing the travel experience. Specific examples will be described below.

[0357] Server Processing

[0358] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, and stores the data in a database. This is done using web scraping technology, such as Python's BeautifulSoup library. The collected data is then stored in a database such as MongoDB.

[0359] The data is then trained using a generative AI model with natural language processing techniques (e.g., the BERT model) to automatically summarize information about attractions and reviews, ensuring that the information provided to travelers is concise and easy to understand.

[0360] When a request for specific spot information is received from a device, the server analyzes the request and retrieves the corresponding spot information from the database, summarizes this information using a generative AI model, and then translates it into the local language using tools such as Google Translate API.

[0361] The server is also equipped with an emotion engine that analyzes voice and text data received from users. For example, if a user says, "I've been feeling a bit stressed lately," the emotion analysis module analyzes this information and recognizes the user's emotion. This emotion information is used in subsequent processing steps.

[0362] When a user requests review information, the server retrieves the information from the database, translates it using an automatic translation engine, and summarizes it using a generative AI model. This summarized information is then customized to reflect the user's emotional information and sent to the device.

[0363] Based on the user's emotions recognized by the emotion engine, the server optimizes the recommendations of spot information and activities. For example, if the server recognizes that the user is under stress, it will prioritize providing information on relaxation spots.

[0364] Terminal handling

[0365] When a user reads the QR code, the device analyzes the code data to extract the spot ID and sends it as a request to the server. The spot information sent from the server is displayed on the device's map app, and the optimal route from the user's current location to the spot is shown.

[0366] The device's emotion engine analyzes the user's voice and text inputs to recognize the user's emotions. This recognized emotion information is sent to the server and reflected in further processing. For example, if a user voice-inputs "I'm looking for a good restaurant," the emotion engine will suggest the most suitable restaurant based on the information analyzed.

[0367] Specific examples

[0368] For example, when a user visits a tourist spot, they scan a QR code for a specific restaurant. The device extracts the spot ID and sends a request to the server. The server summarizes and translates the acquired restaurant information and presents it to the user. If the server determines that the user's current emotional state is stressful, it will also suggest information about relaxing spots.

[0369] Furthermore, when users search for nearby tourist attractions in the map app and select a specific spot, the device will calculate the optimal route and display it on the map. If users want to converse with local people, the device's voice input function will provide real-time translation and emotional feedback, providing a better travel experience.

[0370] Prompt Sentence Examples

[0371] "I want to see summaries of reviews of tourist spots I want to visit when I travel."

[0372] "Please suggest places where I can relax in Tokyo."

[0373] In this way, the present invention makes it possible to recognize the user's emotions, smoothly obtain information across language barriers, and optimize the travel experience.

[0374] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0375] Step 1:

[0376] Data collection and model generation

[0377] Subject: Server

[0378] The server collects information about attractions and reviews from travel sites and tourist guides using web scraping technology with Python's BeautifulSoup library. The collected data is then stored in a database such as MongoDB. The server then uses a generative AI model, such as the BERT model, to train this data and automatically summarize the information about attractions and reviews.

[0379] Input: Spot information and reviews collected from websites

[0380] Output: Automatically summarized information about places and reviews

[0381] Step 2:

[0382] Processing user requests

[0383] Subject: Server

[0384] When a request for specific spot information is received from a device, the server analyzes the request and retrieves the corresponding spot information from the database. It then summarizes it using a generative AI model and translates it into the local language using a Google Translate API or similar. The translated information is then sent back to the device.

[0385] Input: User request for spot information

[0386] Output: Translated spot information

[0387] Step 3:

[0388] Emotion engine processing

[0389] Subject: Server

[0390] The server analyzes the voice and text data received from the user through the emotion analysis module to recognize the user's emotion. This emotion information is stored in a database and taken into account in subsequent processing steps.

[0391] Input: User voice or text data

[0392] Output: Recognized user emotion information

[0393] Step 4:

[0394] Translating and summarizing reviews

[0395] Subject: Server

[0396] When a user requests review information, the server retrieves the corresponding review data from the database, translates it using an automatic translation engine, and then summarizes it using a generative AI model. The resulting information is customized based on the user's emotional information and sent to the device.

[0397] Input: User review request

[0398] Output: Translated and summarized reviews

[0399] Step 5:

[0400] Information distribution and suggestions

[0401] Subject: Server

[0402] Based on the results of the emotion analysis, the server optimizes the information on spots and activities suggested to users. For example, if the server detects that the user is feeling stressed, it will prioritize information on spots where users can relax.

[0403] Input: Recognized user emotion information

[0404] Output: Optimized spot information and recommendations

[0405] Step 6:

[0406] QR code reading

[0407] Subject: Terminal

[0408] When a user reads the QR code, the device analyzes the code data to extract the spot ID, which is then sent to the server as a request.

[0409] Input: QR code scanned by the user

[0410] Output: Spot ID sent to the server

[0411] Step 7:

[0412] Map display and navigation

[0413] Subject: Terminal

[0414] The device displays the spot information received from the server on a map and shows the optimal route from the user's current location to the spot. When the user selects a specific spot, navigation begins.

[0415] Input: Spot information received from the server

[0416] Output: Points of interest on a map and a navigation route

[0417] Step 8:

[0418] Real-time translation and emotion recognition

[0419] Subject: Terminal

[0420] The device's emotion engine analyzes the user's voice and text input and recognizes emotions. The recognized emotion information is sent to the server and reflected in processing. In addition, the user's conversation information is translated in real time and returned as text or voice.

[0421] Input: User voice or text input

[0422] Output: Recognized emotion information and translation results

[0423] This is the specific processing flow of this system. In this way, it is possible to take into account the user's feelings and provide a smooth travel experience that transcends language barriers.

[0424] (Application example 2)

[0425] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0426] Conventional guide systems have difficulty providing information that is sensitive to travelers' emotions, resulting in travel experiences that are not tailored to individual emotions and circumstances. Furthermore, there are issues with multilingual support, real-time navigation, and review summarization and translation, creating a need for more highly personalized experiences. The goal of this project is to resolve these issues and provide optimal information based on travelers' emotions, improving their experiences.

[0427] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0428] In this invention, the server includes means for automatically summarizing spot information to be presented to travelers using a generative model, means for translating the summarized spot information into a local language, means for collecting word-of-mouth information corresponding to multiple languages ​​and automatically translating and summarizing it, means for displaying specific spot information on a map in response to a user request, means for reading QR codes that allow the user to obtain detailed spot information, means for analyzing the user's emotions and suggesting optimal spot information and activities in accordance with the emotions, and means for personalizing and suggesting specific dishes and menus based on the user's emotion data. This makes it possible to provide personalized information in accordance with the travelers' emotions and improve their experience.

[0429] A "generative model" is an algorithm that automatically summarizes and translates collected data using natural language processing technology.

[0430] "Spot information" is detailed information about tourist destinations and activities, and is data about places that travelers visit.

[0431] A "local language" is a language commonly spoken in the region or country the traveler is visiting.

[0432] "Reviews" are reviews and opinions posted by other travelers about a particular place or activity.

[0433] To "summarize" means to briefly summarize the main points or content of information.

[0434] "Translating" means converting information written in one language into another language.

[0435] A "user request" is a request or inquiry from a traveler to the system for specific information.

[0436] "Displaying on a map" means visually showing a specific spot using geographical location information.

[0437] A "QR code reading means" is a function that scans a QR code using a smartphone or dedicated device and analyzes its contents.

[0438] "Analyzing user emotions" means assessing the emotions and psychological state of the traveler at that time based on voice and text data obtained from the traveler.

[0439] "Suggesting the best spot information and activities" means guiding users to the most suitable tourist spots and experiences based on their emotions and situation.

[0440] "Personalization" means tailoring the information and services provided to suit the preferences and feelings of individual users.

[0441] The present invention provides a guide system for assisting travelers in obtaining spot information and optimizing their travel experience. Specific embodiments of the system will be described below.

[0442] Server Processing

[0443] Data collection and model generation

[0444] The server periodically collects information about attractions and reviews from travel sites and tourist guides, and stores this data in a database. A generative model (such as OpenAI's GPT-4) is trained using natural language processing technology to automatically summarize the information about attractions and reviews.

[0445] Processing user requests

[0446] When a request for spot information is received from a user device, the server analyzes the request, retrieves the relevant spot information from the database, and summarizes and translates it using a generative model, allowing the user to obtain concise and easy-to-understand information.

[0447] Emotion engine processing

[0448] The server-based emotion engine (such as Google Cloud's Dialogflow) analyzes the voice and text data received from the user and recognizes the user's emotions. This emotion information is then taken into account in other processing steps to suggest the most suitable spots and activities for the user.

[0449] Translating and summarizing reviews

[0450] When a user requests review information, the server retrieves the review data from the database, translates it using an automatic translation engine (such as DeepL), and then summarizes it using a generative model.The server then customizes the content by reflecting the user's emotional information and sends it to the user's device.

[0451] Information distribution and suggestions

[0452] The emotion engine recognizes the user's emotions and optimizes the recommendations for spots and activities. Also, if the user is under stress, the system prioritizes the display of information about spots where they can relax. Furthermore, the system personalizes and suggests specific dishes and menus based on the user's emotional data.

[0453] Terminal handling

[0454] QR code reading

[0455] When a user reads the QR code, the device analyzes the code data and extracts the spot ID, which is then sent to the server as a request.

[0456] Map display and navigation

[0457] The device displays the spot information received from the server on a map and displays the optimal route from the user's current location to the spot. When the user selects a specific spot, the device begins navigation based on the selection.

[0458] Real-time translation and emotion recognition

[0459] The device receives information about spots and reviews from the server and displays them in the local language, and uses an emotion engine to recognize emotions based on the user's voice and text input. The recognized emotion information is sent to the server and reflected in processing.

[0460] Specific examples

[0461] For example, when a user visits a tourist spot, they scan a QR code for a specific restaurant. The device extracts the spot ID and sends a request to the server. The server summarizes and translates the acquired restaurant information and presents it to the user. If the server determines that the user's current emotions are stressful, it will also suggest information about relaxing spots. Specific dishes and menus can also be personalized and suggested based on the user's emotional data.

[0462] Furthermore, when users search for nearby tourist attractions in the map app and select a specific spot, the device will calculate the optimal route and display it on the map. If users want to converse with local people, real-time translation is provided through voice input. Voice feedback based on emotions is also provided, providing a better travel experience.

[0463] An example of a prompt might be:

[0464] I'm feeling very tired today. Can you recommend something to calm me down?

[0465] In this way, the present invention makes it possible to recognize the user's emotions, smoothly obtain information across language barriers, and optimize the travel experience.

[0466] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0467] Step 1:

[0468] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides.

[0469] Input: Data from travel sites and tourist guides

[0470] Data processing: continually storing new information in the database

[0471] Output: The latest spot information and reviews are stored in the database.

[0472] Step 2:

[0473] The server uses a generative model (e.g., GPT-4) to train the collected spot information and reviews, generating a model with automatic summarization and translation capabilities.

[0474] Input: Spot information and reviews stored in the database

[0475] Data Computing: Training with Natural Language Processing Techniques

[0476] Output: A generative model that automatically summarizes and translates

[0477] Step 3:

[0478] A request for spot information is sent from the user terminal to the server.

[0479] Input: User request for spot information

[0480] Data processing: Analysis of request content

[0481] Output: Search criteria for spot information based on the request

[0482] Step 4:

[0483] The server retrieves the relevant spot information from the database and summarizes and translates it using a generative model.

[0484] Input: Search criteria for spot information based on your request

[0485] Data calculation: Summarization and translation

[0486] Output: Summary and translated spot information

[0487] Step 5:

[0488] The server analyzes the voice and text data received from the user terminal using an emotion engine (e.g., Dialogflow) to recognize the user's emotions.

[0489] Input: Voice or text input from the user

[0490] Data Computing: Emotion Recognition Processing

[0491] Output: User's emotional information

[0492] Step 6:

[0493] The server suggests optimal spot information and activities based on the user's emotional information recognized by the emotion engine. If the user is experiencing a certain type of stress, the server will prioritize suggesting information about relaxing spots.

[0494] Input: User's emotional information

[0495] Data Computation: Customized Information Suggestion Based on Emotional Information

[0496] Output: Personalized spot suggestions

[0497] Step 7:

[0498] The server retrieves the user-requested review information from the database, translates it using an automatic translation engine (e.g., DeepL), and then summarizes it using a generative model.

[0499] Input: User review request

[0500] Data Computing: Automatic Translation and Summarization

[0501] Output: Summary and translated reviews

[0502] Step 8:

[0503] The terminal reads the QR code, analyzes the code data to extract the spot ID, and sends it to the server.

[0504] Input: Spot QR code

[0505] Data processing: QR code analysis and spot ID extraction

[0506] Output: Spot ID

[0507] Step 9:

[0508] The terminal displays the spot information received from the server on a map and displays the optimal route from the user's current location to the spot.

[0509] Input: Spot information from the server

[0510] Data calculation: Calculating the optimal route on map data

[0511] Output: Navigation information on a map

[0512] Step 10:

[0513] The device displays the spot information and reviews received from the server in the local language and uses an emotion engine to recognize emotions based on the user's voice and text input. The recognized emotion information is sent to the server and reflected in processing.

[0514] Input: Voice input or text input from the user, spot information and reviews from the server

[0515] Data processing: Real-time translation and emotion recognition

[0516] Output: Display information in local language, send emotion information to server

[0517] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0518] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0519] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0520] [Second embodiment]

[0521] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0522] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0523] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0524] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0525] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0526] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0527] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0528] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0529] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0530] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0531] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0532] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0533] The present invention relates to a guide system for assisting travelers in obtaining spot information. Specific examples will be described below.

[0534] Server Processing

[0535] Data collection and model generation

[0536] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides. This data is stored in a database and used as training data for the generative model. The generative model is trained using natural language processing techniques to automatically summarize the collected information about tourist spots and convert it into an easy-to-understand text format.

[0537] Processing user requests

[0538] When a request is received from a device, the server analyzes the request and retrieves the relevant spot information from the database, which is then summarized by a generative model and automatically translated into the local language.

[0539] Translating and summarizing reviews

[0540] When a user requests review information, the server retrieves the relevant reviews from the database, translates them into major international languages ​​using an automatic translation engine, and then summarizes them using a generative model and provides them to the user.

[0541] Information distribution

[0542] The server then sends the translated and summarized information about the spots and reviews to the device, allowing users to easily understand the local information.

[0543] Terminal handling

[0544] QR code reading

[0545] When a user scans a QR code at a specific tourist spot, the device analyzes the code information, extracts the spot ID, and sends a request to the server. Once the information is received from the server, the device displays it in the GUI.

[0546] Map display and navigation

[0547] The device displays spot information from the server on a map and uses GPS information to provide navigation from the user's current location to the spot. When the user selects a specific spot, the device calculates the optimal route and displays it on the map.

[0548] Real-time translation

[0549] The device displays spot information and reviews received from the server in the local language. When the user converses with a local person, the device accepts voice or text input and translates the content using a real-time translation function. The translation results are immediately provided to the user.

[0550] Specific examples

[0551] For example, when a user visits a tourist spot in Japan, they scan the QR code of a specific restaurant. The device extracts the spot ID and sends a request to the server. The server retrieves the restaurant's information from the database, summarizes it using a generative model, translates it into the local language, and sends it. The device displays this information, and the user can also view reviews from other tourists.

[0552] In addition, users can use the map app to search for nearby tourist attractions and restaurants and select points of interest on the map. After selection, the device will display the optimal route and begin navigation.

[0553] In this way, the guide system according to the present invention enables travelers to smoothly obtain information across language barriers and enrich their local experiences.

[0554] The processing flow will be explained below.

[0555] Server Processing

[0556] Step 1:

[0557] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, and stores the collected data in a database.

[0558] Step 2:

[0559] The server uses the information stored in the database to train a generative model, which then uses natural language processing techniques to extract important keywords and context and acquire the ability to summarize information.

[0560] Step 3:

[0561] When a request for spot information is received from a terminal, the server analyzes the request and retrieves the relevant spot information from the database based on the analysis results.

[0562] Step 4:

[0563] The server passes the acquired spot information to a generative model, which summarizes and translates it into the local language, and then sends the results to the device.

[0564] Step 5:

[0565] When a user requests review information, the server retrieves the relevant review data from the database, then translates it into major international languages ​​using an automatic translation engine and summarizes it using a generative model. The translated and summarized review information is then sent to the device.

[0566] Terminal handling

[0567] Step 1:

[0568] The user reads the QR code, and the device analyzes the data and extracts the spot ID.

[0569] Step 2:

[0570] The device sends a request for spot information to the server based on the analyzed spot ID.

[0571] Step 3:

[0572] The spot information received from the server is displayed on the GUI, allowing users to check the spot information summarized in the local language.

[0573] Step 4:

[0574] The user opens a map app and searches for a specific spot. The device acquires the user's current location via GPS and displays candidate spots received from the server on the map.

[0575] Step 5:

[0576] When the user selects a specific spot, the device calculates the optimal route and displays route guidance on the map.

[0577] User operation example

[0578] Step 1:

[0579] Users find a QR code at a tourist spot and when they scan the code with their device, a request for information about the spot is automatically sent to the server.

[0580] Step 2:

[0581] The spot information received from the server is displayed on the terminal, and the user can check detailed restaurant information in the local language.

[0582] Step 3:

[0583] When a user taps the review tab, the device sends a request for review information to the server, and the translated and summarized review information received from the server is displayed on the device.

[0584] Step 4:

[0585] When a user searches for nearby tourist spots using a map app, multiple options are displayed on the map. When the user selects a specific spot, the device displays the optimal route and begins navigation.

[0586] Step 5:

[0587] When a user speaks to a local person, they input the content of the conversation using the device's voice input function. The translation is done in real time and the results are displayed on the screen.

[0588] Example 1

[0589] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0590] There is a need for a system that allows travelers to efficiently obtain information about places they are visiting and utilize local information without experiencing language barriers. However, existing systems lack sufficient timely summarization and translation of geographical feature information and review information, and are unable to fully support multiple languages. Furthermore, they are limited in terms of navigation functions that allow travelers to instantly obtain the optimal route from their current location to their destination, and functions that support real-time communication with local people. This can make it difficult for travelers to navigate the local area.

[0591] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0592] In this invention, the server includes a means for automatically summarizing geographical feature information to be presented to travelers using a machine learning model, a means for translating the summarized geographical feature information into a local language, and a means for collecting review information corresponding to multiple languages ​​and automatically translating and summarizing the information. This allows travelers to quickly and appropriately obtain the necessary geographical feature information without experiencing language barriers. Furthermore, the server includes a code reading means for displaying specific geographical feature information on a map in response to a user's request and allowing the user to obtain detailed information about the geographical feature, thereby significantly improving user convenience.

[0593] A "traveler" is someone who travels to different locations for tourism, work, or other purposes and needs information and services related to those locations.

[0594] "Geographical feature information" refers to data that indicates the location and characteristics of tourist attractions, facilities, natural landscapes, etc., as well as related information.

[0595] A "guidance system" refers to a system that assists users in efficiently obtaining the information they desire.

[0596] A "machine learning model" refers to an algorithm that is trained using large amounts of data to automatically perform a specific task (e.g., summarizing or translating text).

[0597] "Automatic summarization" refers to technology that extracts the main points from long textual information and organizes them in a short, easy-to-understand format.

[0598] "Means of translation" refers to the technology that converts information written in one language into another language.

[0599] "Review information" refers to feedback data in which users describe their experiences and evaluations.

[0600] "Means for displaying geographic feature information" refers to a method for visually displaying the collected and processed geographic feature information on a user's terminal.

[0601] "Code reading means" refers to technology that reads coded information such as QR codes and barcodes.

[0602] "Location information" refers to data that indicates the latitude and longitude of a specific location using technology such as GPS.

[0603] "Route guidance display function" refers to technology that calculates the optimal route from the current location to the destination and visually shows that route to the user.

[0604] "Voice or text input" refers to a method in which a user speaks to a terminal or inputs text.

[0605] "Means capable of real-time translation" refers to technology that instantly converts input speech or text into another language and quickly provides it to the user.

[0606] The present invention relates to a guidance system that enables travelers to efficiently obtain information about places they are visiting and to utilize local information without feeling the language barrier.

[0607] Server Processing

[0608] The server periodically crawls geographic feature information and review information from multiple travel sites and tourist guides. The collected data is stored in a database such as MongoDB. The collected data is then analyzed using a generative AI model (e.g., GPT-4), extracting and summarizing the necessary information. The summarized information is formatted as text and presented to users in an easy-to-understand format. The generative model is periodically retrained with training data to improve its accuracy.

[0609] Examples:

[0610] For example, information is collected from "travel review site A" and "tourist guide site B." This data is stored in MongoDB, analyzed using Python, and summarized using GPT-4.

[0611] Terminal handling

[0612] The device accepts input from the user (e.g., reading a QR code or selecting a spot). When the user scans a QR code at a specific tourist attraction, the device analyzes the code information to extract the spot ID and sends a request to the server. The information received from the server is displayed on the device's GUI. Furthermore, the device uses GPS information to identify the user's current location and uses the Google Maps API to display geographical feature information on a map. Route guidance is also provided to the user in real time via voice and text.

[0613] Examples:

[0614] For example, a user visits a tourist spot in Japan and scans a QR code for a specific restaurant. The device extracts the spot ID and sends a request to the server. The server retrieves the restaurant's information from the database, summarizes it using a generative AI model, translates it into the local language, and sends it to the device. The device displays this information, and the user can also view reviews from other travelers.

[0615] Processing reviews

[0616] The server receives a user request for review information. The server retrieves the relevant review information from the database and translates it into major international languages ​​using an automatic translation engine (e.g., Google Translate API). The generative AI model then summarizes the review information and provides it to the user.

[0617] Examples:

[0618] When a user requests to view reviews for a specific restaurant, the server retrieves multiple reviews from the database, translates them using the Google Translate API, summarizes them using GPT-4, and sends them to the device.

[0619] Real-time translation

[0620] The device accepts voice or text input from the user and uses real-time translation to translate the input into the specified language, providing the translation results instantly to the user.

[0621] Examples:

[0622] When a user wants to converse with a local person, they speak into the device. The speech recognition function converts the speech into text, which is then translated by the real-time translation engine and displayed or spoken back to the user.

[0623] Prompt Sentence Examples

[0624] "Please summarize recent reviews of this restaurant in Japanese."

[0625] "Please summarize the tourist information about Tokyo Tower and translate it into English."

[0626] Using this system, travelers can easily obtain detailed local information and enjoy a rich experience, regardless of language barriers.

[0627] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0628] The flow of this system's program processing

[0629] Step 1: Data collection and storage

[0630] explanation:

[0631] The server periodically crawls geographical feature information and review information from multiple travel sites and tourist guides.

[0632] input:

[0633] URLs for travel sites and tourist guides.

[0634] Specific behavior:

[0635] The server uses a crawler to gather the required information from the specified URLs.

[0636] output:

[0637] Collected data (geographical feature information and review information).

[0638] Step 2: Save to the database

[0639] explanation:

[0640] The server stores the collected data in a database.

[0641] input:

[0642] Geographical feature information and review information collected through crawling.

[0643] Specific behavior:

[0644] The server performs operations to insert data into a database such as MongoDB.

[0645] output:

[0646] Geographic feature information and review information stored in a database.

[0647] Step 3: Analyze and summarize data using the model

[0648] explanation:

[0649] The server uses a generative AI model (e.g., GPT-4) to analyze and summarize the collected data.

[0650] input:

[0651] Geographic feature information and review information retrieved from databases.

[0652] Specific behavior:

[0653] The server uses a programming language such as Python to analyze the data using the GPT-4 model, extracting and summarizing key points.

[0654] output:

[0655] Summarized geographic feature and review information.

[0656] Step 4: Translation

[0657] explanation:

[0658] The server translates the summarized information into the local language.

[0659] input:

[0660] Summarized geographic feature and review information.

[0661] Specific behavior:

[0662] The server uses a translation API (eg, Google Translate API) to translate the summary information into other languages.

[0663] output:

[0664] Geographical feature information and review information translated into local languages.

[0665] Step 5: Receiving a user request

[0666] explanation:

[0667] The terminal receives a request from a user.

[0668] input:

[0669] Scan QR codes and select specific spots.

[0670] Specific behavior:

[0671] The device uses a QR code reader to extract the code information or obtain the user's selection.

[0672] output:

[0673] The analyzed spot ID or request content.

[0674] Step 6: Parse the request and get information

[0675] explanation:

[0676] The server receives the request from the terminal, analyzes it, and retrieves the relevant information from the database.

[0677] input:

[0678] The spot ID or request content received from the device.

[0679] Specific behavior:

[0680] The server analyzes the request and issues the appropriate query to the database.

[0681] output:

[0682] Geographic feature information and review information retrieved from databases.

[0683] Step 7: Summarize and translate the information

[0684] explanation:

[0685] The server summarizes the acquired information using a generative AI model and translates it into the local language.

[0686] input:

[0687] Obtained geographic feature information and review information.

[0688] Specific behavior:

[0689] The server summarizes the information using the GPT-4 model and translates it into the local language using a translation API.

[0690] output:

[0691] Summarized and translated geographic feature and review information.

[0692] Step 8: Distributing information

[0693] explanation:

[0694] The server transmits the translated and summarized information to the terminal.

[0695] input:

[0696] Summarized and translated geographic feature and review information.

[0697] Specific behavior:

[0698] The server uses HTTP requests to send information to the device.

[0699] output:

[0700] Information delivered to your device.

[0701] Step 9: View information

[0702] explanation:

[0703] The terminal displays the received information in a GUI.

[0704] input:

[0705] The geographic feature information and review information received from the server.

[0706] Specific behavior:

[0707] Use the device's UI components to visually display information.

[0708] output:

[0709] Information visually displayed to the user.

[0710] Step 10: Real-time translation

[0711] explanation:

[0712] The device accepts and translates the user's voice or text input.

[0713] input:

[0714] User voice or text input.

[0715] Specific behavior:

[0716] The device uses a voice recognition system to convert speech into text, which is then translated using a real-time translation engine.

[0717] output:

[0718] Translated text or audio output.

[0719] By following each processing step of this system, travelers can efficiently obtain local information and gain a rich experience beyond language barriers.

[0720] (Application example 1)

[0721] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0722] Modern travelers want to quickly and efficiently obtain information about tourist attractions and restaurants in their destinations. However, language barriers and the complexity of information present obstacles. Furthermore, with advances in virtual reality technology, more and more people are seeking realistic travel experiences in virtual spaces without actually visiting the destinations. However, there is a lack of systems that support such experiences. There is a need for a system that can resolve these issues and enable users to smoothly enjoy both real and virtual travel experiences.

[0723] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0724] In this invention, the server includes means for automatically summarizing spot information to be presented to travelers using a generative model, means for translating the summarized spot information into a local language, means for collecting word-of-mouth information corresponding to multiple languages ​​and automatically translating and summarizing it, means for displaying information about specific tourist spots in a virtual space on a virtual screen, and navigation means for virtually exploring the local area based on the displayed information. This allows users to quickly and efficiently obtain spot information across language barriers and experience exploring the local area in a virtual space.

[0725] A "generative model" is a machine learning model that automatically summarizes and translates based on collected data.

[0726] "Spot information" refers to detailed data about places to visit, such as tourist attractions and restaurants.

[0727] A "local language" is a language commonly spoken in a given region.

[0728] "Word-of-mouth information" refers to the evaluations and impressions provided by users about specific spots.

[0729] A QR code is a two-dimensional barcode with embedded information, which can be obtained by scanning it with a device.

[0730] "Virtual space" is a virtual environment or world created by a computer.

[0731] "Virtual screen" refers to digital information displayed on a head-mounted display or smart glasses.

[0732] "Navigation means" refers to technologies and functions that assist users in understanding directions and routes to their destinations.

[0733] To implement this invention, the server, the terminal (such as smart glasses or a head-mounted display (HMD)), and the user need to cooperate with each other.

[0734] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides. This data is stored in a database and used as training data for the generative model. The generative model uses natural language processing technology to automatically summarize the collected information about tourist spots and convert it into an easy-to-understand text format. The server also analyzes user requests, retrieves relevant information about tourist spots from the database, and uses the generative model to summarize and automatically translate it into the local language.

[0735] When a user scans a QR code at a specific tourist spot, the device analyzes the code information, extracts the spot ID, and sends a request to the server. Once the information is received from the server, the device displays it on the GUI. The device also uses GPS information to display spot information on a map and provides navigation functions. When a user selects a specific spot, the device calculates the optimal route and displays it on the map. In addition, the device displays spot information and reviews received from the server in the local language and uses a real-time translation function to translate conversations with local people.

[0736] In a virtual space implementation, a device reads a QR code in the virtual space and displays information about a specific tourist spot on a virtual screen. A navigation function is also provided to virtually explore the location based on the displayed information. This allows users to enjoy a realistic travel experience in the virtual space without actually visiting the location.

[0737] Hardware and software used

[0738] Hardware: Server, smart glasses, head-mounted display, GPS receiver

[0739] Software: Natural language processing technology, generative models, databases, real-time translation engines

[0740] The server is built using Python. The program retrieves information about spots and reviews via API and summarizes and translates them using a generative AI model. Qt is used to create a GUI on the device, allowing users to smoothly retrieve information and navigate.

[0741] Specific examples

[0742] For example, when a user scans a specific QR code in the virtual space, detailed information about that tourist spot is instantly displayed. The user can check information such as "You can see a traditional festival at this spot" on the virtual screen. Also, when the user inputs "Take a virtual trip to this spot," the device starts virtual navigation and shows the user the optimal route.

[0743] Prompt Sentence Examples

[0744] "Get information about specific tourist spots and display it as a virtual guide."

[0745] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0746] Step 1:

[0747] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides. The input is data obtained through web scraping and API calls, and the output is organized data stored in a database. This data is later used as training data for generative models. Specifically, it processes data collection from websites using Python's BeautifulSoup and Requests.

[0748] Step 2:

[0749] The server uses the collected data to train a generative AI model. The input is the data collected in step 1, and the output is a model that performs summarization and translation. Specifically, the generative AI model is trained using TensorFlow and PyTorch.

[0750] Step 3:

[0751] The user scans a specific QR code on their device. The input is the image data of the QR code, and the output is the analyzed spot ID. Specifically, the device's camera captures the QR code and decodes it using a library such as OpenCV.

[0752] Step 4:

[0753] After extracting the spot ID, the device sends the ID to the server. The input is the spot ID, and the output is a request to the server. Specifically, it uses the Python Requests library to send the HTTP request.

[0754] Step 5:

[0755] The server retrieves the corresponding spot information from the database based on the received spot ID. The input is the spot ID and the output is the spot information. Specifically, it executes an SQL query to retrieve the information from the database.

[0756] Step 6:

[0757] The server summarizes the acquired spot information using a generative AI model and automatically translates it into the local language. The input is the spot information, and the output is the summarized and translated information. Specifically, data is input into a pre-trained generative AI model, and summary and translation results are obtained.

[0758] Step 7:

[0759] The server sends summarized and translated spot information to the terminal. The input is the summarized and translated information, and the output is the response to the terminal. Specifically, the information is returned as an HTTP response.

[0760] Step 8:

[0761] The terminal displays the received information in a GUI. The input is summarized and translated spot information, and the output is a visual display for the user. Specifically, the information is displayed using a GUI framework such as Qt.

[0762] Step 9:

[0763] The device uses GPS information to display spot information on a map and provide navigation. The input is the current location's GPS data, and the output is a navigation route on the map. Specifically, it calculates the route using Google Maps API and displays it on the map.

[0764] Step 10:

[0765] When a user speaks or inputs text, the device translates it in real time. The input is the user's voice or text, and the output is the translated text or voice. Specifically, the device performs text translation using the Google Translate API or similar.

[0766] Step 11:

[0767] In the virtual space, the device reads the QR code and displays information about a specific tourist spot on a virtual screen. The input is the QR code data, and the output is the spot information displayed on the virtual screen. Specific operations utilize QR code scanning technology and a virtual reality framework.

[0768] Step 12:

[0769] The device supports users in navigating within the virtual space based on the displayed information. The input is spot information and user operations, and the output is a navigation route within the virtual space. Specifically, the navigation system within the virtual space is implemented using an engine such as Unity.

[0770] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0771] The present invention relates to a guide system that recognizes the emotions of travelers and combines an emotion engine to optimize the travel experience. Specific examples will be described below.

[0772] Server Processing

[0773] Data collection and model generation

[0774] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, stores this data in a database, and trains the generative model using natural language processing technology to automatically summarize the information about tourist spots and reviews.

[0775] Processing user requests

[0776] When a request for spot information is received from a device, the server analyzes the request, retrieves the relevant spot information from the database, and summarizes and translates it using a generative model.

[0777] Emotion engine processing

[0778] The emotion engine embedded in the server analyzes the voice and text data received from the user to recognize the user's emotion, which is then taken into account in other processing steps.

[0779] Translating and summarizing reviews

[0780] When a user requests review information, the server retrieves the review data from the database, translates it using an automatic translation engine, and then summarizes it using a generative model.The server then customizes the content by reflecting the user's emotional information and sends it to the device.

[0781] Information distribution and suggestions

[0782] The emotion engine recognizes the user's emotions and optimizes the suggested spots and activities. If the user is under stress, the system prioritizes the display of information about relaxing spots.

[0783] Terminal handling

[0784] QR code reading

[0785] When a user reads the QR code, the device analyzes the code data and extracts the spot ID, which is then sent to the server as a request.

[0786] Map display and navigation

[0787] The device displays the spot information received from the server on a map and displays the optimal route from the user's current location to the spot. When the user selects a specific spot, the device begins navigation based on the selection.

[0788] Real-time translation and emotion recognition

[0789] The device receives information about spots and reviews from the server and displays them in the local language, and uses an emotion engine to recognize emotions based on the user's voice and text input. The recognized emotion information is sent to the server and reflected in processing.

[0790] Specific examples

[0791] For example, when a user visits a tourist spot, they can scan the QR code of a specific restaurant. The device extracts the spot ID and sends a request to the server. The server summarizes and translates the acquired restaurant information and presents it to the user. If the server determines that the user's current emotional state is stressful, it will also suggest information about relaxing spots.

[0792] Furthermore, when users search for nearby tourist attractions in the map app and select a specific spot, the device will calculate the optimal route and display it on the map. If users want to converse with local people, real-time translation is provided through voice input. Voice feedback based on emotions is also provided, providing a better travel experience.

[0793] In this way, the present invention makes it possible to recognize the user's emotions, smoothly obtain information across language barriers, and optimize the travel experience.

[0794] The processing flow will be explained below.

[0795] Server Processing

[0796] Step 1:

[0797] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, and stores the collected data in a database.

[0798] Step 2:

[0799] The server uses the information stored in the database to train a generative model, which uses natural language processing techniques to automatically summarize information about spots and reviews.

[0800] Step 3:

[0801] When a request for spot information is received from a terminal, the server analyzes the request and retrieves the corresponding spot information from the database.

[0802] Step 4:

[0803] The server passes the acquired spot information to the generative model, which summarizes and translates it into the local language. The translated and summarized information is then sent to the device.

[0804] Step 5:

[0805] When a user submits a review request, the server retrieves the review data from the database and translates it into major international languages ​​using an automatic translation engine.Then, a generative model summarizes the review information and customizes it based on the user's sentiment information.

[0806] Step 6:

[0807] The emotion engine built into the server analyzes the voice and text data received from the user and recognizes the user's emotions. The recognized emotion information is reflected in information processing and presentation.

[0808] Step 7:

[0809] Based on the user's emotions recognized by the emotion engine, the server optimizes and suggests information on spots and activities suitable for the traveler. Also, if the user is feeling stressed, it will prioritize providing information on spots where they can relax.

[0810] Terminal handling

[0811] Step 1:

[0812] The user scans the QR code at a tourist spot, and the device analyzes the QR code data to extract the spot ID.

[0813] Step 2:

[0814] The terminal sends a request for spot information to the server based on the analyzed spot ID.

[0815] Step 3:

[0816] The spot information received from the server is displayed in the local language on the terminal's GUI, and the user can check the displayed spot information.

[0817] Step 4:

[0818] The user opens a map app and searches for a specific spot. The device acquires the user's current location via GPS and displays the spot information received from the server on a map.

[0819] Step 5:

[0820] When the user selects a specific spot, the device calculates the optimal route and displays route guidance on the map.

[0821] Step 6:

[0822] The terminal displays the reviews received from the server to the user in the local language, with the reviews automatically translated and summarized.

[0823] Step 7:

[0824] When the user expresses their emotions through voice or text input, the device sends this input data to the server.

[0825] Step 8:

[0826] Based on the emotional feedback received from the server in real time, the device will suggest spot information and activities that correspond to the user's emotions.

[0827] User operation example

[0828] Step 1:

[0829] The user scans the QR code at a tourist spot and sends a request to the server to display the information on the device.

[0830] Step 2:

[0831] The server summarizes the spot information using a generative model, translates it, and sends it to the device, where the user can view the spot information in their local language.

[0832] Step 3:

[0833] If the user wants to check the reviews, they tap the reviews tab to send a request. The server translates and summarizes the reviews and sends them to the device.

[0834] Step 4:

[0835] Users use a map app to search for nearby tourist spots and select a specific spot, and the device calculates the optimal route and displays navigation.

[0836] Step 5:

[0837] When users converse with local people, they use the voice input function. The device sends the input speech to the server and displays the translation results in real time. The emotion engine also analyzes the user's emotions and provides customized feedback based on the results.

[0838] Example 2

[0839] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0840] Conventional guide systems do not adequately consider traveler emotions or language barriers, resulting in suboptimal travel experiences. Travelers often find themselves in situations where they feel stressed or are unable to obtain sufficient information due to not understanding the local language, resulting in lower travel satisfaction.

[0841] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0842] In this invention, the server includes means for automatically summarizing spot information to be presented to travelers using a generative model, means for translating the summarized spot information into a local language, means for collecting word-of-mouth information corresponding to multiple languages ​​and automatically translating and summarizing it, emotion analysis means for recognizing emotions from a user's voice or text input, and suggestion means for optimizing the spot information and suggestions based on the emotion analysis results. This enables travelers to smoothly obtain information across language barriers and receive optimal suggestions according to their emotions, thereby improving satisfaction with their travel experience.

[0843] A "generative model" is a machine learning algorithm that automatically summarizes information based on collected data.

[0844] A "local language" is a language commonly spoken in the area the traveller is visiting.

[0845] "Word-of-mouth information" refers to data in which users write their ratings and opinions about certain spots or services.

[0846] "Machine translation" refers to the technology of automatically converting text between multiple languages.

[0847] "Emotion analysis" is a technology that analyzes a user's voice and text data and recognizes their emotions.

[0848] "Suggestion means" refers to a function that suggests optimal spot information and activities to users based on the results of emotion analysis.

[0849] "Code reading" refers to the technology of reading code information such as QR codes using a device's camera.

[0850] "Navigation" refers to a function that provides guidance on a route from the user's current location to a destination.

[0851] A "user request" is a request for information made by a user to a server via a terminal.

[0852] A "database" is a system that systematically stores and manages collected spot information and reviews.

[0853] The present invention relates to a guide system for recognizing a traveler's emotions and optimizing the travel experience. Specific examples will be described below.

[0854] Server Processing

[0855] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, and stores the data in a database. This is done using web scraping technology, such as Python's BeautifulSoup library. The collected data is then stored in a database such as MongoDB.

[0856] The data is then trained using a generative AI model with natural language processing techniques (e.g., the BERT model) to automatically summarize information about attractions and reviews, ensuring that the information provided to travelers is concise and easy to understand.

[0857] When a request for specific spot information is received from a device, the server analyzes the request and retrieves the corresponding spot information from the database, summarizes this information using a generative AI model, and then translates it into the local language using tools such as Google Translate API.

[0858] The server is also equipped with an emotion engine that analyzes voice and text data received from users. For example, if a user says, "I've been feeling a bit stressed lately," the emotion analysis module analyzes this information and recognizes the user's emotion. This emotion information is used in subsequent processing steps.

[0859] When a user requests review information, the server retrieves the information from the database, translates it using an automatic translation engine, and summarizes it using a generative AI model. This summarized information is then customized to reflect the user's emotional information and sent to the device.

[0860] Based on the user's emotions recognized by the emotion engine, the server optimizes the recommendations of spot information and activities. For example, if the server recognizes that the user is under stress, it will prioritize providing information on relaxation spots.

[0861] Terminal handling

[0862] When a user reads the QR code, the device analyzes the code data to extract the spot ID and sends it as a request to the server. The spot information sent from the server is displayed on the device's map app, and the optimal route from the user's current location to the spot is shown.

[0863] The device's emotion engine analyzes the user's voice and text inputs to recognize the user's emotions. This recognized emotion information is sent to the server and reflected in further processing. For example, if a user voice-inputs "I'm looking for a good restaurant," the emotion engine will suggest the most suitable restaurant based on the information analyzed.

[0864] Specific examples

[0865] For example, when a user visits a tourist spot, they scan a QR code for a specific restaurant. The device extracts the spot ID and sends a request to the server. The server summarizes and translates the acquired restaurant information and presents it to the user. If the server determines that the user's current emotional state is stressful, it will also suggest information about relaxing spots.

[0866] Furthermore, when users search for nearby tourist attractions in the map app and select a specific spot, the device will calculate the optimal route and display it on the map. If users want to converse with local people, the device's voice input function will provide real-time translation and emotional feedback, providing a better travel experience.

[0867] Prompt Sentence Examples

[0868] "I want to see summaries of reviews of tourist spots I want to visit when I travel."

[0869] "Please suggest places where I can relax in Tokyo."

[0870] In this way, the present invention makes it possible to recognize the user's emotions, smoothly obtain information across language barriers, and optimize the travel experience.

[0871] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0872] Step 1:

[0873] Data collection and model generation

[0874] Subject: Server

[0875] The server collects information about attractions and reviews from travel sites and tourist guides using web scraping technology with Python's BeautifulSoup library. The collected data is then stored in a database such as MongoDB. The server then uses a generative AI model, such as the BERT model, to train this data and automatically summarize the information about attractions and reviews.

[0876] Input: Spot information and reviews collected from websites

[0877] Output: Automatically summarized information about places and reviews

[0878] Step 2:

[0879] Processing user requests

[0880] Subject: Server

[0881] When a request for specific spot information is received from a device, the server analyzes the request and retrieves the corresponding spot information from the database. It then summarizes it using a generative AI model and translates it into the local language using a Google Translate API or similar. The translated information is then sent back to the device.

[0882] Input: User request for spot information

[0883] Output: Translated spot information

[0884] Step 3:

[0885] Emotion engine processing

[0886] Subject: Server

[0887] The server analyzes the voice and text data received from the user through the emotion analysis module to recognize the user's emotion. This emotion information is stored in a database and taken into account in subsequent processing steps.

[0888] Input: User voice or text data

[0889] Output: Recognized user emotion information

[0890] Step 4:

[0891] Translating and summarizing reviews

[0892] Subject: Server

[0893] When a user requests review information, the server retrieves the corresponding review data from the database, translates it using an automatic translation engine, and then summarizes it using a generative AI model. The resulting information is customized based on the user's emotional information and sent to the device.

[0894] Input: User review request

[0895] Output: Translated and summarized reviews

[0896] Step 5:

[0897] Information distribution and suggestions

[0898] Subject: Server

[0899] Based on the results of the emotion analysis, the server optimizes the information on spots and activities suggested to users. For example, if the server detects that the user is feeling stressed, it will prioritize information on spots where users can relax.

[0900] Input: Recognized user emotion information

[0901] Output: Optimized spot information and recommendations

[0902] Step 6:

[0903] QR code reading

[0904] Subject: Terminal

[0905] When a user reads the QR code, the device analyzes the code data to extract the spot ID, which is then sent to the server as a request.

[0906] Input: QR code scanned by the user

[0907] Output: Spot ID sent to the server

[0908] Step 7:

[0909] Map display and navigation

[0910] Subject: Terminal

[0911] The device displays the spot information received from the server on a map and shows the optimal route from the user's current location to the spot. When the user selects a specific spot, navigation begins.

[0912] Input: Spot information received from the server

[0913] Output: Points of interest on a map and a navigation route

[0914] Step 8:

[0915] Real-time translation and emotion recognition

[0916] Subject: Terminal

[0917] The device's emotion engine analyzes the user's voice and text input and recognizes emotions. The recognized emotion information is sent to the server and reflected in processing. In addition, the user's conversation information is translated in real time and returned as text or voice.

[0918] Input: User voice or text input

[0919] Output: Recognized emotion information and translation results

[0920] This is the specific processing flow of this system. In this way, it is possible to take into account the user's feelings and provide a smooth travel experience that transcends language barriers.

[0921] (Application example 2)

[0922] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0923] Conventional guide systems have difficulty providing information that is sensitive to travelers' emotions, resulting in travel experiences that are not tailored to individual emotions and circumstances. Furthermore, there are issues with multilingual support, real-time navigation, and review summarization and translation, creating a need for more highly personalized experiences. The goal of this project is to resolve these issues and provide optimal information based on travelers' emotions, improving their experiences.

[0924] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0925] In this invention, the server includes means for automatically summarizing spot information to be presented to travelers using a generative model, means for translating the summarized spot information into a local language, means for collecting word-of-mouth information corresponding to multiple languages ​​and automatically translating and summarizing it, means for displaying specific spot information on a map in response to a user request, means for reading QR codes that allow the user to obtain detailed spot information, means for analyzing the user's emotions and suggesting optimal spot information and activities in accordance with the emotions, and means for personalizing and suggesting specific dishes and menus based on the user's emotion data. This makes it possible to provide personalized information in accordance with the travelers' emotions and improve their experience.

[0926] A "generative model" is an algorithm that automatically summarizes and translates collected data using natural language processing technology.

[0927] "Spot information" is detailed information about tourist destinations and activities, and is data about places that travelers visit.

[0928] A "local language" is a language commonly spoken in the region or country the traveler is visiting.

[0929] "Reviews" are reviews and opinions posted by other travelers about a particular place or activity.

[0930] To "summarize" means to briefly summarize the main points or content of information.

[0931] "Translating" means converting information written in one language into another language.

[0932] A "user request" is a request or inquiry from a traveler to the system for specific information.

[0933] "Displaying on a map" means visually showing a specific spot using geographical location information.

[0934] A "QR code reading means" is a function that scans a QR code using a smartphone or dedicated device and analyzes its contents.

[0935] "Analyzing user emotions" means assessing the emotions and psychological state of the traveler at that time based on voice and text data obtained from the traveler.

[0936] "Suggesting the best spot information and activities" means guiding users to the most suitable tourist spots and experiences based on their emotions and situation.

[0937] "Personalization" means tailoring the information and services provided to suit the preferences and feelings of individual users.

[0938] The present invention provides a guide system for assisting travelers in obtaining spot information and optimizing their travel experience. Specific embodiments of the system will be described below.

[0939] Server Processing

[0940] Data collection and model generation

[0941] The server periodically collects information about attractions and reviews from travel sites and tourist guides, and stores this data in a database. A generative model (such as OpenAI's GPT-4) is trained using natural language processing technology to automatically summarize the information about attractions and reviews.

[0942] Processing user requests

[0943] When a request for spot information is received from a user device, the server analyzes the request, retrieves the relevant spot information from the database, and summarizes and translates it using a generative model, allowing the user to obtain concise and easy-to-understand information.

[0944] Emotion engine processing

[0945] The server-based emotion engine (such as Google Cloud's Dialogflow) analyzes the voice and text data received from the user and recognizes the user's emotions. This emotion information is then taken into account in other processing steps to suggest the most suitable spots and activities for the user.

[0946] Translating and summarizing reviews

[0947] When a user requests review information, the server retrieves the review data from the database, translates it using an automatic translation engine (such as DeepL), and then summarizes it using a generative model.The server then customizes the content by reflecting the user's emotional information and sends it to the user's device.

[0948] Information distribution and suggestions

[0949] The emotion engine recognizes the user's emotions and optimizes the recommendations for spots and activities. Also, if the user is under stress, the system prioritizes the display of information about spots where they can relax. Furthermore, the system personalizes and suggests specific dishes and menus based on the user's emotional data.

[0950] Terminal handling

[0951] QR code reading

[0952] When a user reads the QR code, the device analyzes the code data and extracts the spot ID, which is then sent to the server as a request.

[0953] Map display and navigation

[0954] The device displays the spot information received from the server on a map and displays the optimal route from the user's current location to the spot. When the user selects a specific spot, the device begins navigation based on the selection.

[0955] Real-time translation and emotion recognition

[0956] The device receives information about spots and reviews from the server and displays them in the local language, and uses an emotion engine to recognize emotions based on the user's voice and text input. The recognized emotion information is sent to the server and reflected in processing.

[0957] Specific examples

[0958] For example, when a user visits a tourist spot, they scan a QR code for a specific restaurant. The device extracts the spot ID and sends a request to the server. The server summarizes and translates the acquired restaurant information and presents it to the user. If the server determines that the user's current emotions are stressful, it will also suggest information about relaxing spots. Specific dishes and menus can also be personalized and suggested based on the user's emotional data.

[0959] Furthermore, when users search for nearby tourist attractions in the map app and select a specific spot, the device will calculate the optimal route and display it on the map. If users want to converse with local people, real-time translation is provided through voice input. Voice feedback based on emotions is also provided, providing a better travel experience.

[0960] An example of a prompt might be:

[0961] I'm feeling very tired today. Can you recommend something to calm me down?

[0962] In this way, the present invention makes it possible to recognize the user's emotions, smoothly obtain information across language barriers, and optimize the travel experience.

[0963] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0964] Step 1:

[0965] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides.

[0966] Input: Data from travel sites and tourist guides

[0967] Data processing: continually storing new information in the database

[0968] Output: The latest spot information and reviews are stored in the database.

[0969] Step 2:

[0970] The server uses a generative model (e.g., GPT-4) to train the collected spot information and reviews, generating a model with automatic summarization and translation capabilities.

[0971] Input: Spot information and reviews stored in the database

[0972] Data Computing: Training with Natural Language Processing Techniques

[0973] Output: A generative model that automatically summarizes and translates

[0974] Step 3:

[0975] A request for spot information is sent from the user terminal to the server.

[0976] Input: User request for spot information

[0977] Data processing: Analysis of request content

[0978] Output: Search criteria for spot information based on the request

[0979] Step 4:

[0980] The server retrieves the relevant spot information from the database and summarizes and translates it using a generative model.

[0981] Input: Search criteria for spot information based on your request

[0982] Data calculation: Summarization and translation

[0983] Output: Summary and translated spot information

[0984] Step 5:

[0985] The server analyzes the voice and text data received from the user terminal using an emotion engine (e.g., Dialogflow) to recognize the user's emotions.

[0986] Input: Voice or text input from the user

[0987] Data Computing: Emotion Recognition Processing

[0988] Output: User's emotional information

[0989] Step 6:

[0990] The server suggests optimal spot information and activities based on the user's emotional information recognized by the emotion engine. If the user is experiencing a certain type of stress, the server will prioritize suggesting information about relaxing spots.

[0991] Input: User's emotional information

[0992] Data Computation: Customized Information Suggestion Based on Emotional Information

[0993] Output: Personalized spot suggestions

[0994] Step 7:

[0995] The server retrieves the user-requested review information from the database, translates it using an automatic translation engine (e.g., DeepL), and then summarizes it using a generative model.

[0996] Input: User review request

[0997] Data Computing: Automatic Translation and Summarization

[0998] Output: Summary and translated reviews

[0999] Step 8:

[1000] The terminal reads the QR code, analyzes the code data to extract the spot ID, and sends it to the server.

[1001] Input: Spot QR code

[1002] Data processing: QR code analysis and spot ID extraction

[1003] Output: Spot ID

[1004] Step 9:

[1005] The terminal displays the spot information received from the server on a map and displays the optimal route from the user's current location to the spot.

[1006] Input: Spot information from the server

[1007] Data calculation: Calculating the optimal route on map data

[1008] Output: Navigation information on a map

[1009] Step 10:

[1010] The device displays the spot information and reviews received from the server in the local language and uses an emotion engine to recognize emotions based on the user's voice and text input. The recognized emotion information is sent to the server and reflected in processing.

[1011] Input: Voice input or text input from the user, spot information and reviews from the server

[1012] Data processing: Real-time translation and emotion recognition

[1013] Output: Display information in local language, send emotion information to server

[1014] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1015] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1016] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1017] [Third embodiment]

[1018] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1019] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1020] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1021] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1022] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1023] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1024] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1025] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1026] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1027] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1028] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1029] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1030] The present invention relates to a guide system for assisting travelers in obtaining spot information. Specific examples will be described below.

[1031] Server Processing

[1032] Data collection and model generation

[1033] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides. This data is stored in a database and used as training data for the generative model. The generative model is trained using natural language processing techniques to automatically summarize the collected information about tourist spots and convert it into an easy-to-understand text format.

[1034] Processing user requests

[1035] When a request is received from a device, the server analyzes the request and retrieves the relevant spot information from the database, which is then summarized by a generative model and automatically translated into the local language.

[1036] Translating and summarizing reviews

[1037] When a user requests review information, the server retrieves the relevant reviews from the database, translates them into major international languages ​​using an automatic translation engine, and then summarizes them using a generative model and provides them to the user.

[1038] Information distribution

[1039] The server then sends the translated and summarized information about the spots and reviews to the device, allowing users to easily understand the local information.

[1040] Terminal handling

[1041] QR code reading

[1042] When a user scans a QR code at a specific tourist spot, the device analyzes the code information, extracts the spot ID, and sends a request to the server. Once the information is received from the server, the device displays it in the GUI.

[1043] Map display and navigation

[1044] The device displays spot information from the server on a map and uses GPS information to provide navigation from the user's current location to the spot. When the user selects a specific spot, the device calculates the optimal route and displays it on the map.

[1045] Real-time translation

[1046] The device displays spot information and reviews received from the server in the local language. When the user converses with a local person, the device accepts voice or text input and translates the content using a real-time translation function. The translation results are immediately provided to the user.

[1047] Specific examples

[1048] For example, when a user visits a tourist spot in Japan, they scan the QR code of a specific restaurant. The device extracts the spot ID and sends a request to the server. The server retrieves the restaurant's information from the database, summarizes it using a generative model, translates it into the local language, and sends it. The device displays this information, and the user can also view reviews from other tourists.

[1049] In addition, users can use the map app to search for nearby tourist attractions and restaurants and select points of interest on the map. After selection, the device will display the optimal route and begin navigation.

[1050] In this way, the guide system according to the present invention enables travelers to smoothly obtain information across language barriers and enrich their local experiences.

[1051] The processing flow will be explained below.

[1052] Server Processing

[1053] Step 1:

[1054] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, and stores the collected data in a database.

[1055] Step 2:

[1056] The server uses the information stored in the database to train a generative model, which then uses natural language processing techniques to extract important keywords and context and acquire the ability to summarize information.

[1057] Step 3:

[1058] When a request for spot information is received from a terminal, the server analyzes the request and retrieves the relevant spot information from the database based on the analysis results.

[1059] Step 4:

[1060] The server passes the acquired spot information to a generative model, which summarizes and translates it into the local language, and then sends the results to the device.

[1061] Step 5:

[1062] When a user requests review information, the server retrieves the relevant review data from the database, then translates it into major international languages ​​using an automatic translation engine and summarizes it using a generative model. The translated and summarized review information is then sent to the device.

[1063] Terminal handling

[1064] Step 1:

[1065] The user reads the QR code, and the device analyzes the data and extracts the spot ID.

[1066] Step 2:

[1067] The device sends a request for spot information to the server based on the analyzed spot ID.

[1068] Step 3:

[1069] The spot information received from the server is displayed on the GUI, allowing users to check the spot information summarized in the local language.

[1070] Step 4:

[1071] The user opens a map app and searches for a specific spot. The device acquires the user's current location via GPS and displays candidate spots received from the server on the map.

[1072] Step 5:

[1073] When the user selects a specific spot, the device calculates the optimal route and displays route guidance on the map.

[1074] User operation example

[1075] Step 1:

[1076] Users find a QR code at a tourist spot and when they scan the code with their device, a request for information about the spot is automatically sent to the server.

[1077] Step 2:

[1078] The spot information received from the server is displayed on the terminal, and the user can check detailed restaurant information in the local language.

[1079] Step 3:

[1080] When a user taps the review tab, the device sends a request for review information to the server, and the translated and summarized review information received from the server is displayed on the device.

[1081] Step 4:

[1082] When a user searches for nearby tourist spots using a map app, multiple options are displayed on the map. When the user selects a specific spot, the device displays the optimal route and begins navigation.

[1083] Step 5:

[1084] When a user speaks to a local person, they input the content of the conversation using the device's voice input function. The translation is done in real time and the results are displayed on the screen.

[1085] Example 1

[1086] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1087] There is a need for a system that allows travelers to efficiently obtain information about places they are visiting and utilize local information without experiencing language barriers. However, existing systems lack sufficient timely summarization and translation of geographical feature information and review information, and are unable to fully support multiple languages. Furthermore, they are limited in terms of navigation functions that allow travelers to instantly obtain the optimal route from their current location to their destination, and functions that support real-time communication with local people. This can make it difficult for travelers to navigate the local area.

[1088] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1089] In this invention, the server includes a means for automatically summarizing geographical feature information to be presented to travelers using a machine learning model, a means for translating the summarized geographical feature information into a local language, and a means for collecting review information corresponding to multiple languages ​​and automatically translating and summarizing the information. This allows travelers to quickly and appropriately obtain the necessary geographical feature information without experiencing language barriers. Furthermore, the server includes a code reading means for displaying specific geographical feature information on a map in response to a user's request and allowing the user to obtain detailed information about the geographical feature, thereby significantly improving user convenience.

[1090] A "traveler" is someone who travels to different locations for tourism, work, or other purposes and needs information and services related to those locations.

[1091] "Geographical feature information" refers to data that indicates the location and characteristics of tourist attractions, facilities, natural landscapes, etc., as well as related information.

[1092] A "guidance system" refers to a system that assists users in efficiently obtaining the information they desire.

[1093] A "machine learning model" refers to an algorithm that is trained using large amounts of data to automatically perform a specific task (e.g., summarizing or translating text).

[1094] "Automatic summarization" refers to technology that extracts the main points from long textual information and organizes them in a short, easy-to-understand format.

[1095] "Means of translation" refers to the technology that converts information written in one language into another language.

[1096] "Review information" refers to feedback data in which users describe their experiences and evaluations.

[1097] "Means for displaying geographic feature information" refers to a method for visually displaying the collected and processed geographic feature information on a user's terminal.

[1098] "Code reading means" refers to technology that reads coded information such as QR codes and barcodes.

[1099] "Location information" refers to data that indicates the latitude and longitude of a specific location using technology such as GPS.

[1100] "Route guidance display function" refers to technology that calculates the optimal route from the current location to the destination and visually shows that route to the user.

[1101] "Voice or text input" refers to a method in which a user speaks to a terminal or inputs text.

[1102] "Means capable of real-time translation" refers to technology that instantly converts input speech or text into another language and quickly provides it to the user.

[1103] The present invention relates to a guidance system that enables travelers to efficiently obtain information about places they are visiting and to utilize local information without feeling the language barrier.

[1104] Server Processing

[1105] The server periodically crawls geographic feature information and review information from multiple travel sites and tourist guides. The collected data is stored in a database such as MongoDB. The collected data is then analyzed using a generative AI model (e.g., GPT-4), extracting and summarizing the necessary information. The summarized information is formatted as text and presented to users in an easy-to-understand format. The generative model is periodically retrained with training data to improve its accuracy.

[1106] Examples:

[1107] For example, information is collected from "travel review site A" and "tourist guide site B." This data is stored in MongoDB, analyzed using Python, and summarized using GPT-4.

[1108] Terminal handling

[1109] The device accepts input from the user (e.g., reading a QR code or selecting a spot). When the user scans a QR code at a specific tourist attraction, the device analyzes the code information to extract the spot ID and sends a request to the server. The information received from the server is displayed on the device's GUI. Furthermore, the device uses GPS information to identify the user's current location and uses the Google Maps API to display geographical feature information on a map. Route guidance is also provided to the user in real time via voice and text.

[1110] Examples:

[1111] For example, a user visits a tourist spot in Japan and scans a QR code for a specific restaurant. The device extracts the spot ID and sends a request to the server. The server retrieves the restaurant's information from the database, summarizes it using a generative AI model, translates it into the local language, and sends it to the device. The device displays this information, and the user can also view reviews from other travelers.

[1112] Processing reviews

[1113] The server receives a user request for review information. The server retrieves the relevant review information from the database and translates it into major international languages ​​using an automatic translation engine (e.g., Google Translate API). The generative AI model then summarizes the review information and provides it to the user.

[1114] Examples:

[1115] When a user requests to view reviews for a specific restaurant, the server retrieves multiple reviews from the database, translates them using the Google Translate API, summarizes them using GPT-4, and sends them to the device.

[1116] Real-time translation

[1117] The device accepts voice or text input from the user and uses real-time translation to translate the input into the specified language, providing the translation results instantly to the user.

[1118] Examples:

[1119] When a user wants to converse with a local person, they speak into the device. The speech recognition function converts the speech into text, which is then translated by the real-time translation engine and displayed or spoken back to the user.

[1120] Prompt Sentence Examples

[1121] "Please summarize recent reviews of this restaurant in Japanese."

[1122] "Please summarize the tourist information about Tokyo Tower and translate it into English."

[1123] Using this system, travelers can easily obtain detailed local information and enjoy a rich experience, regardless of language barriers.

[1124] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1125] The flow of this system's program processing

[1126] Step 1: Data collection and storage

[1127] explanation:

[1128] The server periodically crawls geographical feature information and review information from multiple travel sites and tourist guides.

[1129] input:

[1130] URLs for travel sites and tourist guides.

[1131] Specific behavior:

[1132] The server uses a crawler to gather the required information from the specified URLs.

[1133] output:

[1134] Collected data (geographical feature information and review information).

[1135] Step 2: Save to the database

[1136] explanation:

[1137] The server stores the collected data in a database.

[1138] input:

[1139] Geographical feature information and review information collected through crawling.

[1140] Specific behavior:

[1141] The server performs operations to insert data into a database such as MongoDB.

[1142] output:

[1143] Geographic feature information and review information stored in a database.

[1144] Step 3: Analyze and summarize data using the model

[1145] explanation:

[1146] The server uses a generative AI model (e.g., GPT-4) to analyze and summarize the collected data.

[1147] input:

[1148] Geographic feature information and review information retrieved from databases.

[1149] Specific behavior:

[1150] The server uses a programming language such as Python to analyze the data using the GPT-4 model, extracting and summarizing key points.

[1151] output:

[1152] Summarized geographic feature and review information.

[1153] Step 4: Translation

[1154] explanation:

[1155] The server translates the summarized information into the local language.

[1156] input:

[1157] Summarized geographic feature and review information.

[1158] Specific behavior:

[1159] The server uses a translation API (eg, Google Translate API) to translate the summary information into other languages.

[1160] output:

[1161] Geographical feature information and review information translated into local languages.

[1162] Step 5: Receiving a user request

[1163] explanation:

[1164] The terminal receives a request from a user.

[1165] input:

[1166] Scan QR codes and select specific spots.

[1167] Specific behavior:

[1168] The device uses a QR code reader to extract the code information or obtain the user's selection.

[1169] output:

[1170] The analyzed spot ID or request content.

[1171] Step 6: Parse the request and get information

[1172] explanation:

[1173] The server receives the request from the terminal, analyzes it, and retrieves the relevant information from the database.

[1174] input:

[1175] The spot ID or request content received from the device.

[1176] Specific behavior:

[1177] The server analyzes the request and issues the appropriate query to the database.

[1178] output:

[1179] Geographic feature information and review information retrieved from databases.

[1180] Step 7: Summarize and translate the information

[1181] explanation:

[1182] The server summarizes the acquired information using a generative AI model and translates it into the local language.

[1183] input:

[1184] Obtained geographic feature information and review information.

[1185] Specific behavior:

[1186] The server summarizes the information using the GPT-4 model and translates it into the local language using a translation API.

[1187] output:

[1188] Summarized and translated geographic feature and review information.

[1189] Step 8: Distributing information

[1190] explanation:

[1191] The server transmits the translated and summarized information to the terminal.

[1192] input:

[1193] Summarized and translated geographic feature and review information.

[1194] Specific behavior:

[1195] The server uses HTTP requests to send information to the device.

[1196] output:

[1197] Information delivered to your device.

[1198] Step 9: View information

[1199] explanation:

[1200] The terminal displays the received information in a GUI.

[1201] input:

[1202] The geographic feature information and review information received from the server.

[1203] Specific behavior:

[1204] Use the device's UI components to visually display information.

[1205] output:

[1206] Information visually displayed to the user.

[1207] Step 10: Real-time translation

[1208] explanation:

[1209] The device accepts and translates the user's voice or text input.

[1210] input:

[1211] User voice or text input.

[1212] Specific behavior:

[1213] The device uses a voice recognition system to convert speech into text, which is then translated using a real-time translation engine.

[1214] output:

[1215] Translated text or audio output.

[1216] By following each processing step of this system, travelers can efficiently obtain local information and gain a rich experience beyond language barriers.

[1217] (Application example 1)

[1218] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1219] Modern travelers want to quickly and efficiently obtain information about tourist attractions and restaurants in their destinations. However, language barriers and the complexity of information present obstacles. Furthermore, with advances in virtual reality technology, more and more people are seeking realistic travel experiences in virtual spaces without actually visiting the destinations. However, there is a lack of systems that support such experiences. There is a need for a system that can resolve these issues and enable users to smoothly enjoy both real and virtual travel experiences.

[1220] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1221] In this invention, the server includes means for automatically summarizing spot information to be presented to travelers using a generative model, means for translating the summarized spot information into a local language, means for collecting word-of-mouth information corresponding to multiple languages ​​and automatically translating and summarizing it, means for displaying information about specific tourist spots in a virtual space on a virtual screen, and navigation means for virtually exploring the local area based on the displayed information. This allows users to quickly and efficiently obtain spot information across language barriers and experience exploring the local area in a virtual space.

[1222] A "generative model" is a machine learning model that automatically summarizes and translates based on collected data.

[1223] "Spot information" refers to detailed data about places to visit, such as tourist attractions and restaurants.

[1224] A "local language" is a language commonly spoken in a given region.

[1225] "Word-of-mouth information" refers to the evaluations and impressions provided by users about specific spots.

[1226] A QR code is a two-dimensional barcode with embedded information, which can be obtained by scanning it with a device.

[1227] "Virtual space" is a virtual environment or world created by a computer.

[1228] "Virtual screen" refers to digital information displayed on a head-mounted display or smart glasses.

[1229] "Navigation means" refers to technologies and functions that assist users in understanding directions and routes to their destinations.

[1230] To implement this invention, the server, the terminal (such as smart glasses or a head-mounted display (HMD)), and the user need to cooperate with each other.

[1231] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides. This data is stored in a database and used as training data for the generative model. The generative model uses natural language processing technology to automatically summarize the collected information about tourist spots and convert it into an easy-to-understand text format. The server also analyzes user requests, retrieves relevant information about tourist spots from the database, and uses the generative model to summarize and automatically translate it into the local language.

[1232] When a user scans a QR code at a specific tourist spot, the device analyzes the code information, extracts the spot ID, and sends a request to the server. Once the information is received from the server, the device displays it on the GUI. The device also uses GPS information to display spot information on a map and provides navigation functions. When a user selects a specific spot, the device calculates the optimal route and displays it on the map. In addition, the device displays spot information and reviews received from the server in the local language and uses a real-time translation function to translate conversations with local people.

[1233] In a virtual space implementation, a device reads a QR code in the virtual space and displays information about a specific tourist spot on a virtual screen. A navigation function is also provided to virtually explore the location based on the displayed information. This allows users to enjoy a realistic travel experience in the virtual space without actually visiting the location.

[1234] Hardware and software used

[1235] Hardware: Server, smart glasses, head-mounted display, GPS receiver

[1236] Software: Natural language processing technology, generative models, databases, real-time translation engines

[1237] The server is built using Python. The program retrieves information about spots and reviews via API and summarizes and translates them using a generative AI model. Qt is used to create a GUI on the device, allowing users to smoothly retrieve information and navigate.

[1238] Specific examples

[1239] For example, when a user scans a specific QR code in the virtual space, detailed information about that tourist spot is instantly displayed. The user can check information such as "You can see a traditional festival at this spot" on the virtual screen. Also, when the user inputs "Take a virtual trip to this spot," the device starts virtual navigation and shows the user the optimal route.

[1240] Prompt Sentence Examples

[1241] "Get information about specific tourist spots and display it as a virtual guide."

[1242] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1243] Step 1:

[1244] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides. The input is data obtained through web scraping and API calls, and the output is organized data stored in a database. This data is later used as training data for generative models. Specifically, it processes data collection from websites using Python's BeautifulSoup and Requests.

[1245] Step 2:

[1246] The server uses the collected data to train a generative AI model. The input is the data collected in step 1, and the output is a model that performs summarization and translation. Specifically, the generative AI model is trained using TensorFlow and PyTorch.

[1247] Step 3:

[1248] The user scans a specific QR code on their device. The input is the image data of the QR code, and the output is the analyzed spot ID. Specifically, the device's camera captures the QR code and decodes it using a library such as OpenCV.

[1249] Step 4:

[1250] After extracting the spot ID, the device sends the ID to the server. The input is the spot ID, and the output is a request to the server. Specifically, it uses the Python Requests library to send the HTTP request.

[1251] Step 5:

[1252] The server retrieves the corresponding spot information from the database based on the received spot ID. The input is the spot ID and the output is the spot information. Specifically, it executes an SQL query to retrieve the information from the database.

[1253] Step 6:

[1254] The server summarizes the acquired spot information using a generative AI model and automatically translates it into the local language. The input is the spot information, and the output is the summarized and translated information. Specifically, data is input into a pre-trained generative AI model, and summary and translation results are obtained.

[1255] Step 7:

[1256] The server sends summarized and translated spot information to the terminal. The input is the summarized and translated information, and the output is the response to the terminal. Specifically, the information is returned as an HTTP response.

[1257] Step 8:

[1258] The terminal displays the received information in a GUI. The input is summarized and translated spot information, and the output is a visual display for the user. Specifically, the information is displayed using a GUI framework such as Qt.

[1259] Step 9:

[1260] The device uses GPS information to display spot information on a map and provide navigation. The input is the current location's GPS data, and the output is a navigation route on the map. Specifically, it calculates the route using Google Maps API and displays it on the map.

[1261] Step 10:

[1262] When a user speaks or inputs text, the device translates it in real time. The input is the user's voice or text, and the output is the translated text or voice. Specifically, the device performs text translation using the Google Translate API or similar.

[1263] Step 11:

[1264] In the virtual space, the device reads the QR code and displays information about a specific tourist spot on a virtual screen. The input is the QR code data, and the output is the spot information displayed on the virtual screen. Specific operations utilize QR code scanning technology and a virtual reality framework.

[1265] Step 12:

[1266] The device supports users in navigating within the virtual space based on the displayed information. The input is spot information and user operations, and the output is a navigation route within the virtual space. Specifically, the navigation system within the virtual space is implemented using an engine such as Unity.

[1267] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1268] The present invention relates to a guide system that recognizes the emotions of travelers and combines an emotion engine to optimize the travel experience. Specific examples will be described below.

[1269] Server Processing

[1270] Data collection and model generation

[1271] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, stores this data in a database, and trains the generative model using natural language processing technology to automatically summarize the information about tourist spots and reviews.

[1272] Processing user requests

[1273] When a request for spot information is received from a device, the server analyzes the request, retrieves the relevant spot information from the database, and summarizes and translates it using a generative model.

[1274] Emotion engine processing

[1275] The emotion engine embedded in the server analyzes the voice and text data received from the user to recognize the user's emotion, which is then taken into account in other processing steps.

[1276] Translating and summarizing reviews

[1277] When a user requests review information, the server retrieves the review data from the database, translates it using an automatic translation engine, and then summarizes it using a generative model.The server then customizes the content by reflecting the user's emotional information and sends it to the device.

[1278] Information distribution and suggestions

[1279] The emotion engine recognizes the user's emotions and optimizes the suggested spots and activities. If the user is under stress, the system prioritizes the display of information about relaxing spots.

[1280] Terminal handling

[1281] QR code reading

[1282] When a user reads the QR code, the device analyzes the code data and extracts the spot ID, which is then sent to the server as a request.

[1283] Map display and navigation

[1284] The device displays the spot information received from the server on a map and displays the optimal route from the user's current location to the spot. When the user selects a specific spot, the device begins navigation based on the selection.

[1285] Real-time translation and emotion recognition

[1286] The device receives information about spots and reviews from the server and displays them in the local language, and uses an emotion engine to recognize emotions based on the user's voice and text input. The recognized emotion information is sent to the server and reflected in processing.

[1287] Specific examples

[1288] For example, when a user visits a tourist spot, they can scan the QR code of a specific restaurant. The device extracts the spot ID and sends a request to the server. The server summarizes and translates the acquired restaurant information and presents it to the user. If the server determines that the user's current emotional state is stressful, it will also suggest information about relaxing spots.

[1289] Furthermore, when users search for nearby tourist attractions in the map app and select a specific spot, the device will calculate the optimal route and display it on the map. If users want to converse with local people, real-time translation is provided through voice input. Voice feedback based on emotions is also provided, providing a better travel experience.

[1290] In this way, the present invention makes it possible to recognize the user's emotions, smoothly obtain information across language barriers, and optimize the travel experience.

[1291] The processing flow will be explained below.

[1292] Server Processing

[1293] Step 1:

[1294] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, and stores the collected data in a database.

[1295] Step 2:

[1296] The server uses the information stored in the database to train a generative model, which uses natural language processing techniques to automatically summarize information about spots and reviews.

[1297] Step 3:

[1298] When a request for spot information is received from a terminal, the server analyzes the request and retrieves the corresponding spot information from the database.

[1299] Step 4:

[1300] The server passes the acquired spot information to the generative model, which summarizes and translates it into the local language. The translated and summarized information is then sent to the device.

[1301] Step 5:

[1302] When a user submits a review request, the server retrieves the review data from the database and translates it into major international languages ​​using an automatic translation engine.Then, a generative model summarizes the review information and customizes it based on the user's sentiment information.

[1303] Step 6:

[1304] The emotion engine built into the server analyzes the voice and text data received from the user and recognizes the user's emotions. The recognized emotion information is reflected in information processing and presentation.

[1305] Step 7:

[1306] Based on the user's emotions recognized by the emotion engine, the server optimizes and suggests information on spots and activities suitable for the traveler. Also, if the user is feeling stressed, it will prioritize providing information on spots where they can relax.

[1307] Terminal handling

[1308] Step 1:

[1309] The user scans the QR code at a tourist spot, and the device analyzes the QR code data to extract the spot ID.

[1310] Step 2:

[1311] The terminal sends a request for spot information to the server based on the analyzed spot ID.

[1312] Step 3:

[1313] The spot information received from the server is displayed in the local language on the terminal's GUI, and the user can check the displayed spot information.

[1314] Step 4:

[1315] The user opens a map app and searches for a specific spot. The device acquires the user's current location via GPS and displays the spot information received from the server on a map.

[1316] Step 5:

[1317] When the user selects a specific spot, the device calculates the optimal route and displays route guidance on the map.

[1318] Step 6:

[1319] The terminal displays the reviews received from the server to the user in the local language, with the reviews automatically translated and summarized.

[1320] Step 7:

[1321] When the user expresses their emotions through voice or text input, the device sends this input data to the server.

[1322] Step 8:

[1323] Based on the emotional feedback received from the server in real time, the device will suggest spot information and activities that correspond to the user's emotions.

[1324] User operation example

[1325] Step 1:

[1326] The user scans the QR code at a tourist spot and sends a request to the server to display the information on the device.

[1327] Step 2:

[1328] The server summarizes the spot information using a generative model, translates it, and sends it to the device, where the user can view the spot information in their local language.

[1329] Step 3:

[1330] If the user wants to check the reviews, they tap the reviews tab to send a request. The server translates and summarizes the reviews and sends them to the device.

[1331] Step 4:

[1332] Users use a map app to search for nearby tourist spots and select a specific spot, and the device calculates the optimal route and displays navigation.

[1333] Step 5:

[1334] When users converse with local people, they use the voice input function. The device sends the input speech to the server and displays the translation results in real time. The emotion engine also analyzes the user's emotions and provides customized feedback based on the results.

[1335] Example 2

[1336] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1337] Conventional guide systems do not adequately consider traveler emotions or language barriers, resulting in suboptimal travel experiences. Travelers often find themselves in situations where they feel stressed or are unable to obtain sufficient information due to not understanding the local language, resulting in lower travel satisfaction.

[1338] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1339] In this invention, the server includes means for automatically summarizing spot information to be presented to travelers using a generative model, means for translating the summarized spot information into a local language, means for collecting word-of-mouth information corresponding to multiple languages ​​and automatically translating and summarizing it, emotion analysis means for recognizing emotions from a user's voice or text input, and suggestion means for optimizing the spot information and suggestions based on the emotion analysis results. This enables travelers to smoothly obtain information across language barriers and receive optimal suggestions according to their emotions, thereby improving satisfaction with their travel experience.

[1340] A "generative model" is a machine learning algorithm that automatically summarizes information based on collected data.

[1341] A "local language" is a language commonly spoken in the area the traveller is visiting.

[1342] "Word-of-mouth information" refers to data in which users write their ratings and opinions about certain spots or services.

[1343] "Machine translation" refers to the technology of automatically converting text between multiple languages.

[1344] "Emotion analysis" is a technology that analyzes a user's voice and text data and recognizes their emotions.

[1345] "Suggestion means" refers to a function that suggests optimal spot information and activities to users based on the results of emotion analysis.

[1346] "Code reading" refers to the technology of reading code information such as QR codes using a device's camera.

[1347] "Navigation" refers to a function that provides guidance on a route from the user's current location to a destination.

[1348] A "user request" is a request for information made by a user to a server via a terminal.

[1349] A "database" is a system that systematically stores and manages collected spot information and reviews.

[1350] The present invention relates to a guide system for recognizing a traveler's emotions and optimizing the travel experience. Specific examples will be described below.

[1351] Server Processing

[1352] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, and stores the data in a database. This is done using web scraping technology, such as Python's BeautifulSoup library. The collected data is then stored in a database such as MongoDB.

[1353] The data is then trained using a generative AI model with natural language processing techniques (e.g., the BERT model) to automatically summarize information about attractions and reviews, ensuring that the information provided to travelers is concise and easy to understand.

[1354] When a request for specific spot information is received from a device, the server analyzes the request and retrieves the corresponding spot information from the database, summarizes this information using a generative AI model, and then translates it into the local language using tools such as Google Translate API.

[1355] The server is also equipped with an emotion engine that analyzes voice and text data received from users. For example, if a user says, "I've been feeling a bit stressed lately," the emotion analysis module analyzes this information and recognizes the user's emotion. This emotion information is used in subsequent processing steps.

[1356] When a user requests review information, the server retrieves the information from the database, translates it using an automatic translation engine, and summarizes it using a generative AI model. This summarized information is then customized to reflect the user's emotional information and sent to the device.

[1357] Based on the user's emotions recognized by the emotion engine, the server optimizes the recommendations of spot information and activities. For example, if the server recognizes that the user is under stress, it will prioritize providing information on relaxation spots.

[1358] Terminal handling

[1359] When a user reads the QR code, the device analyzes the code data to extract the spot ID and sends it as a request to the server. The spot information sent from the server is displayed on the device's map app, and the optimal route from the user's current location to the spot is shown.

[1360] The device's emotion engine analyzes the user's voice and text inputs to recognize the user's emotions. This recognized emotion information is sent to the server and reflected in further processing. For example, if a user voice-inputs "I'm looking for a good restaurant," the emotion engine will suggest the most suitable restaurant based on the information analyzed.

[1361] Specific examples

[1362] For example, when a user visits a tourist spot, they scan a QR code for a specific restaurant. The device extracts the spot ID and sends a request to the server. The server summarizes and translates the acquired restaurant information and presents it to the user. If the server determines that the user's current emotional state is stressful, it will also suggest information about relaxing spots.

[1363] Furthermore, when users search for nearby tourist attractions in the map app and select a specific spot, the device will calculate the optimal route and display it on the map. If users want to converse with local people, the device's voice input function will provide real-time translation and emotional feedback, providing a better travel experience.

[1364] Prompt Sentence Examples

[1365] "I want to see summaries of reviews of tourist spots I want to visit when I travel."

[1366] "Please suggest places where I can relax in Tokyo."

[1367] In this way, the present invention makes it possible to recognize the user's emotions, smoothly obtain information across language barriers, and optimize the travel experience.

[1368] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1369] Step 1:

[1370] Data collection and model generation

[1371] Subject: Server

[1372] The server collects information about attractions and reviews from travel sites and tourist guides using web scraping technology with Python's BeautifulSoup library. The collected data is then stored in a database such as MongoDB. The server then uses a generative AI model, such as the BERT model, to train this data and automatically summarize the information about attractions and reviews.

[1373] Input: Spot information and reviews collected from websites

[1374] Output: Automatically summarized information about places and reviews

[1375] Step 2:

[1376] Processing user requests

[1377] Subject: Server

[1378] When a request for specific spot information is received from a device, the server analyzes the request and retrieves the corresponding spot information from the database. It then summarizes it using a generative AI model and translates it into the local language using a Google Translate API or similar. The translated information is then sent back to the device.

[1379] Input: User request for spot information

[1380] Output: Translated spot information

[1381] Step 3:

[1382] Emotion engine processing

[1383] Subject: Server

[1384] The server analyzes the voice and text data received from the user through the emotion analysis module to recognize the user's emotion. This emotion information is stored in a database and taken into account in subsequent processing steps.

[1385] Input: User voice or text data

[1386] Output: Recognized user emotion information

[1387] Step 4:

[1388] Translating and summarizing reviews

[1389] Subject: Server

[1390] When a user requests review information, the server retrieves the corresponding review data from the database, translates it using an automatic translation engine, and then summarizes it using a generative AI model. The resulting information is customized based on the user's emotional information and sent to the device.

[1391] Input: User review request

[1392] Output: Translated and summarized reviews

[1393] Step 5:

[1394] Information distribution and suggestions

[1395] Subject: Server

[1396] Based on the results of the emotion analysis, the server optimizes the information on spots and activities suggested to users. For example, if the server detects that the user is feeling stressed, it will prioritize information on spots where users can relax.

[1397] Input: Recognized user emotion information

[1398] Output: Optimized spot information and recommendations

[1399] Step 6:

[1400] QR code reading

[1401] Subject: Terminal

[1402] When a user reads the QR code, the device analyzes the code data to extract the spot ID, which is then sent to the server as a request.

[1403] Input: QR code scanned by the user

[1404] Output: Spot ID sent to the server

[1405] Step 7:

[1406] Map display and navigation

[1407] Subject: Terminal

[1408] The device displays the spot information received from the server on a map and shows the optimal route from the user's current location to the spot. When the user selects a specific spot, navigation begins.

[1409] Input: Spot information received from the server

[1410] Output: Points of interest on a map and a navigation route

[1411] Step 8:

[1412] Real-time translation and emotion recognition

[1413] Subject: Terminal

[1414] The device's emotion engine analyzes the user's voice and text input and recognizes emotions. The recognized emotion information is sent to the server and reflected in processing. In addition, the user's conversation information is translated in real time and returned as text or voice.

[1415] Input: User voice or text input

[1416] Output: Recognized emotion information and translation results

[1417] This is the specific processing flow of this system. In this way, it is possible to take into account the user's feelings and provide a smooth travel experience that transcends language barriers.

[1418] (Application example 2)

[1419] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1420] Conventional guide systems have difficulty providing information that is sensitive to travelers' emotions, resulting in travel experiences that are not tailored to individual emotions and circumstances. Furthermore, there are issues with multilingual support, real-time navigation, and review summarization and translation, creating a need for more highly personalized experiences. The goal of this project is to resolve these issues and provide optimal information based on travelers' emotions, improving their experiences.

[1421] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1422] In this invention, the server includes means for automatically summarizing spot information to be presented to travelers using a generative model, means for translating the summarized spot information into a local language, means for collecting word-of-mouth information corresponding to multiple languages ​​and automatically translating and summarizing it, means for displaying specific spot information on a map in response to a user request, means for reading QR codes that allow the user to obtain detailed spot information, means for analyzing the user's emotions and suggesting optimal spot information and activities in accordance with the emotions, and means for personalizing and suggesting specific dishes and menus based on the user's emotion data. This makes it possible to provide personalized information in accordance with the travelers' emotions and improve their experience.

[1423] A "generative model" is an algorithm that automatically summarizes and translates collected data using natural language processing technology.

[1424] "Spot information" is detailed information about tourist destinations and activities, and is data about places that travelers visit.

[1425] A "local language" is a language commonly spoken in the region or country the traveler is visiting.

[1426] "Reviews" are reviews and opinions posted by other travelers about a particular place or activity.

[1427] To "summarize" means to briefly summarize the main points or content of information.

[1428] "Translating" means converting information written in one language into another language.

[1429] A "user request" is a request or inquiry from a traveler to the system for specific information.

[1430] "Displaying on a map" means visually showing a specific spot using geographical location information.

[1431] A "QR code reading means" is a function that scans a QR code using a smartphone or dedicated device and analyzes its contents.

[1432] "Analyzing user emotions" means assessing the emotions and psychological state of the traveler at that time based on voice and text data obtained from the traveler.

[1433] "Suggesting the best spot information and activities" means guiding users to the most suitable tourist spots and experiences based on their emotions and situation.

[1434] "Personalization" means tailoring the information and services provided to suit the preferences and feelings of individual users.

[1435] The present invention provides a guide system for assisting travelers in obtaining spot information and optimizing their travel experience. Specific embodiments of the system will be described below.

[1436] Server Processing

[1437] Data collection and model generation

[1438] The server periodically collects information about attractions and reviews from travel sites and tourist guides, and stores this data in a database. A generative model (such as OpenAI's GPT-4) is trained using natural language processing technology to automatically summarize the information about attractions and reviews.

[1439] Processing user requests

[1440] When a request for spot information is received from a user device, the server analyzes the request, retrieves the relevant spot information from the database, and summarizes and translates it using a generative model, allowing the user to obtain concise and easy-to-understand information.

[1441] Emotion engine processing

[1442] The server-based emotion engine (such as Google Cloud's Dialogflow) analyzes the voice and text data received from the user and recognizes the user's emotions. This emotion information is then taken into account in other processing steps to suggest the most suitable spots and activities for the user.

[1443] Translating and summarizing reviews

[1444] When a user requests review information, the server retrieves the review data from the database, translates it using an automatic translation engine (such as DeepL), and then summarizes it using a generative model.The server then customizes the content by reflecting the user's emotional information and sends it to the user's device.

[1445] Information distribution and suggestions

[1446] The emotion engine recognizes the user's emotions and optimizes the recommendations for spots and activities. Also, if the user is under stress, the system prioritizes the display of information about spots where they can relax. Furthermore, the system personalizes and suggests specific dishes and menus based on the user's emotional data.

[1447] Terminal handling

[1448] QR code reading

[1449] When a user reads the QR code, the device analyzes the code data and extracts the spot ID, which is then sent to the server as a request.

[1450] Map display and navigation

[1451] The device displays the spot information received from the server on a map and displays the optimal route from the user's current location to the spot. When the user selects a specific spot, the device begins navigation based on the selection.

[1452] Real-time translation and emotion recognition

[1453] The device receives information about spots and reviews from the server and displays them in the local language, and uses an emotion engine to recognize emotions based on the user's voice and text input. The recognized emotion information is sent to the server and reflected in processing.

[1454] Specific examples

[1455] For example, when a user visits a tourist spot, they scan a QR code for a specific restaurant. The device extracts the spot ID and sends a request to the server. The server summarizes and translates the acquired restaurant information and presents it to the user. If the server determines that the user's current emotions are stressful, it will also suggest information about relaxing spots. Specific dishes and menus can also be personalized and suggested based on the user's emotional data.

[1456] Furthermore, when users search for nearby tourist attractions in the map app and select a specific spot, the device will calculate the optimal route and display it on the map. If users want to converse with local people, real-time translation is provided through voice input. Voice feedback based on emotions is also provided, providing a better travel experience.

[1457] An example of a prompt might be:

[1458] I'm feeling very tired today. Can you recommend something to calm me down?

[1459] In this way, the present invention makes it possible to recognize the user's emotions, smoothly obtain information across language barriers, and optimize the travel experience.

[1460] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1461] Step 1:

[1462] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides.

[1463] Input: Data from travel sites and tourist guides

[1464] Data processing: continually storing new information in the database

[1465] Output: The latest spot information and reviews are stored in the database.

[1466] Step 2:

[1467] The server uses a generative model (e.g., GPT-4) to train the collected spot information and reviews, generating a model with automatic summarization and translation capabilities.

[1468] Input: Spot information and reviews stored in the database

[1469] Data Computing: Training with Natural Language Processing Techniques

[1470] Output: A generative model that automatically summarizes and translates

[1471] Step 3:

[1472] A request for spot information is sent from the user terminal to the server.

[1473] Input: User request for spot information

[1474] Data processing: Analysis of request content

[1475] Output: Search criteria for spot information based on the request

[1476] Step 4:

[1477] The server retrieves the relevant spot information from the database and summarizes and translates it using a generative model.

[1478] Input: Search criteria for spot information based on your request

[1479] Data calculation: Summarization and translation

[1480] Output: Summary and translated spot information

[1481] Step 5:

[1482] The server analyzes the voice and text data received from the user terminal using an emotion engine (e.g., Dialogflow) to recognize the user's emotions.

[1483] Input: Voice or text input from the user

[1484] Data Computing: Emotion Recognition Processing

[1485] Output: User's emotional information

[1486] Step 6:

[1487] The server suggests optimal spot information and activities based on the user's emotional information recognized by the emotion engine. If the user is experiencing a certain type of stress, the server will prioritize suggesting information about relaxing spots.

[1488] Input: User's emotional information

[1489] Data Computation: Customized Information Suggestion Based on Emotional Information

[1490] Output: Personalized spot suggestions

[1491] Step 7:

[1492] The server retrieves the user-requested review information from the database, translates it using an automatic translation engine (e.g., DeepL), and then summarizes it using a generative model.

[1493] Input: User review request

[1494] Data Computing: Automatic Translation and Summarization

[1495] Output: Summary and translated reviews

[1496] Step 8:

[1497] The terminal reads the QR code, analyzes the code data to extract the spot ID, and sends it to the server.

[1498] Input: Spot QR code

[1499] Data processing: QR code analysis and spot ID extraction

[1500] Output: Spot ID

[1501] Step 9:

[1502] The terminal displays the spot information received from the server on a map and displays the optimal route from the user's current location to the spot.

[1503] Input: Spot information from the server

[1504] Data calculation: Calculating the optimal route on map data

[1505] Output: Navigation information on a map

[1506] Step 10:

[1507] The device displays the spot information and reviews received from the server in the local language and uses an emotion engine to recognize emotions based on the user's voice and text input. The recognized emotion information is sent to the server and reflected in processing.

[1508] Input: Voice input or text input from the user, spot information and reviews from the server

[1509] Data processing: Real-time translation and emotion recognition

[1510] Output: Display information in local language, send emotion information to server

[1511] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1512] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1513] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1514] [Fourth embodiment]

[1515] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1516] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1517] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1518] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1519] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1520] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1521] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1522] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1523] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1524] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1525] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1526] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1527] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1528] The present invention relates to a guide system for assisting travelers in obtaining spot information. Specific examples will be described below.

[1529] Server Processing

[1530] Data collection and model generation

[1531] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides. This data is stored in a database and used as training data for the generative model. The generative model is trained using natural language processing techniques to automatically summarize the collected information about tourist spots and convert it into an easy-to-understand text format.

[1532] Processing user requests

[1533] When a request is received from a device, the server analyzes the request and retrieves the relevant spot information from the database, which is then summarized by a generative model and automatically translated into the local language.

[1534] Translating and summarizing reviews

[1535] When a user requests review information, the server retrieves the relevant reviews from the database, translates them into major international languages ​​using an automatic translation engine, and then summarizes them using a generative model and provides them to the user.

[1536] Information distribution

[1537] The server then sends the translated and summarized information about the spots and reviews to the device, allowing users to easily understand the local information.

[1538] Terminal handling

[1539] QR code reading

[1540] When a user scans a QR code at a specific tourist spot, the device analyzes the code information, extracts the spot ID, and sends a request to the server. Once the information is received from the server, the device displays it in the GUI.

[1541] Map display and navigation

[1542] The device displays spot information from the server on a map and uses GPS information to provide navigation from the user's current location to the spot. When the user selects a specific spot, the device calculates the optimal route and displays it on the map.

[1543] Real-time translation

[1544] The device displays spot information and reviews received from the server in the local language. When the user converses with a local person, the device accepts voice or text input and translates the content using a real-time translation function. The translation results are immediately provided to the user.

[1545] Specific examples

[1546] For example, when a user visits a tourist spot in Japan, they scan the QR code of a specific restaurant. The device extracts the spot ID and sends a request to the server. The server retrieves the restaurant's information from the database, summarizes it using a generative model, translates it into the local language, and sends it. The device displays this information, and the user can also view reviews from other tourists.

[1547] In addition, users can use the map app to search for nearby tourist attractions and restaurants and select points of interest on the map. After selection, the device will display the optimal route and begin navigation.

[1548] In this way, the guide system according to the present invention enables travelers to smoothly obtain information across language barriers and enrich their local experiences.

[1549] The processing flow will be explained below.

[1550] Server Processing

[1551] Step 1:

[1552] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, and stores the collected data in a database.

[1553] Step 2:

[1554] The server uses the information stored in the database to train a generative model, which then uses natural language processing techniques to extract important keywords and context and acquire the ability to summarize information.

[1555] Step 3:

[1556] When a request for spot information is received from a terminal, the server analyzes the request and retrieves the relevant spot information from the database based on the analysis results.

[1557] Step 4:

[1558] The server passes the acquired spot information to a generative model, which summarizes and translates it into the local language, and then sends the results to the device.

[1559] Step 5:

[1560] When a user requests review information, the server retrieves the relevant review data from the database, then translates it into major international languages ​​using an automatic translation engine and summarizes it using a generative model. The translated and summarized review information is then sent to the device.

[1561] Terminal handling

[1562] Step 1:

[1563] The user reads the QR code, and the device analyzes the data and extracts the spot ID.

[1564] Step 2:

[1565] The device sends a request for spot information to the server based on the analyzed spot ID.

[1566] Step 3:

[1567] The spot information received from the server is displayed on the GUI, allowing users to check the spot information summarized in the local language.

[1568] Step 4:

[1569] The user opens a map app and searches for a specific spot. The device acquires the user's current location via GPS and displays candidate spots received from the server on the map.

[1570] Step 5:

[1571] When the user selects a specific spot, the device calculates the optimal route and displays route guidance on the map.

[1572] User operation example

[1573] Step 1:

[1574] Users find a QR code at a tourist spot and when they scan the code with their device, a request for information about the spot is automatically sent to the server.

[1575] Step 2:

[1576] The spot information received from the server is displayed on the terminal, and the user can check detailed restaurant information in the local language.

[1577] Step 3:

[1578] When a user taps the review tab, the device sends a request for review information to the server, and the translated and summarized review information received from the server is displayed on the device.

[1579] Step 4:

[1580] When a user searches for nearby tourist spots using a map app, multiple options are displayed on the map. When the user selects a specific spot, the device displays the optimal route and begins navigation.

[1581] Step 5:

[1582] When a user speaks to a local person, they input the content of the conversation using the device's voice input function. The translation is done in real time and the results are displayed on the screen.

[1583] Example 1

[1584] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1585] There is a need for a system that allows travelers to efficiently obtain information about places they are visiting and utilize local information without experiencing language barriers. However, existing systems lack sufficient timely summarization and translation of geographical feature information and review information, and are unable to fully support multiple languages. Furthermore, they are limited in terms of navigation functions that allow travelers to instantly obtain the optimal route from their current location to their destination, and functions that support real-time communication with local people. This can make it difficult for travelers to navigate the local area.

[1586] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1587] In this invention, the server includes a means for automatically summarizing geographical feature information to be presented to travelers using a machine learning model, a means for translating the summarized geographical feature information into a local language, and a means for collecting review information corresponding to multiple languages ​​and automatically translating and summarizing the information. This allows travelers to quickly and appropriately obtain the necessary geographical feature information without experiencing language barriers. Furthermore, the server includes a code reading means for displaying specific geographical feature information on a map in response to a user's request and allowing the user to obtain detailed information about the geographical feature, thereby significantly improving user convenience.

[1588] A "traveler" is someone who travels to different locations for tourism, work, or other purposes and needs information and services related to those locations.

[1589] "Geographical feature information" refers to data that indicates the location and characteristics of tourist attractions, facilities, natural landscapes, etc., as well as related information.

[1590] A "guidance system" refers to a system that assists users in efficiently obtaining the information they desire.

[1591] A "machine learning model" refers to an algorithm that is trained using large amounts of data to automatically perform a specific task (e.g., summarizing or translating text).

[1592] "Automatic summarization" refers to technology that extracts the main points from long textual information and organizes them in a short, easy-to-understand format.

[1593] "Means of translation" refers to the technology that converts information written in one language into another language.

[1594] "Review information" refers to feedback data in which users describe their experiences and evaluations.

[1595] "Means for displaying geographic feature information" refers to a method for visually displaying the collected and processed geographic feature information on a user's terminal.

[1596] "Code reading means" refers to technology that reads coded information such as QR codes and barcodes.

[1597] "Location information" refers to data that indicates the latitude and longitude of a specific location using technology such as GPS.

[1598] "Route guidance display function" refers to technology that calculates the optimal route from the current location to the destination and visually shows that route to the user.

[1599] "Voice or text input" refers to a method in which a user speaks to a terminal or inputs text.

[1600] "Means capable of real-time translation" refers to technology that instantly converts input speech or text into another language and quickly provides it to the user.

[1601] The present invention relates to a guidance system that enables travelers to efficiently obtain information about places they are visiting and to utilize local information without feeling the language barrier.

[1602] Server Processing

[1603] The server periodically crawls geographic feature information and review information from multiple travel sites and tourist guides. The collected data is stored in a database such as MongoDB. The collected data is then analyzed using a generative AI model (e.g., GPT-4), extracting and summarizing the necessary information. The summarized information is formatted as text and presented to users in an easy-to-understand format. The generative model is periodically retrained with training data to improve its accuracy.

[1604] Examples:

[1605] For example, information is collected from "travel review site A" and "tourist guide site B." This data is stored in MongoDB, analyzed using Python, and summarized using GPT-4.

[1606] Terminal handling

[1607] The device accepts input from the user (e.g., reading a QR code or selecting a spot). When the user scans a QR code at a specific tourist attraction, the device analyzes the code information to extract the spot ID and sends a request to the server. The information received from the server is displayed on the device's GUI. Furthermore, the device uses GPS information to identify the user's current location and uses the Google Maps API to display geographical feature information on a map. Route guidance is also provided to the user in real time via voice and text.

[1608] Examples:

[1609] For example, a user visits a tourist spot in Japan and scans a QR code for a specific restaurant. The device extracts the spot ID and sends a request to the server. The server retrieves the restaurant's information from the database, summarizes it using a generative AI model, translates it into the local language, and sends it to the device. The device displays this information, and the user can also view reviews from other travelers.

[1610] Processing reviews

[1611] The server receives a user request for review information. The server retrieves the relevant review information from the database and translates it into major international languages ​​using an automatic translation engine (e.g., Google Translate API). The generative AI model then summarizes the review information and provides it to the user.

[1612] Examples:

[1613] When a user requests to view reviews for a specific restaurant, the server retrieves multiple reviews from the database, translates them using the Google Translate API, summarizes them using GPT-4, and sends them to the device.

[1614] Real-time translation

[1615] The device accepts voice or text input from the user and uses real-time translation to translate the input into the specified language, providing the translation results instantly to the user.

[1616] Examples:

[1617] When a user wants to converse with a local person, they speak into the device. The speech recognition function converts the speech into text, which is then translated by the real-time translation engine and displayed or spoken back to the user.

[1618] Prompt Sentence Examples

[1619] "Please summarize recent reviews of this restaurant in Japanese."

[1620] "Please summarize the tourist information about Tokyo Tower and translate it into English."

[1621] Using this system, travelers can easily obtain detailed local information and enjoy a rich experience, regardless of language barriers.

[1622] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1623] The flow of this system's program processing

[1624] Step 1: Data collection and storage

[1625] explanation:

[1626] The server periodically crawls geographical feature information and review information from multiple travel sites and tourist guides.

[1627] input:

[1628] URLs for travel sites and tourist guides.

[1629] Specific behavior:

[1630] The server uses a crawler to gather the required information from the specified URLs.

[1631] output:

[1632] Collected data (geographical feature information and review information).

[1633] Step 2: Save to the database

[1634] explanation:

[1635] The server stores the collected data in a database.

[1636] input:

[1637] Geographical feature information and review information collected through crawling.

[1638] Specific behavior:

[1639] The server performs operations to insert data into a database such as MongoDB.

[1640] output:

[1641] Geographic feature information and review information stored in a database.

[1642] Step 3: Analyze and summarize data using the model

[1643] explanation:

[1644] The server uses a generative AI model (e.g., GPT-4) to analyze and summarize the collected data.

[1645] input:

[1646] Geographic feature information and review information retrieved from databases.

[1647] Specific behavior:

[1648] The server uses a programming language such as Python to analyze the data using the GPT-4 model, extracting and summarizing key points.

[1649] output:

[1650] Summarized geographic feature and review information.

[1651] Step 4: Translation

[1652] explanation:

[1653] The server translates the summarized information into the local language.

[1654] input:

[1655] Summarized geographic feature and review information.

[1656] Specific behavior:

[1657] The server uses a translation API (eg, Google Translate API) to translate the summary information into other languages.

[1658] output:

[1659] Geographical feature information and review information translated into local languages.

[1660] Step 5: Receiving a user request

[1661] explanation:

[1662] The terminal receives a request from a user.

[1663] input:

[1664] Scan QR codes and select specific spots.

[1665] Specific behavior:

[1666] The device uses a QR code reader to extract the code information or obtain the user's selection.

[1667] output:

[1668] The analyzed spot ID or request content.

[1669] Step 6: Parse the request and get information

[1670] explanation:

[1671] The server receives the request from the terminal, analyzes it, and retrieves the relevant information from the database.

[1672] input:

[1673] The spot ID or request content received from the device.

[1674] Specific behavior:

[1675] The server analyzes the request and issues the appropriate query to the database.

[1676] output:

[1677] Geographic feature information and review information retrieved from databases.

[1678] Step 7: Summarize and translate the information

[1679] explanation:

[1680] The server summarizes the acquired information using a generative AI model and translates it into the local language.

[1681] input:

[1682] Obtained geographic feature information and review information.

[1683] Specific behavior:

[1684] The server summarizes the information using the GPT-4 model and translates it into the local language using a translation API.

[1685] output:

[1686] Summarized and translated geographic feature and review information.

[1687] Step 8: Distributing information

[1688] explanation:

[1689] The server transmits the translated and summarized information to the terminal.

[1690] input:

[1691] Summarized and translated geographic feature and review information.

[1692] Specific behavior:

[1693] The server uses HTTP requests to send information to the device.

[1694] output:

[1695] Information delivered to your device.

[1696] Step 9: View information

[1697] explanation:

[1698] The terminal displays the received information in a GUI.

[1699] input:

[1700] The geographic feature information and review information received from the server.

[1701] Specific behavior:

[1702] Use the device's UI components to visually display information.

[1703] output:

[1704] Information visually displayed to the user.

[1705] Step 10: Real-time translation

[1706] explanation:

[1707] The device accepts and translates the user's voice or text input.

[1708] input:

[1709] User voice or text input.

[1710] Specific behavior:

[1711] The device uses a voice recognition system to convert speech into text, which is then translated using a real-time translation engine.

[1712] output:

[1713] Translated text or audio output.

[1714] By following each processing step of this system, travelers can efficiently obtain local information and gain a rich experience beyond language barriers.

[1715] (Application example 1)

[1716] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1717] Modern travelers want to quickly and efficiently obtain information about tourist attractions and restaurants in their destinations. However, language barriers and the complexity of information present obstacles. Furthermore, with advances in virtual reality technology, more and more people are seeking realistic travel experiences in virtual spaces without actually visiting the destinations. However, there is a lack of systems that support such experiences. There is a need for a system that can resolve these issues and enable users to smoothly enjoy both real and virtual travel experiences.

[1718] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1719] In this invention, the server includes means for automatically summarizing spot information to be presented to travelers using a generative model, means for translating the summarized spot information into a local language, means for collecting word-of-mouth information corresponding to multiple languages ​​and automatically translating and summarizing it, means for displaying information about specific tourist spots in a virtual space on a virtual screen, and navigation means for virtually exploring the local area based on the displayed information. This allows users to quickly and efficiently obtain spot information across language barriers and experience exploring the local area in a virtual space.

[1720] A "generative model" is a machine learning model that automatically summarizes and translates based on collected data.

[1721] "Spot information" refers to detailed data about places to visit, such as tourist attractions and restaurants.

[1722] A "local language" is a language commonly spoken in a given region.

[1723] "Word-of-mouth information" refers to the evaluations and impressions provided by users about specific spots.

[1724] A QR code is a two-dimensional barcode with embedded information, which can be obtained by scanning it with a device.

[1725] "Virtual space" is a virtual environment or world created by a computer.

[1726] "Virtual screen" refers to digital information displayed on a head-mounted display or smart glasses.

[1727] "Navigation means" refers to technologies and functions that assist users in understanding directions and routes to their destinations.

[1728] To implement this invention, the server, the terminal (such as smart glasses or a head-mounted display (HMD)), and the user need to cooperate with each other.

[1729] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides. This data is stored in a database and used as training data for the generative model. The generative model uses natural language processing technology to automatically summarize the collected information about tourist spots and convert it into an easy-to-understand text format. The server also analyzes user requests, retrieves relevant information about tourist spots from the database, and uses the generative model to summarize and automatically translate it into the local language.

[1730] When a user scans a QR code at a specific tourist spot, the device analyzes the code information, extracts the spot ID, and sends a request to the server. Once the information is received from the server, the device displays it on the GUI. The device also uses GPS information to display spot information on a map and provides navigation functions. When a user selects a specific spot, the device calculates the optimal route and displays it on the map. In addition, the device displays spot information and reviews received from the server in the local language and uses a real-time translation function to translate conversations with local people.

[1731] In a virtual space implementation, a device reads a QR code in the virtual space and displays information about a specific tourist spot on a virtual screen. A navigation function is also provided to virtually explore the location based on the displayed information. This allows users to enjoy a realistic travel experience in the virtual space without actually visiting the location.

[1732] Hardware and software used

[1733] Hardware: Server, smart glasses, head-mounted display, GPS receiver

[1734] Software: Natural language processing technology, generative models, databases, real-time translation engines

[1735] The server is built using Python. The program retrieves information about spots and reviews via API and summarizes and translates them using a generative AI model. Qt is used to create a GUI on the device, allowing users to smoothly retrieve information and navigate.

[1736] Specific examples

[1737] For example, when a user scans a specific QR code in the virtual space, detailed information about that tourist spot is instantly displayed. The user can check information such as "You can see a traditional festival at this spot" on the virtual screen. Also, when the user inputs "Take a virtual trip to this spot," the device starts virtual navigation and shows the user the optimal route.

[1738] Prompt Sentence Examples

[1739] "Get information about specific tourist spots and display it as a virtual guide."

[1740] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1741] Step 1:

[1742] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides. The input is data obtained through web scraping and API calls, and the output is organized data stored in a database. This data is later used as training data for generative models. Specifically, it processes data collection from websites using Python's BeautifulSoup and Requests.

[1743] Step 2:

[1744] The server uses the collected data to train a generative AI model. The input is the data collected in step 1, and the output is a model that performs summarization and translation. Specifically, the generative AI model is trained using TensorFlow and PyTorch.

[1745] Step 3:

[1746] The user scans a specific QR code on their device. The input is the image data of the QR code, and the output is the analyzed spot ID. Specifically, the device's camera captures the QR code and decodes it using a library such as OpenCV.

[1747] Step 4:

[1748] After extracting the spot ID, the device sends the ID to the server. The input is the spot ID, and the output is a request to the server. Specifically, it uses the Python Requests library to send the HTTP request.

[1749] Step 5:

[1750] The server retrieves the corresponding spot information from the database based on the received spot ID. The input is the spot ID and the output is the spot information. Specifically, it executes an SQL query to retrieve the information from the database.

[1751] Step 6:

[1752] The server summarizes the acquired spot information using a generative AI model and automatically translates it into the local language. The input is the spot information, and the output is the summarized and translated information. Specifically, data is input into a pre-trained generative AI model, and summary and translation results are obtained.

[1753] Step 7:

[1754] The server sends summarized and translated spot information to the terminal. The input is the summarized and translated information, and the output is the response to the terminal. Specifically, the information is returned as an HTTP response.

[1755] Step 8:

[1756] The terminal displays the received information in a GUI. The input is summarized and translated spot information, and the output is a visual display for the user. Specifically, the information is displayed using a GUI framework such as Qt.

[1757] Step 9:

[1758] The device uses GPS information to display spot information on a map and provide navigation. The input is the current location's GPS data, and the output is a navigation route on the map. Specifically, it calculates the route using Google Maps API and displays it on the map.

[1759] Step 10:

[1760] When a user speaks or inputs text, the device translates it in real time. The input is the user's voice or text, and the output is the translated text or voice. Specifically, the device performs text translation using the Google Translate API or similar.

[1761] Step 11:

[1762] In the virtual space, the device reads the QR code and displays information about a specific tourist spot on a virtual screen. The input is the QR code data, and the output is the spot information displayed on the virtual screen. Specific operations utilize QR code scanning technology and a virtual reality framework.

[1763] Step 12:

[1764] The device supports users in navigating within the virtual space based on the displayed information. The input is spot information and user operations, and the output is a navigation route within the virtual space. Specifically, the navigation system within the virtual space is implemented using an engine such as Unity.

[1765] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1766] The present invention relates to a guide system that recognizes the emotions of travelers and combines an emotion engine to optimize the travel experience. Specific examples will be described below.

[1767] Server Processing

[1768] Data collection and model generation

[1769] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, stores this data in a database, and trains the generative model using natural language processing technology to automatically summarize the information about tourist spots and reviews.

[1770] Processing user requests

[1771] When a request for spot information is received from a device, the server analyzes the request, retrieves the relevant spot information from the database, and summarizes and translates it using a generative model.

[1772] Emotion engine processing

[1773] The emotion engine embedded in the server analyzes the voice and text data received from the user to recognize the user's emotion, which is then taken into account in other processing steps.

[1774] Translating and summarizing reviews

[1775] When a user requests review information, the server retrieves the review data from the database, translates it using an automatic translation engine, and then summarizes it using a generative model.The server then customizes the content by reflecting the user's emotional information and sends it to the device.

[1776] Information distribution and suggestions

[1777] The emotion engine recognizes the user's emotions and optimizes the suggested spots and activities. If the user is under stress, the system prioritizes the display of information about relaxing spots.

[1778] Terminal handling

[1779] QR code reading

[1780] When a user reads the QR code, the device analyzes the code data and extracts the spot ID, which is then sent to the server as a request.

[1781] Map display and navigation

[1782] The device displays the spot information received from the server on a map and displays the optimal route from the user's current location to the spot. When the user selects a specific spot, the device begins navigation based on the selection.

[1783] Real-time translation and emotion recognition

[1784] The device receives information about spots and reviews from the server and displays them in the local language, and uses an emotion engine to recognize emotions based on the user's voice and text input. The recognized emotion information is sent to the server and reflected in processing.

[1785] Specific examples

[1786] For example, when a user visits a tourist spot, they can scan the QR code of a specific restaurant. The device extracts the spot ID and sends a request to the server. The server summarizes and translates the acquired restaurant information and presents it to the user. If the server determines that the user's current emotional state is stressful, it will also suggest information about relaxing spots.

[1787] Furthermore, when users search for nearby tourist attractions in the map app and select a specific spot, the device will calculate the optimal route and display it on the map. If users want to converse with local people, real-time translation is provided through voice input. Voice feedback based on emotions is also provided, providing a better travel experience.

[1788] In this way, the present invention makes it possible to recognize the user's emotions, smoothly obtain information across language barriers, and optimize the travel experience.

[1789] The processing flow will be explained below.

[1790] Server Processing

[1791] Step 1:

[1792] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, and stores the collected data in a database.

[1793] Step 2:

[1794] The server uses the information stored in the database to train a generative model, which uses natural language processing techniques to automatically summarize spot information and reviews.

[1795] Step 3:

[1796] When a request for spot information is received from a terminal, the server analyzes the request and retrieves the corresponding spot information from the database.

[1797] Step 4:

[1798] The server passes the acquired spot information to the generative model, which summarizes and translates it into the local language. The translated and summarized information is then sent to the device.

[1799] Step 5:

[1800] When a user submits a review request, the server retrieves the review data from the database and translates it into major international languages ​​using an automatic translation engine.Then, a generative model summarizes the review information and customizes it based on the user's sentiment information.

[1801] Step 6:

[1802] The emotion engine built into the server analyzes the voice and text data received from the user and recognizes the user's emotions. The recognized emotion information is reflected in information processing and presentation.

[1803] Step 7:

[1804] Based on the user's emotions recognized by the emotion engine, the server optimizes and suggests information on spots and activities suitable for the traveler. Also, if the user is feeling stressed, it will prioritize providing information on spots where they can relax.

[1805] Terminal handling

[1806] Step 1:

[1807] The user scans the QR code at a tourist spot, and the device analyzes the QR code data to extract the spot ID.

[1808] Step 2:

[1809] The terminal sends a request for spot information to the server based on the analyzed spot ID.

[1810] Step 3:

[1811] The spot information received from the server is displayed in the local language on the terminal's GUI, and the user can check the displayed spot information.

[1812] Step 4:

[1813] The user opens a map app and searches for a specific spot. The device acquires the user's current location via GPS and displays the spot information received from the server on a map.

[1814] Step 5:

[1815] When the user selects a specific spot, the device calculates the optimal route and displays route guidance on the map.

[1816] Step 6:

[1817] The terminal displays the reviews received from the server to the user in the local language, with the reviews automatically translated and summarized.

[1818] Step 7:

[1819] When the user expresses their emotions through voice or text input, the device sends this input data to the server.

[1820] Step 8:

[1821] Based on the emotional feedback received from the server in real time, the device will suggest spot information and activities that correspond to the user's emotions.

[1822] User operation example

[1823] Step 1:

[1824] The user scans the QR code at a tourist spot and sends a request to the server to display the information on the device.

[1825] Step 2:

[1826] The server summarizes the spot information using a generative model, translates it, and sends it to the device, where the user can view the spot information in their local language.

[1827] Step 3:

[1828] If the user wants to check the reviews, they tap the reviews tab to send a request. The server translates and summarizes the reviews and sends them to the device.

[1829] Step 4:

[1830] Users use a map app to search for nearby tourist spots and select a specific spot, and the device calculates the optimal route and displays navigation.

[1831] Step 5:

[1832] When users converse with local people, they use the voice input function. The device sends the input speech to the server and displays the translation results in real time. The emotion engine also analyzes the user's emotions and provides customized feedback based on the results.

[1833] Example 2

[1834] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1835] Conventional guide systems do not adequately consider traveler emotions or language barriers, resulting in suboptimal travel experiences. Travelers often find themselves in situations where they feel stressed or are unable to obtain sufficient information due to not understanding the local language, resulting in lower travel satisfaction.

[1836] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1837] In this invention, the server includes means for automatically summarizing spot information to be presented to travelers using a generative model, means for translating the summarized spot information into a local language, means for collecting word-of-mouth information corresponding to multiple languages ​​and automatically translating and summarizing it, emotion analysis means for recognizing emotions from a user's voice or text input, and suggestion means for optimizing the spot information and suggestions based on the emotion analysis results. This enables travelers to smoothly obtain information across language barriers and receive optimal suggestions according to their emotions, thereby improving satisfaction with their travel experience.

[1838] A "generative model" is a machine learning algorithm that automatically summarizes information based on collected data.

[1839] A "local language" is a language commonly spoken in the area the traveller is visiting.

[1840] "Word-of-mouth information" refers to data in which users write their ratings and opinions about certain spots or services.

[1841] "Machine translation" refers to the technology of automatically converting text between multiple languages.

[1842] "Emotion analysis" is a technology that analyzes a user's voice and text data and recognizes their emotions.

[1843] "Suggestion means" refers to a function that suggests optimal spot information and activities to users based on the results of emotion analysis.

[1844] "Code reading" refers to the technology of reading code information such as QR codes using a device's camera.

[1845] "Navigation" refers to a function that provides guidance on a route from the user's current location to a destination.

[1846] A "user request" is a request for information made by a user to a server via a terminal.

[1847] A "database" is a system that systematically stores and manages collected spot information and reviews.

[1848] The present invention relates to a guide system for recognizing a traveler's emotions and optimizing the travel experience. Specific examples will be described below.

[1849] Server Processing

[1850] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides, and stores the data in a database. This is done using web scraping technology, such as Python's BeautifulSoup library. The collected data is then stored in a database such as MongoDB.

[1851] The data is then trained using a generative AI model with natural language processing techniques (e.g., the BERT model) to automatically summarize information about attractions and reviews, ensuring that the information provided to travelers is concise and easy to understand.

[1852] When a request for specific spot information is received from a device, the server analyzes the request and retrieves the corresponding spot information from the database, summarizes this information using a generative AI model, and then translates it into the local language using tools such as Google Translate API.

[1853] The server is also equipped with an emotion engine that analyzes voice and text data received from users. For example, if a user says, "I've been feeling a bit stressed lately," the emotion analysis module analyzes this information and recognizes the user's emotion. This emotion information is used in subsequent processing steps.

[1854] When a user requests review information, the server retrieves the information from the database, translates it using an automatic translation engine, and summarizes it using a generative AI model. This summarized information is then customized to reflect the user's emotional information and sent to the device.

[1855] Based on the user's emotions recognized by the emotion engine, the server optimizes the recommendations of spot information and activities. For example, if the server recognizes that the user is under stress, it will prioritize providing information on relaxation spots.

[1856] Terminal handling

[1857] When a user reads the QR code, the device analyzes the code data to extract the spot ID and sends it as a request to the server. The spot information sent from the server is displayed on the device's map app, and the optimal route from the user's current location to the spot is shown.

[1858] The device's emotion engine analyzes the user's voice and text inputs to recognize the user's emotions. This recognized emotion information is sent to the server and reflected in further processing. For example, if a user voice-inputs "I'm looking for a good restaurant," the emotion engine will suggest the most suitable restaurant based on the information analyzed.

[1859] Specific examples

[1860] For example, when a user visits a tourist spot, they scan a QR code for a specific restaurant. The device extracts the spot ID and sends a request to the server. The server summarizes and translates the acquired restaurant information and presents it to the user. If the server determines that the user's current emotional state is stressful, it will also suggest information about relaxing spots.

[1861] Furthermore, when users search for nearby tourist attractions in the map app and select a specific spot, the device will calculate the optimal route and display it on the map. If users want to converse with local people, the device's voice input function will provide real-time translation and emotional feedback, providing a better travel experience.

[1862] Prompt Sentence Examples

[1863] "I want to see summaries of reviews of tourist spots I want to visit when I travel."

[1864] "Please suggest places where I can relax in Tokyo."

[1865] In this way, the present invention makes it possible to recognize the user's emotions, smoothly obtain information across language barriers, and optimize the travel experience.

[1866] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1867] Step 1:

[1868] Data collection and model generation

[1869] Subject: Server

[1870] The server collects information about attractions and reviews from travel sites and tourist guides using web scraping technology with Python's BeautifulSoup library. The collected data is then stored in a database such as MongoDB. The server then uses a generative AI model, such as the BERT model, to train this data and automatically summarize the information about attractions and reviews.

[1871] Input: Spot information and reviews collected from websites

[1872] Output: Automatically summarized information about places and reviews

[1873] Step 2:

[1874] Processing user requests

[1875] Subject: Server

[1876] When a request for specific spot information is received from a device, the server analyzes the request and retrieves the corresponding spot information from the database. It then summarizes it using a generative AI model and translates it into the local language using a Google Translate API or similar. The translated information is then sent back to the device.

[1877] Input: User request for spot information

[1878] Output: Translated spot information

[1879] Step 3:

[1880] Emotion engine processing

[1881] Subject: Server

[1882] The server analyzes the voice and text data received from the user through the emotion analysis module to recognize the user's emotion. This emotion information is stored in a database and taken into account in subsequent processing steps.

[1883] Input: User voice or text data

[1884] Output: Recognized user emotion information

[1885] Step 4:

[1886] Translating and summarizing reviews

[1887] Subject: Server

[1888] When a user requests review information, the server retrieves the corresponding review data from the database, translates it using an automatic translation engine, and then summarizes it using a generative AI model. The resulting information is customized based on the user's emotional information and sent to the device.

[1889] Input: User review request

[1890] Output: Translated and summarized reviews

[1891] Step 5:

[1892] Information distribution and suggestions

[1893] Subject: Server

[1894] Based on the results of the emotion analysis, the server optimizes the information on spots and activities suggested to users. For example, if the server detects that the user is feeling stressed, it will prioritize information on spots where users can relax.

[1895] Input: Recognized user emotion information

[1896] Output: Optimized spot information and recommendations

[1897] Step 6:

[1898] QR code reading

[1899] Subject: Terminal

[1900] When a user reads the QR code, the device analyzes the code data to extract the spot ID, which is then sent to the server as a request.

[1901] Input: QR code scanned by the user

[1902] Output: Spot ID sent to the server

[1903] Step 7:

[1904] Map display and navigation

[1905] Subject: Terminal

[1906] The device displays the spot information received from the server on a map and shows the optimal route from the user's current location to the spot. When the user selects a specific spot, navigation begins.

[1907] Input: Spot information received from the server

[1908] Output: Points of interest on a map and a navigation route

[1909] Step 8:

[1910] Real-time translation and emotion recognition

[1911] Subject: Terminal

[1912] The device's emotion engine analyzes the user's voice and text input and recognizes emotions. The recognized emotion information is sent to the server and reflected in processing. In addition, the user's conversation information is translated in real time and returned as text or voice.

[1913] Input: User voice or text input

[1914] Output: Recognized emotion information and translation results

[1915] This is the specific processing flow of this system. In this way, it is possible to take into account the user's feelings and provide a smooth travel experience that transcends language barriers.

[1916] (Application example 2)

[1917] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1918] Conventional guide systems have difficulty providing information that is sensitive to travelers' emotions, resulting in travel experiences that are not tailored to individual emotions and circumstances. Furthermore, there are issues with multilingual support, real-time navigation, and review summarization and translation, creating a need for more highly personalized experiences. The goal of this project is to resolve these issues and provide optimal information based on travelers' emotions, improving their experiences.

[1919] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1920] In this invention, the server includes means for automatically summarizing spot information to be presented to travelers using a generative model, means for translating the summarized spot information into a local language, means for collecting word-of-mouth information corresponding to multiple languages ​​and automatically translating and summarizing it, means for displaying specific spot information on a map in response to a user request, means for reading QR codes that allow the user to obtain detailed spot information, means for analyzing the user's emotions and suggesting optimal spot information and activities in accordance with the emotions, and means for personalizing and suggesting specific dishes and menus based on the user's emotion data. This makes it possible to provide personalized information in accordance with the travelers' emotions and improve their experience.

[1921] A "generative model" is an algorithm that automatically summarizes and translates collected data using natural language processing technology.

[1922] "Spot information" is detailed information about tourist destinations and activities, and is data about places that travelers visit.

[1923] A "local language" is a language commonly spoken in the region or country the traveler is visiting.

[1924] "Reviews" are reviews and opinions posted by other travelers about a particular place or activity.

[1925] To "summarize" means to briefly summarize the main points or content of information.

[1926] "Translating" means converting information written in one language into another language.

[1927] A "user request" is a request or inquiry from a traveler to the system for specific information.

[1928] "Displaying on a map" means visually showing a specific spot using geographical location information.

[1929] A "QR code reading means" is a function that scans a QR code using a smartphone or dedicated device and analyzes its contents.

[1930] "Analyzing user emotions" means assessing the emotions and psychological state of the traveler at that time based on voice and text data obtained from the traveler.

[1931] "Suggesting the best spot information and activities" means guiding users to the most suitable tourist spots and experiences based on their emotions and situation.

[1932] "Personalization" means tailoring the information and services provided to suit the preferences and feelings of individual users.

[1933] The present invention provides a guide system for assisting travelers in obtaining spot information and optimizing their travel experience. Specific embodiments of the system will be described below.

[1934] Server Processing

[1935] Data collection and model generation

[1936] The server periodically collects information about attractions and reviews from travel sites and tourist guides, and stores this data in a database. A generative model (such as OpenAI's GPT-4) is trained using natural language processing technology to automatically summarize the information about attractions and reviews.

[1937] Processing user requests

[1938] When a request for spot information is received from a user device, the server analyzes the request, retrieves the relevant spot information from the database, and summarizes and translates it using a generative model, allowing the user to obtain concise and easy-to-understand information.

[1939] Emotion engine processing

[1940] The server-based emotion engine (such as Google Cloud's Dialogflow) analyzes the voice and text data received from the user and recognizes the user's emotions. This emotion information is then taken into account in other processing steps to suggest the most suitable spots and activities for the user.

[1941] Translating and summarizing reviews

[1942] When a user requests review information, the server retrieves the review data from the database, translates it using an automatic translation engine (such as DeepL), and then summarizes it using a generative model.The server then customizes the content by reflecting the user's emotional information and sends it to the user's device.

[1943] Information distribution and suggestions

[1944] The emotion engine recognizes the user's emotions and optimizes the recommendations for spots and activities. Also, if the user is under stress, the system prioritizes the display of information about spots where they can relax. Furthermore, the system personalizes and suggests specific dishes and menus based on the user's emotional data.

[1945] Terminal handling

[1946] QR code reading

[1947] When a user reads the QR code, the device analyzes the code data and extracts the spot ID, which is then sent to the server as a request.

[1948] Map display and navigation

[1949] The device displays the spot information received from the server on a map and displays the optimal route from the user's current location to the spot. When the user selects a specific spot, the device begins navigation based on the selection.

[1950] Real-time translation and emotion recognition

[1951] The device receives information about spots and reviews from the server and displays them in the local language, and uses an emotion engine to recognize emotions based on the user's voice and text input. The recognized emotion information is sent to the server and reflected in processing.

[1952] Specific examples

[1953] For example, when a user visits a tourist spot, they scan a QR code for a specific restaurant. The device extracts the spot ID and sends a request to the server. The server summarizes and translates the acquired restaurant information and presents it to the user. If the server determines that the user's current emotions are stressful, it will also suggest information about relaxing spots. Specific dishes and menus can also be personalized and suggested based on the user's emotional data.

[1954] Furthermore, when users search for nearby tourist attractions in the map app and select a specific spot, the device will calculate the optimal route and display it on the map. If users want to converse with local people, real-time translation is provided through voice input. Voice feedback based on emotions is also provided, providing a better travel experience.

[1955] An example of a prompt might be:

[1956] I'm feeling very tired today. Can you recommend something to calm me down?

[1957] In this way, the present invention makes it possible to recognize the user's emotions, smoothly obtain information across language barriers, and optimize the travel experience.

[1958] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1959] Step 1:

[1960] The server periodically collects information about tourist spots and reviews from travel sites and tourist guides.

[1961] Input: Data from travel sites and tourist guides

[1962] Data processing: continually storing new information in the database

[1963] Output: The latest spot information and reviews are stored in the database.

[1964] Step 2:

[1965] The server uses a generative model (e.g., GPT-4) to train the collected spot information and reviews, generating a model with automatic summarization and translation capabilities.

[1966] Input: Spot information and reviews stored in the database

[1967] Data Computing: Training with Natural Language Processing Techniques

[1968] Output: A generative model that automatically summarizes and translates

[1969] Step 3:

[1970] A request for spot information is sent from the user terminal to the server.

[1971] Input: User request for spot information

[1972] Data processing: Analysis of request content

[1973] Output: Search criteria for spot information based on the request

[1974] Step 4:

[1975] The server retrieves the relevant spot information from the database and summarizes and translates it using a generative model.

[1976] Input: Search criteria for spot information based on your request

[1977] Data calculation: Summarization and translation

[1978] Output: Summary and translated spot information

[1979] Step 5:

[1980] The server analyzes the voice and text data received from the user terminal using an emotion engine (e.g., Dialogflow) to recognize the user's emotions.

[1981] Input: Voice or text input from the user

[1982] Data Computing: Emotion Recognition Processing

[1983] Output: User's emotional information

[1984] Step 6:

[1985] The server suggests optimal spot information and activities based on the user's emotional information recognized by the emotion engine. If the user is experiencing a certain type of stress, the server will prioritize suggesting information about relaxing spots.

[1986] Input: User's emotional information

[1987] Data Computation: Customized Information Suggestion Based on Emotional Information

[1988] Output: Personalized spot suggestions

[1989] Step 7:

[1990] The server retrieves the user-requested review information from the database, translates it using an automatic translation engine (e.g., DeepL), and then summarizes it using a generative model.

[1991] Input: User review request

[1992] Data Computing: Automatic Translation and Summarization

[1993] Output: Summary and translated reviews

[1994] Step 8:

[1995] The terminal reads the QR code, analyzes the code data to extract the spot ID, and sends it to the server.

[1996] Input: Spot QR code

[1997] Data processing: QR code analysis and spot ID extraction

[1998] Output: Spot ID

[1999] Step 9:

[2000] The terminal displays the spot information received from the server on a map and displays the optimal route from the user's current location to the spot.

[2001] Input: Spot information from the server

[2002] Data calculation: Calculating the optimal route on map data

[2003] Output: Navigation information on a map

[2004] Step 10:

[2005] The device displays the spot information and reviews received from the server in the local language and uses an emotion engine to recognize emotions based on the user's voice and text input. The recognized emotion information is sent to the server and reflected in processing.

[2006] Input: Voice input or text input from the user, spot information and reviews from the server

[2007] Data processing: Real-time translation and emotion recognition

[2008] Output: Display information in local language, send emotion information to server

[2009] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2010] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2011] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2012] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2013] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2014] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2015] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2016] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2017] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2018] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2019] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2020] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2021] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2022] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2023] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2024] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2025] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2026] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2027] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2028] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2029] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2030] The following is further disclosed regarding the above embodiment.

[2031] (Claim 1)

[2032] A guide system for assisting travelers in obtaining spot information,

[2033] A means for automatically summarizing spot information to be presented to travelers using a generative model;

[2034] A means of translating summarized spot information into local languages;

[2035] A means to collect, automatically translate, and summarize reviews in multiple languages,

[2036] a means for displaying specific spot information on a map in response to a user request;

[2037] A QR code reading means for a user to obtain detailed information about a spot;

[2038] A system including:

[2039] (Claim 2)

[2040] 2. The system according to claim 1, further comprising a function of calculating an optimal route based on the traveler's current location and displaying navigation to the spot.

[2041] (Claim 3)

[2042] 10. The system of claim 1, further comprising means for enabling a user to translate conversations with local people in real time via voice or text input.

[2043] "Example 1"

[2044] (Claim 1)

[2045] A guidance system for assisting travelers in obtaining geographical feature information,

[2046] a means for automatically summarizing geographic feature information to be presented to a traveler using a machine learning model;

[2047] a means for translating the summarized geographic feature information into a local language;

[2048] A means to collect, automatically translate, and summarize review information in multiple languages;

[2049] means for displaying specific geographic feature information on a map in response to a user's request;

[2050] code reading means for a user to obtain detailed information about the geographic feature;

[2051] A system including:

[2052] (Claim 2)

[2053] 10. The system of claim 1, further comprising the capability of calculating an optimal route based on the traveler's location information and displaying route guidance to the geographic feature.

[2054] (Claim 3)

[2055] 10. The system of claim 1, further comprising means for enabling a user to translate conversations with local people in real time via voice or text input.

[2056] "Application Example 1"

[2057] (Claim 1)

[2058] A guide system for assisting travelers in obtaining spot information,

[2059] A means for automatically summarizing spot information to be presented to travelers using a generative model;

[2060] A means of translating summarized spot information into local languages;

[2061] A means to collect, automatically translate, and summarize reviews in multiple languages,

[2062] a means for displaying specific spot information on a map in response to a user request;

[2063] A QR code reading means for a user to obtain detailed information about a spot;

[2064] A means for displaying information about specific tourist spots on a virtual screen in a virtual space;

[2065] A navigation method for virtually exploring the area based on the displayed information;

[2066] A system including:

[2067] (Claim 2)

[2068] 2. The system according to claim 1, further comprising a function of calculating an optimal route based on the traveler's current location and displaying navigation to the spot.

[2069] (Claim 3)

[2070] 10. The system of claim 1, further comprising means for enabling a user to translate conversations with local people in real time via voice or text input.

[2071] "Example 2: Combining Emotion Engines"

[2072] (Claim 1)

[2073] A guide system for assisting travelers in obtaining spot information,

[2074] A means for automatically summarizing spot information to be presented to travelers using a generative model;

[2075] A means of translating summarized spot information into local languages;

[2076] A means to collect, automatically translate, and summarize reviews in multiple languages,

[2077] a means for displaying specific spot information on a map in response to a user request;

[2078] A code reading means for a user to obtain detailed information about a spot;

[2079] emotion analysis means for recognizing emotions from a user's voice or text input;

[2080] A proposal method that optimizes spot information and proposal content based on the results of sentiment analysis;

[2081] A system including:

[2082] (Claim 2)

[2083] 2. The system according to claim 1, further comprising a function of calculating an optimal route based on the traveler's current location and displaying navigation to the spot.

[2084] (Claim 3)

[2085] 10. The system of claim 1, further comprising means for enabling a user to translate conversations with local people in real time via voice or text input.

[2086] "Application example 2 when combining emotion engines"

[2087] (Claim 1)

[2088] A guide system for assisting travelers in obtaining spot information,

[2089] A means for automatically summarizing spot information to be presented to travelers using a generative model;

[2090] A means of translating summarized spot information into local languages;

[2091] A means to collect, automatically translate, and summarize reviews in multiple languages,

[2092] a means for displaying specific spot information on a map in response to a user request;

[2093] A QR code reading means for a user to obtain detailed information about a spot;

[2094] A method to analyze user emotions and suggest the best spot information and activities based on those emotions,

[2095] A way to personalize and suggest specific dishes or menu items based on the user's emotional data, and

[2096] A system including:

[2097] (Claim 2)

[2098] 2. The system according to claim 1, further comprising a function of calculating an optimal route based on the traveler's current location and displaying navigation to the spot.

[2099] (Claim 3)

[2100] 10. The system of claim 1, further comprising means for enabling a user to translate conversations with local people in real time via voice or text input. [Explanation of symbols]

[2101] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A guide system for assisting travelers in obtaining spot information, A means for automatically summarizing spot information to be presented to travelers using a generative model; A means of translating summarized spot information into local languages; A means to collect, automatically translate, and summarize reviews in multiple languages, a means for displaying specific spot information on a map in response to a user request; A QR code reading means for a user to obtain detailed information about a spot; A system including:

2. The system according to claim 1, further comprising a function of calculating an optimal route based on the traveler's current location and displaying navigation to the spot.

3. 10. The system of claim 1, further comprising means for enabling a user to translate conversations with local people in real time through voice or text input.

Images