system

Abstract

A system is provided.SOLUTION: The system includes a means for inputting profile information by a user through an installed application and transmitting the information to a server, a means for confirming the present location of the user by the server on the basis of GPS data, a means for generating local guidance data on the basis of the present location confirmed by the server and transmitting it to a terminal, and a means for displaying and reproducing the local guidance data received by the terminal to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, there is a problem that travelers have few opportunities to enjoy the attractions of rural areas outside of cities. It is also difficult to provide the enjoyment of travel to people who have difficulty traveling to these areas, especially the elderly. Furthermore, as local governments lack the financial resources to build new tourist spots, there is a growing need to effectively utilize existing local customs and history to attract tourists. There is a need for ways to enhance the local tourist experience while also providing similar experiences in remote areas. [Means for solving the problem]

[0005] This invention provides a means for users to input profile information through an application they have installed and send that information to a server. The server verifies the user's current location based on GPS data, generates local guidance data based on the confirmed current location, and sends it to the terminal. The terminal then displays and plays the received local guidance data to provide the user with local information.

[0006] The system further includes means for the server to generate questions about the local history and culture based on the mode selected by the user and send them to the terminal. The terminal presents the questions to the user, collects the user's answers, and sends them to the server. The server evaluates the user's answers, generates the next information, and sends it to the terminal. The terminal displays information about the next spot and directions to that location to the user, allowing the user to enjoy sightseeing in a game-like manner.

[0007] It also provides a means for users to input memories of past travels or upload photos and notes. The server analyzes the uploaded data and generates content that recreates the virtual trip. The device displays and plays the generated virtual travel content to the user, allowing them to relive past trips from the comfort of their own home. This makes it possible to provide the enjoyment of travel to people who have difficulty visiting the destination, such as the elderly.

[0008] "User" refers to an individual who uses the system.

[0009] "Server" refers to a computer system that stores and processes data and provides information in cooperation with user terminals.

[0010] A "terminal" is a device that a user can carry around and that can display and operate, and specifically refers to a smartphone, tablet, etc.

[0011] "Application" refers to software that a user installs on a terminal and uses.

[0012] "Profile Information" refers to data regarding personal information and preferences that a User registers through an Application.

[0013] "GPS Data" means location information obtained by the Global Positioning System.

[0014] "Local guide data" refers to data such as text, audio, and images that contain information about tourist destinations and historical spots.

[0015] "Questions" refer to quizzes and puzzles that are generated by the server based on local history and culture and presented to users.

[0016] "Answer" refers to the content of the answer given by the user to the question presented.

[0017] "Virtual travel content" refers to videos and slideshows of virtual tours recreated by AI based on memories and data from past trips.

[0018] "Reproduction" refers to virtually reconstructing something based on specific memories and data from the past.

[0019] A "mystery solving game" is an entertaining course in which you are guided to the next spot by solving questions about the history and culture of a tourist destination.

[0020] A "mode" refers to the types of functions and operation methods that a user can select in an application. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] Overall flow

[0043] This invention is a system that makes it easier for travelers to enjoy the attractions of rural areas outside of cities, enriching the local tourist experience while also providing a similar experience in remote areas. Furthermore, it also provides the enjoyment of travel to elderly people who find it difficult to travel to the local area. The system supports the tourist experience by linking the user's smartphone with a server.

[0044] User registration and information management

[0045] 1. The user installs a dedicated application on their smartphone.

[0046] 2. The user launches the application and enters profile information such as name, age, and historical topics of interest.

[0047] 3. The device sends the entered information to the server.

[0048] 4. The server stores the received information in a database and sends a confirmation message to the user.

[0049] Automatic guidance upon arrival at destination

[0050] 1. The device periodically obtains the user's current location using GPS.

[0051] 2. The device sends the acquired location data to the server.

[0052] 3. The server analyzes the user's current location and verifies that it is one of the tourist destinations set.

[0053] 4. The server generates local guidance data and sends it to the terminal.

[0054] 5. The terminal displays and plays the received guidance data (audio guide, text, images) to the user.

[0055] Sightseeing with a Game-like Feel

[0056] 1. The user selects a puzzle game or quiz mode within the app.

[0057] 2. The server generates questions about local history and culture according to the selected mode and sends them to the terminal.

[0058] 3. The device presents the user with a question, collects the user's answers, and sends them to the server.

[0059] 4. The server evaluates the user's answer, generates information about the next spot, and sends it to the device.

[0060] 5. The device displays information about the next spot and provides directions to the location to the user.

[0061] Relive your travel memories

[0062] 1. Users enter memories of past trips or upload photos and notes within the app.

[0063] 2. The server parses the uploaded data.

[0064] 3. The server generates content (in the form of videos or slideshows) that recreates the virtual journey.

[0065] 4. The terminal displays and plays the generated virtual travel content to the user.

[0066] Virtual travel at home

[0067] 1. The user selects the "Virtual Travel" mode within the app.

[0068] 2. The server collects information on the desired travel destinations based on the user's selection and generates a virtual sightseeing tour.

[0069] 3. The device streams the generated virtual tour and provides it to the user.

[0070] Specific examples

[0071] For example, consider a case where a user visits a historical spot in Nara. When the user arrives at the location, the smartphone automatically displays an audio guide and text about the Nara period. Furthermore, if the user selects the quiz mode, the smartphone asks the question, "What is a representative building from the Nara period?" If the answer is correct, directions to the next spot will be displayed.

[0072] If a user wants to relive a past trip to Kyoto, they can upload past photos to the app. The server analyzes the photos and generates a video that virtually recreates the trip to Kyoto. The user can then enjoy the video at home.

[0073] This will enable people to gain a deeper understanding of local history and culture and experience the joy of travel without having to go to the actual place. Virtual travel will also be a new way to provide the joy of travel to elderly people and those with mobility difficulties.

[0074] The processing flow will be explained below.

[0075] User registration and information management

[0076] Step 1:

[0077] The user downloads and installs the application on their smartphone.

[0078] Step 2:

[0079] A user launches the application and enters profile information such as name, age, and historical topics of interest.

[0080] Step 3:

[0081] The device checks the entered information and temporarily stores it.

[0082] Step 4:

[0083] The device sends the profile information to the server.

[0084] Step 5:

[0085] The server analyzes the received information and stores it in a database.

[0086] Step 6:

[0087] The server sends a message to the terminal indicating that the information has been saved.

[0088] Step 7:

[0089] The terminal displays a confirmation message to the user.

[0090] Automatic guidance upon arrival at destination

[0091] Step 1:

[0092] The device periodically obtains the user's current location using GPS.

[0093] Step 2:

[0094] The terminal transmits the obtained current location data to the server.

[0095] Step 3:

[0096] The server analyzes the user's current location and checks whether it is the tourist spot you have set.

[0097] Step 4:

[0098] If the server determines that the area is a tourist destination, it generates local guide data for the area.

[0099] Step 5:

[0100] The server transmits local guide data to the terminal.

[0101] Step 6:

[0102] The terminal displays the received local guidance data to the user, and the audio guide is automatically played.

[0103] Sightseeing with a Game-like Feel

[0104] Step 1:

[0105] The user selects a puzzle game or quiz mode within the app.

[0106] Step 2:

[0107] The terminal transmits the user's selection information to the server.

[0108] Step 3:

[0109] The server generates questions about local history and culture depending on the mode selected.

[0110] Step 4:

[0111] The server generates questions and sends them to the device.

[0112] Step 5:

[0113] The terminal presents the problem to the user.

[0114] Step 6:

[0115] The user answers the question.

[0116] Step 7:

[0117] The terminal sends the user's answer to the server.

[0118] Step 8:

[0119] The server evaluates the user's answer and determines whether it is correct.

[0120] Step 9:

[0121] The server generates information about the next spot and sends it to the terminal.

[0122] Step 10:

[0123] The device displays information about the next spot and provides directions to the location to the user.

[0124] Relive your travel memories

[0125] Step 1:

[0126] Users can enter memories of past trips within the app, or upload photos and notes.

[0127] Step 2:

[0128] The device sends the uploaded information to the server.

[0129] Step 3:

[0130] The server parses the data it receives.

[0131] Step 4:

[0132] The server generates content that recreates the virtual journey.

[0133] Step 5:

[0134] The server transmits the generated content to the terminal.

[0135] Step 6:

[0136] The terminal displays and plays the virtual travel content to the user.

[0137] Virtual travel at home

[0138] Step 1:

[0139] The user selects the "Virtual Travel" mode within the app.

[0140] Step 2:

[0141] The terminal transmits the selection information to the server.

[0142] Step 3:

[0143] The server generates a virtual sightseeing tour based on information about the desired travel destination.

[0144] Step 4:

[0145] The server transmits the generated virtual tour data to the terminal.

[0146] Step 5:

[0147] The terminal streams the virtual tour and provides it to the user.

[0148] Example 1

[0149] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0150] Conventional tourism systems often lack tourist information when visiting rural areas outside of cities, making it difficult to fully enjoy the attractions of those areas. There is also the problem that tourism itself is difficult for elderly people and people with mobility restrictions who have difficulty traveling to those areas. Furthermore, there are limited ways to recreate and re-enjoy memories of past trips. The goal of this project is to solve these problems and make travel and tourism attractive and easy for everyone to enjoy.

[0151] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0152] In this invention, the server includes a means for users to send profile information, a means for checking current location based on GPS data, and a means for generating local guide data based on the current location. This allows users to easily enjoy the attractions of local areas. The server also includes a means for inputting memories of past travels or uploading photos and notes, a means for analyzing the uploaded data and generating content that recreates a virtual travel, and a means for displaying and playing the generated virtual travel content to the user, allowing users to relive past travels. Furthermore, the server includes a means for selecting a "virtual travel" mode, a means for collecting information on desired travel destinations and generating a virtual sightseeing tour, and a means for streaming and providing the generated virtual tour to the user, allowing users who have difficulty visiting the destinations to enjoy the fun of travel.

[0153] "User" refers to a human user who interfaces with the server and terminal.

[0154] "Device" refers to a smartphone, tablet, or other mobile information device on which a user installs and uses an application.

[0155] "Application" refers to a software program that a user installs on a terminal and uses.

[0156] "Profile Information" refers to personal information about you, such as your name, age, and topics of interest.

[0157] "Server" means a remote computer system that processes and stores data received from a client device and executes server-side actions.

[0158] "GPS Data" refers to your location information obtained using the Global Positioning System.

[0159] "Current Location" refers to a user's real-time geographic location as determined based on GPS data.

[0160] "Local guide data" refers to data containing information about a specific tourist destination (audio guide, text, images, etc.).

[0161] "Virtual travel content" refers to content (such as videos or slideshows) that is displayed and played back to users and is generated based on past travel memories and uploaded data.

[0162] "Virtual travel" refers to a sightseeing tour that allows you to collect information about your desired travel destination and experience it virtually without physically going there.

[0163] "Questions" refer to quizzes and puzzles about history and culture that are generated by the server and presented to the user.

[0164] A "mode" refers to a particular type of operation or function that a user selects within an application.

[0165] This invention is a system for enhancing the appeal of visiting regional tourist destinations outside of urban areas. This system links the user's smartphone with a server, aiming to enrich the tourist experience on-site while also providing a similar experience in remote locations. Furthermore, it is possible to provide the enjoyment of travel to elderly users and users with mobility difficulties.

[0166] 1. User registration and information management

[0167] Users first install a dedicated application on their smartphone. They then launch the application and enter their profile information, such as their name, age, and historical topics of interest. The information is then sent from the device to a server, which stores it in a database. This is how the user profile is managed.

[0168] 2. Automatic guidance when arriving at your destination

[0169] The user's device periodically obtains its current location using GPS and sends this data to the server. The server analyzes the current location data and confirms that the user has arrived at the tourist destination. The server generates local guide data about the tourist destination and sends it to the device. The device receives this data and displays and plays audio guides, text, images, etc. to the user. Technologies used include Google Maps API and Azure Cognitive Services.

[0170] 3. Sightseeing like a game

[0171] Within the app, users can select a puzzle game or quiz mode. Depending on the selected mode, the server generates questions about local history and culture and sends them to the device. The device presents the questions to the user and collects their answers. The server evaluates the answers, generates information about the next spot, and sends it to the device. The device then displays information about the next spot and directions to the location to the user. A specific example is the question, "What is a representative building from the Nara period?", and the user inputs the answer.

[0172] 4. Relive your travel memories

[0173] Users can input memories of past travels and upload photos and notes. The server analyzes the uploaded data and generates content that recreates the virtual trip. The device displays and plays the generated virtual trip content to the user. Technologies used include Amazon Rekognition and Adobe Premiere API. An example prompt is, "Users upload photos of travel destinations they have visited in the past, and a video recreating that trip is generated and displayed."

[0174] 5. Virtual travel at home

[0175] Users can select the "Virtual Travel" mode within the app. The server collects information on the desired travel destination based on the user's selection and generates a virtual sightseeing tour. The device streams the generated virtual tour and provides it to the user. The technology used includes the Google Earth API. An example of a specific prompt is, "The user selects the 'Virtual Travel' mode and a virtual tour of Paris begins."

[0176] This makes it possible for the present invention to increase the enjoyment of traveling to rural areas, providing a similar experience from remote locations, and bringing the joy of travel to elderly people and those with mobility difficulties.

[0177] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0178] Step 1:

[0179] The user installs the dedicated application on their smartphone. The user downloads the application from the app store and completes the installation procedure. The application can then be used on the smartphone.

[0180] Step 2:

[0181] A user launches the application and enters profile information (name, age, historical topics of interest). The information entered by the user is temporarily stored on the device. Profile information is constructed based on the input (e.g., name "Taro Tanaka," age "65," interest "ancient history").

[0182] Step 3:

[0183] The device sends the profile information to the server. The input data from the device is transferred to the server via the Internet and received by the server. The received profile information is saved in the server's database. The server generates a save completion message and sends it to the device. The device displays a "Registration Complete" message to the user.

[0184] Step 4:

[0185] The device periodically obtains the user's current location using GPS. The device uses the GPS sensor to collect location information and generate that data. The generated GPS data is stored on the device.

[0186] Step 5:

[0187] The device transmits the acquired current location data to the server. The location information data from the device is transferred to the server and received by the server. The received data is sent to the server's analysis module.

[0188] Step 6:

[0189] The server analyzes the current location data and confirms that the user has arrived at the designated tourist spot. The server compares the location data with tourist spot information in the database to determine whether the user's current location matches the tourist spot. If the location information matches the tourist spot, the server generates local guidance data.

[0190] Step 7:

[0191] The server generates local guidance data (audio guide, text, images) and sends it to the terminal. The generated guidance data is transferred from the server to the terminal. The terminal displays and plays the received local guidance data to the user. The user plays the audio guide and receives the information.

[0192] Step 8:

[0193] The user selects a puzzle game or quiz mode within the app. The user operates the mode selection screen and selects the desired game mode. The selected mode information is saved on the device.

[0194] Step 9:

[0195] The server generates questions about local history and culture according to the selected mode and sends them to the device. The server generates question data based on the game mode and transfers the data to the device. The device presents the questions to the user and collects the user's answers.

[0196] Step 10:

[0197] The terminal sends the user's answer data to the server. The answer data from the terminal is transferred to the server and received by the server. The received data is sent to the server's analysis module.

[0198] Step 11:

[0199] The server evaluates the user's answers, generates information about the next spot, and sends it to the device. The server analyzes the answer data and generates information about the next tourist spot based on the evaluation results. The generated spot information is then transferred to the device.

[0200] Step 12:

[0201] The device displays information about the next spot and guidance to the user. The device receives the spot information and displays it to the user. The user confirms the guidance to the next spot and begins moving.

[0202] Step 13:

[0203] The user inputs memories of past trips or uploads photos and notes. The user inputs past trip information within the app or selects and uploads photo files. The input and uploaded data is saved on the device.

[0204] Step 14:

[0205] The device sends the uploaded data to the server. The uploaded data from the device is transferred to the server and received by the server. The received data is sent to the server's analysis module.

[0206] Step 15:

[0207] The server analyzes the uploaded data and generates virtual travel content. The server processes the data using image analysis technology and generates content for recreating the virtual travel. The generated content is stored on the server.

[0208] Step 16:

[0209] The server sends the virtual travel content to the terminal. The generated content data is transferred from the server to the terminal. The terminal receives the virtual travel content and displays and plays it for the user. The user experiences the virtual travel.

[0210] Step 17:

[0211] The user selects the "Virtual Travel" mode within the app. The user operates the mode selection screen and selects the Virtual Travel mode. The selection information is saved on the device.

[0212] Step 18:

[0213] The server collects information on the desired travel destination based on the user's selection and generates a virtual sightseeing tour. The server extracts the travel destination information from the database and creates a virtual tour based on the collected data. The generated tour information is saved on the server.

[0214] Step 19:

[0215] The server sends the generated virtual tour to the terminal. The tour information data is transferred from the server to the terminal. The terminal receives the virtual tour and starts streaming playback. The user experiences the virtual trip.

[0216] (Application example 1)

[0217] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0218] There is a need to provide tourism experiences for elderly people and travel enthusiasts living in remote areas who find it difficult to visit local sites. However, current systems rely on on-site visits and lack ways to enhance tourism experiences in virtual environments. This invention aims to solve this problem by providing a virtual travel system that allows users to enjoy the charms of local areas from the comfort of their own homes.

[0219] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0220] In this invention, the server includes means for inputting profile information through an application installed by the user and transmitting the information to the server, means for confirming the user's current location based on GPS data, means for generating local guidance data based on the confirmed current location and transmitting it to the terminal, means for the user to select a mode within the application for enjoying a virtual trip at home, means for generating virtual sightseeing tour content based on the selected travel destination and transmitting it to the terminal, and means for displaying and playing the received virtual sightseeing tour content to the user, thereby enabling the user to enjoy an advanced virtual sightseeing experience from the comfort of their own home.

[0221] "Profile Information" refers to personal information such as a User's name, age, and topics of interest that is used by the Application to provide the User with an optimal tourism experience.

[0222] "GPS Data" means location information obtained using the Global Positioning System and is used to determine a user's current location.

[0223] "Local guide data" refers to data that includes information such as audio guides, text, and images about tourist spots that users visit, and is provided to support the user's sightseeing experience.

[0224] "Device" refers to an electronic device used by a user, such as a smartphone, tablet, or head-mounted display, which is used to display and play data sent from the server.

[0225] "Virtual travel" refers to the process by which users virtually visit tourist spots through an application and experience their culture and history without actually traveling.

[0226] "Virtual sightseeing tour content" refers to content such as 3D visuals of tourist spots, audio guides, and text that is generated by the server and sent to the user's terminal, and is the main information that allows the user to experience a virtual trip.

[0227] "Quiz mode" refers to a function that, when selected by the user, causes the application to ask questions about tourist spots, and allows the user to enjoy sightseeing in a game-like manner by answering the questions.

[0228] A "head-mounted display" is a display device worn by the user on the head to display virtual reality (VR) and augmented reality (AR) content, and is used to provide an advanced virtual experience.

[0229] The present invention provides a virtual travel system that allows users to experience the attractions of local areas from the comfort of their own homes. Detailed embodiments are described below.

[0230] System Configuration

[0231] This system consists of an application installed by the user, a terminal (such as a smartphone or a head-mounted display), and a server.

[0232] Registering profile information and acquiring GPS data

[0233] Through the application, users enter profile information such as their name, age, and topics of interest, and submit that information to the server, which stores the user's profile information in a database.

[0234] The device periodically acquires the user's current location using GPS and sends that data to the server, which then analyzes the GPS data to confirm the user's current location.

[0235] Generation and display of on-site guidance data

[0236] The server generates local guidance data based on the user's current location and sends it to the device. The device then displays and plays the received local guidance data to the user as audio guides, text, and images, allowing the user to obtain a wealth of information about the local history and culture.

[0237] Providing virtual travel modes

[0238] Within the application, users can select a mode to enjoy a virtual trip at home. The server generates virtual sightseeing tour content based on the selected travel destination and sends it to the device. The device then displays and plays the received virtual sightseeing tour content to the user as an audio guide or 3D visuals. Specifically, users can experience the virtual trip through a smartphone or head-mounted display.

[0239] Quiz mode available

[0240] The application has a quiz mode. Based on the quiz mode selected by the user, the server generates questions about local history and culture and sends them to the device. The device presents the questions to the user, collects the user's answers, and sends them to the server. The server evaluates the user's answers, generates information about the next spot, and sends it to the device. This allows the user to enjoy sightseeing in a game-like manner.

[0241] Recreating a virtual journey

[0242] Users can input memories of past trips or upload photos and notes. The server analyzes the uploaded data and generates content that recreates a virtual trip. The device displays and plays the generated virtual trip content to the user.

[0243] Hardware and software used

[0244] This system uses the following hardware and software:

[0245] Hardware: Smartphone, Head-Mounted Display (HMD), Server

[0246] Software: Application, server program, HTTP request library (requests), JSON parser

[0247] Examples of concrete examples and prompts

[0248] As a concrete example, consider a scenario in which a user experiences historical spots in Nara from the comfort of their own home. The user launches the app and selects the "Virtual Travel" mode. The server generates an audio guide and text about Nara culture based on the user's profile data and selected travel destination, and sends them to the device along with 3D visuals. The user puts on a head-mounted display and experiences Nara's history in a virtual environment. In addition, in quiz mode, the user is asked the question, "What is a representative building from the Nara period?" and by answering the question, directions to the next spot are displayed.

[0249] Below is an example of a prompt sentence to input to the generative AI model.

[0250] Prompt Sentence Examples

[0251] ---

[0252] We are developing an application that allows users to enjoy virtual travel from the comfort of their own home. The application runs on smartphones and head-mounted displays (HMDs) and provides users with a virtual sightseeing experience related to local culture and history.

[0253] example:

[0254] The user uses the app to experience Nara's historical spots. The app retrieves audio guides and text about Nara based on the user's profile data, and displays them in 3D visuals through the HMD. The app also has a quiz mode, and after the user answers questions, the app guides the user to the next virtual spot.

[0255] Expected output:

[0256] Audio guide about historical spots from the Nara period

[0257] Coding audio files for broadcast

[0258] A quiz about representative buildings from the Nara period

[0259] Directions to the next virtual spot

[0260] Please explain in detail how this application works.

[0261] ---

[0262] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0263] Step 1:

[0264] A user installs the application, enters profile information (such as name, age, and topics of interest), and sends that information to the server, which stores the information in a database and sends a confirmation message to the user.

[0265] Input: User profile information Output: Confirmation message

[0266] Step 2:

[0267] The device periodically acquires GPS data and sends the user's current location to the server. The server analyzes the received GPS data and confirms the user's current location. When the user arrives at the location, the server generates local guidance data and sends it to the device.

[0268] Input: GPS data Output: Local guidance data

[0269] Step 3:

[0270] The device displays and plays the received local information to the user. Specifically, it displays audio guides, text, and images, allowing the user to learn about the history and culture of the area.

[0271] Input: On-site guidance data Output: Displayed / played guidance data

[0272] Step 4:

[0273] The user selects the "Virtual Travel" mode in the application. The terminal sends this selection to the server. The server generates virtual sightseeing tour content based on the selected travel destinations and sends it to the terminal.

[0274] Input: Virtual travel mode selection information Output: Virtual sightseeing tour content

[0275] Step 5:

[0276] The device displays and plays the received virtual sightseeing tour content to the user. When using a head-mounted display, 3D visuals are displayed, allowing the user to experience traveling in a virtual environment.

[0277] Input: Virtual sightseeing tour content Output: Displayed / played tour content

[0278] Step 6:

[0279] The user selects the quiz mode. The terminal transmits this selection information to the server. The server generates questions about the local history and culture and transmits them to the terminal.

[0280] Input: Quiz mode selection information Output: Quiz question data

[0281] Step 7:

[0282] The terminal presents the received quiz question data to the user, collects the user's answers, and sends them to the server. The server evaluates the user's answers, generates guidance data for the next spot, and sends it to the terminal. The terminal displays it to the user.

[0283] Input: User response data Output: Next spot guidance data

[0284] Step 8:

[0285] Users input memories of past trips or upload photos and notes. The device sends this data to the server, which analyzes the uploaded data, generates content that recreates the virtual trip, and sends it to the device.

[0286] Input: Past travel data Output: Virtual travel reenactment content

[0287] Step 9:

[0288] The device displays and plays the received virtual travel recreation content to the user, allowing the user to enjoy memories of past trips from the comfort of their own home.

[0289] Input: Virtual travel content Output: Displayed / played travel content

[0290] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0291] This system makes it easier for travelers to enjoy the attractions of rural areas outside of cities, enriching the local tourist experience while also providing a similar experience in remote locations. It also provides the enjoyment of travel to elderly people who find it difficult to travel to the local area. The system supports the tourist experience by linking the user's smartphone with a server and an emotion engine.

[0292] User registration and information management

[0293] 1. The user installs a dedicated application on their smartphone.

[0294] 2. The user launches the application and enters profile information such as name, age, and historical topics of interest.

[0295] 3. The device checks the entered information and temporarily stores it.

[0296] 4. The device sends the profile information to the server.

[0297] 5. The server analyzes the received information and stores it in a database.

[0298] 6. The server sends a message to the device indicating that the information has been saved.

[0299] 7. The terminal displays a confirmation message to the user.

[0300] Automatic guidance upon arrival at destination

[0301] 1. The device periodically obtains the user's current location using GPS.

[0302] 2. The device sends the location data it has obtained to the server.

[0303] 3. The server analyzes the user's current location and confirms that it is the tourist spot that was set.

[0304] 4. If the server determines that the area is a tourist destination, it generates local guide data for the area.

[0305] 5. The server sends the local guide data to the terminal.

[0306] 6. The device displays the received local guidance data to the user, and the audio guide is automatically played.

[0307] Sightseeing with a Game-like Feel

[0308] 1. The user selects a puzzle game or quiz mode within the app.

[0309] 2. The terminal sends the user's selection information to the server.

[0310] 3. The server generates questions about local history and culture depending on the mode selected.

[0311] 4. The server sends the generated questions to the device.

[0312] 5. The device presents the problem to the user.

[0313] 6. The user answers the question.

[0314] 7. The device sends the user's answer to the server.

[0315] 8. The server evaluates the user's answer and determines whether it is correct.

[0316] 9. The server generates information about the next spot and sends it to the terminal.

[0317] 10. The device displays information about the next spot and directions to the location to the user.

[0318] Relive your travel memories

[0319] 1. Users enter memories of past trips or upload photos and notes within the app.

[0320] 2. The device sends the uploaded information to the server.

[0321] 3. The server analyzes the received data.

[0322] 4. The server generates content that recreates the virtual journey.

[0323] 5. The server sends the generated content to the device.

[0324] 6. The device displays and plays the virtual travel content to the user.

[0325] Virtual travel at home

[0326] 1. The user selects the "Virtual Travel" mode within the app.

[0327] 2. The device sends the selection information to the server.

[0328] 3. The server generates a virtual sightseeing tour based on information about the desired travel destination.

[0329] 4. The server sends the generated virtual tour data to the device.

[0330] 5. The device streams the virtual tour and provides it to the user.

[0331] Optimizing user experience with an emotion engine

[0332] 1. The device analyzes the user's facial expressions and voice using an emotion engine to recognize emotions.

[0333] 2. The server adjusts the local guidance data and the difficulty of the game questions based on the recognized emotion data.

[0334] 3. The server sends the data generated based on the emotion to the device.

[0335] 4. The terminal displays and plays the adjusted guidance data and questions to the user.

[0336] Specific examples

[0337] For example, if a user visits a historical spot in Nara, the smartphone will automatically display an audio guide and text about the Nara period. Furthermore, if the user selects the quiz mode, they will be asked, "What is a representative building from the Nara period?" If they answer correctly, directions to the next spot will be displayed.

[0338] By using the emotion engine, the difficulty of the questions can be adjusted even if the user is tired, for example. Also, if a user wants to relive a past trip to Kyoto, they can upload past photos to the app. The server analyzes the photos and generates a video that virtually recreates the Kyoto trip. The user can then enjoy the video at home.

[0339] This will enable people to gain a deeper understanding of local history and culture and experience the joy of travel without having to actually visit the area. Virtual travel will also be a new way to provide the joy of travel to elderly people and those with mobility difficulties.

[0340] The processing flow will be explained below.

[0341] User registration and information management

[0342] Step 1:

[0343] The user downloads and installs a dedicated application onto their smartphone.

[0344] Step 2:

[0345] A user launches the application and enters profile information such as name, age, and historical topics of interest.

[0346] Step 3:

[0347] The device checks the entered information and temporarily stores it.

[0348] Step 4:

[0349] The device sends the profile information to the server.

[0350] Step 5:

[0351] The server analyzes the received information and stores it in a database.

[0352] Step 6:

[0353] The server sends a message to the terminal indicating that the information has been saved.

[0354] Step 7:

[0355] The terminal displays a confirmation message to the user.

[0356] Automatic guidance upon arrival at destination

[0357] Step 1:

[0358] The device periodically obtains the user's current location using GPS.

[0359] Step 2:

[0360] The terminal transmits the obtained current location data to the server.

[0361] Step 3:

[0362] The server analyzes the user's current location and verifies that it is the tourist spot that was set.

[0363] Step 4:

[0364] If the server determines that the area is a tourist destination, it generates local guide data for the area.

[0365] Step 5:

[0366] The server transmits local guide data to the terminal.

[0367] Step 6:

[0368] The terminal displays the received local guidance data to the user, and the audio guide is automatically played.

[0369] Sightseeing with a Game-like Feel

[0370] Step 1:

[0371] The user selects a puzzle game or quiz mode within the app.

[0372] Step 2:

[0373] The terminal transmits the user's selection information to the server.

[0374] Step 3:

[0375] The server generates questions about local history and culture depending on the mode selected.

[0376] Step 4:

[0377] The server generates questions and sends them to the device.

[0378] Step 5:

[0379] The terminal presents the problem to the user.

[0380] Step 6:

[0381] The user answers the question.

[0382] Step 7:

[0383] The terminal sends the user's answer to the server.

[0384] Step 8:

[0385] The server evaluates the user's answer and determines whether it is correct.

[0386] Step 9:

[0387] The server generates information about the next spot and sends it to the terminal.

[0388] Step 10:

[0389] The device displays information about the next spot and provides directions to the location to the user.

[0390] Optimizing user experience with an emotion engine

[0391] Step 1:

[0392] The device analyzes the user's facial expressions and voice using an emotion engine to recognize emotions.

[0393] Step 2:

[0394] The device transmits the recognized emotion data to the server.

[0395] Step 3:

[0396] The server analyzes the emotional data and adjusts the local guide data and quiz questions.

[0397] Step 4:

[0398] The server transmits the adjusted data to the terminal.

[0399] Step 5:

[0400] The terminal displays and plays back the adjusted local guidance data and problems to the user.

[0401] Relive your travel memories

[0402] Step 1:

[0403] Users can enter memories of past trips within the app, or upload photos and notes.

[0404] Step 2:

[0405] The device sends the uploaded information to the server.

[0406] Step 3:

[0407] The server parses the data it receives.

[0408] Step 4:

[0409] The server generates content that recreates the virtual journey.

[0410] Step 5:

[0411] The server transmits the generated content to the terminal.

[0412] Step 6:

[0413] The terminal displays and plays the virtual travel content to the user.

[0414] Virtual travel at home

[0415] Step 1:

[0416] The user selects the "Virtual Travel" mode within the app.

[0417] Step 2:

[0418] The terminal transmits the selection information to the server.

[0419] Step 3:

[0420] The server generates a virtual sightseeing tour based on information about the desired travel destination.

[0421] Step 4:

[0422] The server transmits the generated virtual tour data to the terminal.

[0423] Step 5:

[0424] The terminal streams the virtual tour and provides it to the user.

[0425] Example 2

[0426] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0427] Conventional travel guide systems have the drawback of making it difficult for elderly people and those with mobility issues to use, as they require users to actually visit the destination to enjoy its attractions. Furthermore, the user experience is limited due to the lack of guidance that takes into account the user's emotions while traveling and the lack of functionality to recreate memories of past trips.

[0428] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0429] In this invention, the server includes means for inputting profile information through an application installed by the user and transmitting the information to the server, means for the server to confirm the user's current location based on GPS data, means for the server to generate local guidance data based on the current location confirmed by the server and transmit the data to the terminal, means for the terminal to display and play the received local guidance data to the user, means for the terminal to analyze the user's emotions and transmit the emotion data to the server, means for the server to adjust the local guidance data and the difficulty of the questions based on the emotion data and transmit the adjustment data to the terminal, means for the user to input past travel memories or upload photos and notes through the application and transmit the data to the server, means for the server to analyze the transmitted data, generate virtual travel content and transmit it to the terminal, and means for the terminal to display and play the virtual travel content to the user. This makes it possible to provide optimal guidance according to the user's emotions and to enhance the user experience through virtual travel.

[0430] "Profile Information" is information entered by a user that identifies them, such as their name, age, and historical topics of interest.

[0431] A "server" is a central computer device on a network that receives, analyzes, and stores data sent by users.

[0432] "GPS Data" means a user's current latitude and longitude information obtained from the Global Positioning System.

[0433] "Local guide data" is information and guide data about tourist spots that is generated based on the user's current location and interests.

[0434] "Terminal" refers to a mobile device such as a smartphone or tablet operated by a user.

[0435] "Emotion data" is information about the emotional state obtained by analyzing the user's facial expressions and voice.

[0436] "Virtual travel content" is digital content that provides a virtual travel experience, generated based on the user's travel records and uploaded photos and notes.

[0437] The "virtual travel mode" is a system mode that can be selected by the user to have a virtual travel experience.

[0438] "Difficulty of questions" refers to the level of difficulty of the questions in the generated quiz or puzzle game.

[0439] The present invention is a system for improving travel experiences through user-installed applications, providing appropriate guidance, quizzes, and virtual tours based on the user's emotions and interests. The system operates by combining the user's smartphone, a central server, and an emotion analysis engine.

[0440] User registration and information management

[0441] Users first install the application on their smartphone and enter their profile information, which includes their name, age, and historical topics of interest. The device temporarily stores this information and sends it to a server using the HTTPS protocol. The server analyzes the received data and stores it in a database, such as MySQL or PostgreSQL. The server then sends a message to the device indicating that the information has been saved, and the device notifies the user.

[0442] Automatic guidance upon arrival at destination

[0443] The device periodically obtains the user's current location using a GPS module and sends it to the server. The server analyzes the user's current location, and if that location is a tourist spot, it generates tourist spot information and sends it to the device. The device then displays the received tourist spot information to the user and plays an audio guide. This allows the user to immediately obtain information about the location when they arrive at a tourist spot.

[0444] Sightseeing with a Game-like Feel

[0445] Within the app, users can select a puzzle game or quiz mode. The device sends the user's selection information to the server, which then generates questions related to the local history and culture according to the selected mode. The questions are automatically generated using a generative AI model. The server sends the generated questions to the device, which then presents them to the user. When the user answers, the answer is sent to the server, which evaluates it and sends information and directions to the next spot to the device.

[0446] Relive your travel memories

[0447] Users can enter memories of past trips and upload photos and notes within the app. The device sends this data to the server, which analyzes the data and generates virtual travel content. The generated content is sent to the device, where users can view and play it to virtually relive past trips.

[0448] Virtual travel at home

[0449] Users select the "Virtual Travel" mode within the app and input their desired travel destination. The device sends the selected information to the server, which then generates a virtual sightseeing tour based on that information. The virtual sightseeing tour is created using a generative AI model. The generated tour data is sent to the device, allowing users to enjoy the virtual tour at home via streaming playback.

[0450] Optimizing user experience with an emotion engine

[0451] The device captures the user's facial expressions and voice in real time and uses an emotion analysis engine to obtain emotional data. The emotional data is sent to a server, which then adjusts the on-site guidance data and the difficulty of the questions based on that data. The adjusted data is then sent back to the device, allowing the user to receive customized guidance and quizzes tailored to their own state.

[0452] Specific examples

[0453] For example, if a user selects "Visit historical spots in Nara," the smartphone's GPS acquires the user's current location and sends it to the server. The server generates guide information about the "Nara period" and sends it to the device. The device then begins providing audio and text guidance. If the user selects quiz mode, a sample question such as "What is a representative building from the Nara period?" is displayed. If the user answers "Todaiji Temple," the server evaluates the answer and displays guidance to the next spot.

[0454] An example of a prompt sentence is "Please generate a prompt sentence for a system in which a user uses the emotion engine at tourist spots in Nara and is shown a quiz about the Nara period."

[0455] This will allow people to gain a deeper understanding of local history and culture and experience the joys of travel without having to actually visit the area. It will also make it possible to offer the joys of travel in a new way to elderly people and those with mobility difficulties.

[0456] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0457] Step 1:

[0458] The user installs the application on their smartphone.

[0459] The user downloads and installs the application from the smartphone's app store, then launches the application.

[0460] Input: App store download link

[0461] Output: Installed applications

[0462] Step 2:

[0463] The user launches the application and enters profile information.

[0464] Users launch the application and fill in fields with information such as their name, age, and historical topics of interest.

[0465] Input: Profile information entered by the user (name, age, topics of interest)

[0466] Output: Temporarily saved profile information

[0467] Step 3:

[0468] The device checks the entered information and temporarily stores it.

[0469] The terminal checks whether the input is in the correct format (for example, whether the name is not empty or the age is in numeric format), and then stores the information temporarily in memory.

[0470] Input: Profile information entered by the user

[0471] Output: Format-checked and temporarily saved profile information

[0472] Step 4:

[0473] The device sends the profile information to the server.

[0474] The device sends the checked profile information to the server using the HTTPS protocol.

[0475] Input: Format-checked and temporarily saved profile information

[0476] Output: Profile information sent to the server

[0477] Step 5:

[0478] The server analyzes the received information and stores it in a database.

[0479] The server parses the received profile information in JSON format and stores it in a database, for example, adding the user ID and timestamp.

[0480] Input: Profile information sent to the server

[0481] Output: Profile information stored in the database

[0482] Step 6:

[0483] The server sends a message to the terminal indicating that the information has been saved.

[0484] The server confirms that the data has been saved to the database, generates a message indicating that the data has been saved, and sends it to the terminal.

[0485] Input: Profile information stored in the database

[0486] Output: Message indicating information has been saved

[0487] Step 7:

[0488] The terminal displays a confirmation message to the user.

[0489] The terminal displays a message indicating that the information has been saved as received from the server as a pop-up notification or on the screen.

[0490] Input: Message that information has been saved

[0491] Output: User notification

[0492] Step 8:

[0493] The device periodically obtains the user's current location using GPS.

[0494] The device uses the smartphone's GPS module to obtain the user's current location at regular intervals (e.g., every 5 minutes).

[0495] Input: None (periodic acquisition of GPS module)

[0496] Output: User's current location data (latitude, longitude)

[0497] Step 9:

[0498] The terminal transmits the obtained current location data to the server.

[0499] The device converts the acquired location data into JSON format and sends it to the server using the HTTPS protocol.

[0500] Input: User's current location data (latitude, longitude)

[0501] Output: Location data sent to the server

[0502] Step 10:

[0503] The server analyzes the user's current location and confirms that it is a tourist spot.

[0504] The server compares the received current location data with a list of pre-registered tourist spots and analyzes whether the user's location is within a tourist spot.

[0505] Input: Location data sent to the server

[0506] Output: Tourist destination determination result

[0507] Step 11:

[0508] The server generates tourist destination information.

[0509] If the server confirms that the destination is a tourist spot, it generates information about the historical background and attractions of the destination.

[0510] Input: Tourist destination determination result

[0511] Output: Generated tourist destination information

[0512] Step 12:

[0513] The server transmits local guide data to the terminal.

[0514] The server sends the generated tourist destination information to the terminal in JSON format.

[0515] Input: Generated tourist destination information

[0516] Output: Local guide data sent to the terminal

[0517] Step 13:

[0518] The terminal displays the received local guidance data to the user and plays back the audio guide.

[0519] The device displays the received guidance data on the display, plays the audio guide file, and may notify the user using push notifications or alerts.

[0520] Input: Local guide data sent to the terminal

[0521] Output: Display to user, play audio guide

[0522] Step 14:

[0523] The user selects a puzzle game or quiz mode within the app.

[0524] Users select the "Play" tab from the in-app menu and choose their preferred mode.

[0525] Input: Mode selected by user

[0526] Output: Selected mode information

[0527] Step 15:

[0528] The terminal transmits the user's selection information to the server.

[0529] The device converts the selected mode information into JSON format and sends it to the server.

[0530] Input: Selected mode information

[0531] Output: Mode information sent to the server

[0532] Step 16:

[0533] The server generates questions related to local history and culture.

[0534] Based on the received mode information, the server uses a generative AI model to generate quizzes and questions about local history and culture.

[0535] Input: Mode information sent to the server

[0536] Output: Generated quizzes and questions

[0537] Step 17:

[0538] The server generates questions and sends them to the device.

[0539] The server sends the generated questions to the terminal in JSON format.

[0540] Input: Generated quizzes and questions

[0541] Output: Problems sent to terminal

[0542] Step 18:

[0543] The terminal presents the problem to the user.

[0544] The terminal displays the received questions on the screen so that the user can answer them.

[0545] Input: Question sent to terminal

[0546] Output: The problem presented to the user

[0547] Step 19:

[0548] The user answers the question.

[0549] Users can answer the questions by tapping on the appropriate answer option or by writing their own answer.

[0550] Input: User response

[0551] Output: User response data

[0552] Step 20:

[0553] The terminal sends the user's answer to the server.

[0554] The device converts the user's response into JSON format and sends it to the server.

[0555] Input: User response data

[0556] Output: Response data sent to the server

[0557] Step 21:

[0558] The server evaluates the user's answer and determines whether it is correct.

[0559] Based on the received answers, the server runs an algorithm to evaluate whether the answers are correct or incorrect and generates a judgment result.

[0560] Input: Response data sent to the server

[0561] Output: Judgment result (correct or incorrect)

[0562] Step 22:

[0563] The server generates information about the next spot and sends it to the terminal.

[0564] Based on the results of the determination, the server generates information about the next tourist spot and guidance data to that location, and sends it to the terminal.

[0565] Input: Judgment result

[0566] Output: Next spot information and guidance data

[0567] Step 23:

[0568] The terminal displays information and directions to the next spot to the user.

[0569] The device displays the received next spot information and provides the user with information such as maps and the shortest route.

[0570] Input: Next spot information and guidance data

[0571] Output: Next stop information displayed to the user

[0572] (Application example 2)

[0573] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0574] Conventional travel support systems have the problem that users cannot experience sightseeing unless they go to the destination, making it difficult for elderly people and those with mobility difficulties to enjoy the pleasure of traveling. In addition, the user experience is not personalized enough, and tourist information is provided uniformly. This makes it difficult to sufficiently increase user satisfaction.

[0575] Furthermore, virtual stores do not provide services that take into account the user's emotional state, making it difficult to provide a personalized shopping experience.

[0576] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0577] In this invention, the server includes means for inputting profile information through an application installed by the user and transmitting the information to the server, means for confirming the user's current location based on GPS data, means for generating local guidance data based on the confirmed current location and transmitting the data to the terminal, means for the terminal to display and play back the received local guidance data to the user, means for exploring a virtual store selected by the user, means for analyzing the user's emotions using an emotion engine and suggesting products and services based on the analysis results, and means for the terminal to display the suggested products and services to the user. This allows even elderly people and those with mobility difficulties to enjoy virtual travel, and enables personalized tourist information and shopping experiences tailored to the user's emotional state.

[0578] "Profile Information" means personal data such as a User's name, age, and interests.

[0579] "Server" refers to a device that receives, stores, and analyzes data from users, and generates and transmits information as instructed.

[0580] "GPS data" refers to data that includes location information for identifying a user's current location.

[0581] "Local guide data" means data including tourist information and guide information about the place the user is visiting.

[0582] A "terminal" is a device that is directly operated by a user, and includes, for example, a smartphone or a tablet.

[0583] A "virtual store" refers to a store that does not have a physical presence, but operates in a virtual space.

[0584] An "emotion engine" refers to software or hardware that analyzes a user's facial expressions, voice, etc. and recognizes their emotional state.

[0585] "Product and service suggestions" refers to the act of recommending the most suitable products and services to users based on the analysis results of the emotion engine.

[0586] "Quiz and mini-games" refers to entertainment elements in which users can participate and receive rewards or discounts based on their scores or results.

[0587] "Benefits and discounts" refers to benefits and discounts offered to users depending on the results of quizzes and mini-games.

[0588] "Virtual travel content" means content that recreates past travel data or uploaded information and allows users to virtually experience travel.

[0589] "Facial and voice analysis" refers to the process of analyzing facial and voice data to recognize the user's emotional state.

[0590] The present invention is a system implemented in the following specific steps: First, a user installs an application on a smartphone or head-mounted display and enters profile information. The application acquires information such as the user's name, age, and interests, and sends the information to a server. The server then stores the profile information in a database and manages the user's basic information.

[0591] Next, the user launches the application and can explore the virtual store. The application continuously acquires the user's current location using the GPS function and sends the data to the server. The server analyzes the acquired current location data and identifies the location within the virtual store selected by the user. Based on the user's location, the server then generates local guide data and product information for the virtual store and sends it to the device. The device then displays the received data to the user, providing intuitive navigation.

[0592] Furthermore, an emotion engine is used to analyze the user's emotional state. Specifically, the device acquires the user's facial expressions and voice data and sends that data to the server. The server then uses the emotion engine to analyze the user's emotions. Based on the analysis results, the server suggests the most suitable products and services for the user and sends that information to the device. This allows the device to display the suggested products and services to the user. For example, if the user has a happy expression, the server can provide special campaign and discount information tailored to that emotion.

[0593] Users can participate in quizzes and mini-games within the virtual store, which adds an entertainment element and improves the user experience. Based on the results of the quizzes and mini-games, rewards and discounts are offered, allowing users to enjoy shopping and earn rewards.

[0594] On the other hand, users can input memories of past trips or upload photos and notes. The server then analyzes the uploaded data and generates content that recreates a virtual trip. The content is sent to the device, allowing users to virtually re-experience past trips. This function is particularly useful for elderly people and those with mobility difficulties, who can enjoy virtual travel.

[0595] Specific examples

[0596] For example, consider a scenario where a 35-year-old user installs an application and explores a virtual store. The user's profile information includes their name, age, and categories of interest (e.g., electronics, books, and fashion). Using an emotion engine, the system suggests products that the user is interested in. For example, new smartphones, best-selling books, and trending fashion items may be suggested. The user can also participate in a quiz and earn discount coupons based on their score.

[0597] Example prompt for a generative AI model:

[0598] "A 35-year-old user is interested in electronics, books, and fashion. He enters a virtual store using his smartphone. Based on sentiment analysis, recommend the following products to him:

[0599] 1. The latest smartphone

[0600] 2. Bestselling books

[0601] 3. Trendy fashion items

[0602] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0603] Step 1:

[0604] A user installs an application on a smartphone or head-mounted display, which involves a download and installation process from an application store. The input is the application installation file, and the output is the installed application.

[0605] Step 2:

[0606] The user launches the application and enters profile information. Information such as name, age, and interest categories is received as input from the user, and the device temporarily stores that information. The input is the profile information entered directly by the user, and the output is the temporarily stored profile information.

[0607] Step 3:

[0608] The device sends profile information to the server. The server receives this information and stores it in a database. The input is the profile information sent from the device, and the output is the profile information stored in the database.

[0609] Step 4:

[0610] The user selects a virtual store and begins searching. The device uses its GPS function to obtain current location data and sends it to the server. The input is the device's GPS data, and the output is the current location data sent to the server.

[0611] Step 5:

[0612] The server analyzes the acquired current location data and identifies the location within the virtual store selected by the user. The server generates local guidance data and product information based on that location and sends it to the terminal. The input is the current location data, and the output is the generated local guidance data and product information.

[0613] Step 6:

[0614] The terminal displays the received local guide data and product information to the user. The display can be in the form of text, images, audio, etc. The input is the local guide data and product information sent from the server, and the output is the information displayed to the user.

[0615] Step 7:

[0616] The device acquires the user's facial expression and voice data and sends it to the server. The server uses an emotion engine to analyze the user's emotional state. The input is facial expression and voice data, and the output is the emotion analysis results.

[0617] Step 8:

[0618] The server generates data to suggest optimal products and services to the user based on the results of the emotion analysis and sends it to the terminal. The input is the emotion analysis results, and the output is the generated product and service suggestion data.

[0619] Step 9:

[0620] The terminal displays the suggested product and service data to the user, and the user can proceed with shopping based on that information. The input is the suggested data sent from the server, and the output is the product and service information displayed to the user.

[0621] Step 10:

[0622] Users participate in quizzes and mini-games in a virtual store. The terminal sends the game results to the server, which then generates rewards and discount data based on the results. The input is the user's game results, and the output is the rewards and discount data.

[0623] Step 11:

[0624] The server sends the generated reward and discount data to the terminal, which then displays it to the user. The input is the reward and discount data, and the output is the reward information displayed to the user.

[0625] Step 12:

[0626] Users input memories of past travels or upload photos and notes. The device sends the data to the server, which then generates content that recreates the virtual trip. The input is the data uploaded by the user, and the output is the virtual trip content.

[0627] Step 13:

[0628] The server sends the generated virtual travel content to the terminal, which then displays and plays it for the user. The input is the virtual travel content, and the output is the virtual travel content that is displayed and played for the user.

[0629] Examples:

[0630] For example, if a user is looking to buy a fashion item in a virtual store, sentiment analysis will suggest new items that the user is interested in. Users can also participate in quizzes and earn discount coupons depending on their scores.

[0631] Example prompt for a generative AI model:

[0632] "A 35-year-old user is interested in electronics, books, and fashion. He enters a virtual store using his smartphone. Based on sentiment analysis, recommend the following products to him:

[0633] 1. The latest smartphone

[0634] 2. Bestselling books

[0635] 3. Trendy fashion items

[0636] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0637] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0638] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0639] [Second embodiment]

[0640] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0641] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0642] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0643] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0644] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0645] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0646] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0647] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0648] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0649] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0650] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0651] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0652] Overall flow

[0653] This invention is a system that makes it easier for travelers to enjoy the attractions of rural areas outside of cities, enriching the local tourist experience while also providing a similar experience in remote areas. Furthermore, it also provides the enjoyment of travel to elderly people who find it difficult to travel to the local area. The system supports the tourist experience by linking the user's smartphone with a server.

[0654] User registration and information management

[0655] 1. The user installs a dedicated application on their smartphone.

[0656] 2. The user launches the application and enters profile information such as name, age, and historical topics of interest.

[0657] 3. The device sends the entered information to the server.

[0658] 4. The server stores the received information in a database and sends a confirmation message to the user.

[0659] Automatic guidance upon arrival at destination

[0660] 1. The device periodically obtains the user's current location using GPS.

[0661] 2. The device sends the acquired location data to the server.

[0662] 3. The server analyzes the user's current location and verifies that it is one of the tourist destinations set.

[0663] 4. The server generates local guidance data and sends it to the terminal.

[0664] 5. The terminal displays and plays the received guidance data (audio guide, text, images) to the user.

[0665] Sightseeing with a Game-like Feel

[0666] 1. The user selects a puzzle game or quiz mode within the app.

[0667] 2. The server generates questions about local history and culture according to the selected mode and sends them to the terminal.

[0668] 3. The device presents the user with a question, collects the user's answers, and sends them to the server.

[0669] 4. The server evaluates the user's answer, generates information about the next spot, and sends it to the device.

[0670] 5. The device displays information about the next spot and provides directions to the location to the user.

[0671] Relive your travel memories

[0672] 1. Users enter memories of past trips or upload photos and notes within the app.

[0673] 2. The server parses the uploaded data.

[0674] 3. The server generates content (in the form of videos or slideshows) that recreates the virtual journey.

[0675] 4. The terminal displays and plays the generated virtual travel content to the user.

[0676] Virtual travel at home

[0677] 1. The user selects the "Virtual Travel" mode within the app.

[0678] 2. The server collects information on the desired travel destinations based on the user's selection and generates a virtual sightseeing tour.

[0679] 3. The device streams the generated virtual tour and provides it to the user.

[0680] Specific examples

[0681] For example, consider a case where a user visits a historical spot in Nara. When the user arrives at the location, the smartphone automatically displays an audio guide and text about the Nara period. Furthermore, if the user selects the quiz mode, the smartphone asks the question, "What is a representative building from the Nara period?" If the answer is correct, directions to the next spot will be displayed.

[0682] If a user wants to relive a past trip to Kyoto, they can upload past photos to the app. The server analyzes the photos and generates a video that virtually recreates the trip to Kyoto. The user can then enjoy the video at home.

[0683] This will enable people to gain a deeper understanding of local history and culture and experience the joy of travel without having to go to the actual place. Virtual travel will also be a new way to provide the joy of travel to elderly people and those with mobility difficulties.

[0684] The processing flow will be explained below.

[0685] User registration and information management

[0686] Step 1:

[0687] The user downloads and installs the application on their smartphone.

[0688] Step 2:

[0689] A user launches the application and enters profile information such as name, age, and historical topics of interest.

[0690] Step 3:

[0691] The device checks the entered information and temporarily stores it.

[0692] Step 4:

[0693] The device sends the profile information to the server.

[0694] Step 5:

[0695] The server analyzes the received information and stores it in a database.

[0696] Step 6:

[0697] The server sends a message to the terminal indicating that the information has been saved.

[0698] Step 7:

[0699] The terminal displays a confirmation message to the user.

[0700] Automatic guidance upon arrival at destination

[0701] Step 1:

[0702] The device periodically obtains the user's current location using GPS.

[0703] Step 2:

[0704] The terminal transmits the obtained current location data to the server.

[0705] Step 3:

[0706] The server analyzes the user's current location and checks whether it is the tourist spot you have set.

[0707] Step 4:

[0708] If the server determines that the area is a tourist destination, it generates local guide data for the area.

[0709] Step 5:

[0710] The server transmits local guide data to the terminal.

[0711] Step 6:

[0712] The terminal displays the received local guidance data to the user, and the audio guide is automatically played.

[0713] Sightseeing with a Game-like Feel

[0714] Step 1:

[0715] The user selects a puzzle game or quiz mode within the app.

[0716] Step 2:

[0717] The terminal transmits the user's selection information to the server.

[0718] Step 3:

[0719] The server generates questions about local history and culture depending on the mode selected.

[0720] Step 4:

[0721] The server generates questions and sends them to the device.

[0722] Step 5:

[0723] The terminal presents the problem to the user.

[0724] Step 6:

[0725] The user answers the question.

[0726] Step 7:

[0727] The terminal sends the user's answer to the server.

[0728] Step 8:

[0729] The server evaluates the user's answer and determines whether it is correct.

[0730] Step 9:

[0731] The server generates information about the next spot and sends it to the terminal.

[0732] Step 10:

[0733] The device displays information about the next spot and provides directions to the location to the user.

[0734] Relive your travel memories

[0735] Step 1:

[0736] Users can enter memories of past trips within the app, or upload photos and notes.

[0737] Step 2:

[0738] The device sends the uploaded information to the server.

[0739] Step 3:

[0740] The server parses the data it receives.

[0741] Step 4:

[0742] The server generates content that recreates the virtual journey.

[0743] Step 5:

[0744] The server transmits the generated content to the terminal.

[0745] Step 6:

[0746] The terminal displays and plays the virtual travel content to the user.

[0747] Virtual travel at home

[0748] Step 1:

[0749] The user selects the "Virtual Travel" mode within the app.

[0750] Step 2:

[0751] The terminal transmits the selection information to the server.

[0752] Step 3:

[0753] The server generates a virtual sightseeing tour based on information about the desired travel destination.

[0754] Step 4:

[0755] The server transmits the generated virtual tour data to the terminal.

[0756] Step 5:

[0757] The terminal streams the virtual tour and provides it to the user.

[0758] Example 1

[0759] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0760] Conventional tourism systems often lack tourist information when visiting rural areas outside of cities, making it difficult to fully enjoy the attractions of those areas. There is also the problem that tourism itself is difficult for elderly people and people with mobility restrictions who have difficulty traveling to those areas. Furthermore, there are limited ways to recreate and re-enjoy memories of past trips. The goal of this project is to solve these problems and make travel and tourism attractive and easy for everyone to enjoy.

[0761] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0762] In this invention, the server includes a means for users to send profile information, a means for checking current location based on GPS data, and a means for generating local guide data based on the current location. This allows users to easily enjoy the attractions of local areas. The server also includes a means for inputting memories of past travels or uploading photos and notes, a means for analyzing the uploaded data and generating content that recreates a virtual travel, and a means for displaying and playing the generated virtual travel content to the user, allowing users to relive past travels. Furthermore, the server includes a means for selecting a "virtual travel" mode, a means for collecting information on desired travel destinations and generating a virtual sightseeing tour, and a means for streaming and providing the generated virtual tour to the user, allowing users who have difficulty visiting the destinations to enjoy the fun of travel.

[0763] "User" refers to a human user who interfaces with the server and terminal.

[0764] "Device" refers to a smartphone, tablet, or other mobile information device on which a user installs and uses an application.

[0765] "Application" refers to a software program that a user installs on a terminal and uses.

[0766] "Profile Information" refers to personal information about you, such as your name, age, and topics of interest.

[0767] "Server" means a remote computer system that processes and stores data received from a client device and executes server-side actions.

[0768] "GPS Data" refers to your location information obtained using the Global Positioning System.

[0769] "Current Location" refers to a user's real-time geographic location as determined based on GPS data.

[0770] "Local guide data" refers to data containing information about a specific tourist destination (audio guide, text, images, etc.).

[0771] "Virtual travel content" refers to content (such as videos or slideshows) that is displayed and played back to users and is generated based on past travel memories and uploaded data.

[0772] "Virtual travel" refers to a sightseeing tour that allows you to collect information about your desired travel destination and experience it virtually without physically going there.

[0773] "Questions" refer to quizzes and puzzles about history and culture that are generated by the server and presented to the user.

[0774] A "mode" refers to a particular type of operation or function that a user selects within an application.

[0775] This invention is a system for enhancing the appeal of visiting regional tourist destinations outside of urban areas. This system links the user's smartphone with a server, aiming to enrich the tourist experience on-site while also providing a similar experience in remote locations. Furthermore, it is possible to provide the enjoyment of travel to elderly users and users with mobility difficulties.

[0776] 1. User registration and information management

[0777] Users first install a dedicated application on their smartphone. They then launch the application and enter their profile information, such as their name, age, and historical topics of interest. The information is then sent from the device to a server, which stores it in a database. This is how the user profile is managed.

[0778] 2. Automatic guidance when arriving at your destination

[0779] The user's device periodically obtains its current location using GPS and sends this data to the server. The server analyzes the current location data and confirms that the user has arrived at the tourist destination. The server generates local guide data about the tourist destination and sends it to the device. The device receives this data and displays and plays audio guides, text, images, etc. to the user. Technologies used include Google Maps API and Azure Cognitive Services.

[0780] 3. Sightseeing like a game

[0781] Within the app, users can select a puzzle game or quiz mode. Depending on the selected mode, the server generates questions about local history and culture and sends them to the device. The device presents the questions to the user and collects their answers. The server evaluates the answers, generates information about the next spot, and sends it to the device. The device then displays information about the next spot and directions to the location to the user. A specific example is the question, "What is a representative building from the Nara period?", and the user inputs the answer.

[0782] 4. Relive your travel memories

[0783] Users can input memories of past travels and upload photos and notes. The server analyzes the uploaded data and generates content that recreates the virtual trip. The device displays and plays the generated virtual trip content to the user. Technologies used include Amazon Rekognition and Adobe Premiere API. An example prompt is, "Users upload photos of travel destinations they have visited in the past, and a video recreating that trip is generated and displayed."

[0784] 5. Virtual travel at home

[0785] Users can select the "Virtual Travel" mode within the app. The server collects information on the desired travel destination based on the user's selection and generates a virtual sightseeing tour. The device streams the generated virtual tour and provides it to the user. The technology used includes the Google Earth API. An example of a specific prompt is, "The user selects the 'Virtual Travel' mode and a virtual tour of Paris begins."

[0786] This makes it possible for the present invention to increase the enjoyment of traveling to rural areas, providing a similar experience from remote locations, and bringing the joy of travel to elderly people and those with mobility difficulties.

[0787] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0788] Step 1:

[0789] The user installs the dedicated application on their smartphone. The user downloads the application from the app store and completes the installation procedure. The application can then be used on the smartphone.

[0790] Step 2:

[0791] A user launches the application and enters profile information (name, age, historical topics of interest). The information entered by the user is temporarily stored on the device. Profile information is constructed based on the input (e.g., name "Taro Tanaka," age "65," interest "ancient history").

[0792] Step 3:

[0793] The device sends the profile information to the server. The input data from the device is transferred to the server via the Internet and received by the server. The received profile information is saved in the server's database. The server generates a save completion message and sends it to the device. The device displays a "Registration Complete" message to the user.

[0794] Step 4:

[0795] The device periodically obtains the user's current location using GPS. The device uses the GPS sensor to collect location information and generate that data. The generated GPS data is stored on the device.

[0796] Step 5:

[0797] The device transmits the acquired current location data to the server. The location information data from the device is transferred to the server and received by the server. The received data is sent to the server's analysis module.

[0798] Step 6:

[0799] The server analyzes the current location data and confirms that the user has arrived at the designated tourist spot. The server compares the location data with tourist spot information in the database to determine whether the user's current location matches the tourist spot. If the location information matches the tourist spot, the server generates local guidance data.

[0800] Step 7:

[0801] The server generates local guidance data (audio guide, text, images) and sends it to the terminal. The generated guidance data is transferred from the server to the terminal. The terminal displays and plays the received local guidance data to the user. The user plays the audio guide and receives the information.

[0802] Step 8:

[0803] The user selects a puzzle game or quiz mode within the app. The user operates the mode selection screen and selects the desired game mode. The selected mode information is saved on the device.

[0804] Step 9:

[0805] The server generates questions about local history and culture according to the selected mode and sends them to the device. The server generates question data based on the game mode and transfers the data to the device. The device presents the questions to the user and collects the user's answers.

[0806] Step 10:

[0807] The terminal sends the user's answer data to the server. The answer data from the terminal is transferred to the server and received by the server. The received data is sent to the server's analysis module.

[0808] Step 11:

[0809] The server evaluates the user's answers, generates information about the next spot, and sends it to the device. The server analyzes the answer data and generates information about the next tourist spot based on the evaluation results. The generated spot information is then transferred to the device.

[0810] Step 12:

[0811] The device displays information about the next spot and guidance to the user. The device receives the spot information and displays it to the user. The user confirms the guidance to the next spot and begins moving.

[0812] Step 13:

[0813] The user inputs memories of past trips or uploads photos and notes. The user inputs past trip information within the app or selects and uploads photo files. The input and uploaded data is saved on the device.

[0814] Step 14:

[0815] The device sends the uploaded data to the server. The uploaded data from the device is transferred to the server and received by the server. The received data is sent to the server's analysis module.

[0816] Step 15:

[0817] The server analyzes the uploaded data and generates virtual travel content. The server processes the data using image analysis technology and generates content for recreating the virtual travel. The generated content is stored on the server.

[0818] Step 16:

[0819] The server sends the virtual travel content to the terminal. The generated content data is transferred from the server to the terminal. The terminal receives the virtual travel content and displays and plays it for the user. The user experiences the virtual travel.

[0820] Step 17:

[0821] The user selects the "Virtual Travel" mode within the app. The user operates the mode selection screen and selects the Virtual Travel mode. The selection information is saved on the device.

[0822] Step 18:

[0823] The server collects information on the desired travel destination based on the user's selection and generates a virtual sightseeing tour. The server extracts the travel destination information from the database and creates a virtual tour based on the collected data. The generated tour information is saved on the server.

[0824] Step 19:

[0825] The server sends the generated virtual tour to the terminal. The tour information data is transferred from the server to the terminal. The terminal receives the virtual tour and starts streaming playback. The user experiences the virtual trip.

[0826] (Application example 1)

[0827] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0828] There is a need to provide tourism experiences for elderly people and travel enthusiasts living in remote areas who find it difficult to visit local sites. However, current systems rely on on-site visits and lack ways to enhance tourism experiences in virtual environments. This invention aims to solve this problem by providing a virtual travel system that allows users to enjoy the charms of local areas from the comfort of their own homes.

[0829] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0830] In this invention, the server includes means for inputting profile information through an application installed by the user and transmitting the information to the server, means for confirming the user's current location based on GPS data, means for generating local guidance data based on the confirmed current location and transmitting it to the terminal, means for the user to select a mode within the application for enjoying a virtual trip at home, means for generating virtual sightseeing tour content based on the selected travel destination and transmitting it to the terminal, and means for displaying and playing the received virtual sightseeing tour content to the user, thereby enabling the user to enjoy an advanced virtual sightseeing experience from the comfort of their own home.

[0831] "Profile Information" refers to personal information such as a User's name, age, and topics of interest that is used by the Application to provide the User with an optimal tourism experience.

[0832] "GPS Data" means location information obtained using the Global Positioning System and is used to determine a user's current location.

[0833] "Local guide data" refers to data that includes information such as audio guides, text, and images about tourist spots that users visit, and is provided to support the user's sightseeing experience.

[0834] "Device" refers to an electronic device used by a user, such as a smartphone, tablet, or head-mounted display, which is used to display and play data sent from the server.

[0835] "Virtual travel" refers to the process by which users virtually visit tourist spots through an application and experience their culture and history without actually traveling.

[0836] "Virtual sightseeing tour content" refers to content such as 3D visuals of tourist spots, audio guides, and text that is generated by the server and sent to the user's terminal, and is the main information that allows the user to experience a virtual trip.

[0837] "Quiz mode" refers to a function that, when selected by the user, causes the application to ask questions about tourist spots, and allows the user to enjoy sightseeing in a game-like manner by answering the questions.

[0838] A "head-mounted display" is a display device worn by the user on the head to display virtual reality (VR) and augmented reality (AR) content, and is used to provide an advanced virtual experience.

[0839] The present invention provides a virtual travel system that allows users to experience the attractions of local areas from the comfort of their own homes. Detailed embodiments are described below.

[0840] System Configuration

[0841] This system consists of an application installed by the user, a terminal (such as a smartphone or a head-mounted display), and a server.

[0842] Registering profile information and acquiring GPS data

[0843] Through the application, users enter profile information such as their name, age, and topics of interest, and submit that information to the server, which stores the user's profile information in a database.

[0844] The device periodically acquires the user's current location using GPS and sends that data to the server, which then analyzes the GPS data to confirm the user's current location.

[0845] Generation and display of on-site guidance data

[0846] The server generates local guidance data based on the user's current location and sends it to the device. The device then displays and plays the received local guidance data to the user as audio guides, text, and images, allowing the user to obtain a wealth of information about the local history and culture.

[0847] Providing virtual travel modes

[0848] Within the application, users can select a mode to enjoy a virtual trip at home. The server generates virtual sightseeing tour content based on the selected travel destination and sends it to the device. The device then displays and plays the received virtual sightseeing tour content to the user as an audio guide or 3D visuals. Specifically, users can experience the virtual trip through a smartphone or head-mounted display.

[0849] Quiz mode available

[0850] The application has a quiz mode. Based on the quiz mode selected by the user, the server generates questions about local history and culture and sends them to the device. The device presents the questions to the user, collects the user's answers, and sends them to the server. The server evaluates the user's answers, generates information about the next spot, and sends it to the device. This allows the user to enjoy sightseeing in a game-like manner.

[0851] Recreating a virtual journey

[0852] Users can input memories of past trips or upload photos and notes. The server analyzes the uploaded data and generates content that recreates a virtual trip. The device displays and plays the generated virtual trip content to the user.

[0853] Hardware and software used

[0854] This system uses the following hardware and software:

[0855] Hardware: Smartphone, Head-Mounted Display (HMD), Server

[0856] Software: Application, server program, HTTP request library (requests), JSON parser

[0857] Examples of concrete examples and prompts

[0858] As a concrete example, consider a scenario in which a user experiences historical spots in Nara from the comfort of their own home. The user launches the app and selects the "Virtual Travel" mode. The server generates an audio guide and text about Nara culture based on the user's profile data and selected travel destination, and sends them to the device along with 3D visuals. The user puts on a head-mounted display and experiences Nara's history in a virtual environment. In addition, in quiz mode, the user is asked the question, "What is a representative building from the Nara period?" and by answering the question, directions to the next spot are displayed.

[0859] Below is an example of a prompt sentence to input to the generative AI model.

[0860] Prompt Sentence Examples

[0861] ---

[0862] We are developing an application that allows users to enjoy virtual travel from the comfort of their own home. The application runs on smartphones and head-mounted displays (HMDs) and provides users with a virtual sightseeing experience related to local culture and history.

[0863] example:

[0864] The user uses the app to experience Nara's historical spots. The app retrieves audio guides and text about Nara based on the user's profile data, and displays them in 3D visuals through the HMD. The app also has a quiz mode, and after the user answers questions, the app guides the user to the next virtual spot.

[0865] Expected output:

[0866] Audio guide about historical spots from the Nara period

[0867] Coding audio files for broadcast

[0868] A quiz about representative buildings from the Nara period

[0869] Directions to the next virtual spot

[0870] Please explain in detail how this application works.

[0871] ---

[0872] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0873] Step 1:

[0874] A user installs the application, enters profile information (such as name, age, and topics of interest), and sends that information to the server, which stores the information in a database and sends a confirmation message to the user.

[0875] Input: User profile information Output: Confirmation message

[0876] Step 2:

[0877] The device periodically acquires GPS data and sends the user's current location to the server. The server analyzes the received GPS data and confirms the user's current location. When the user arrives at the location, the server generates local guidance data and sends it to the device.

[0878] Input: GPS data Output: Local guidance data

[0879] Step 3:

[0880] The device displays and plays the received local information to the user. Specifically, it displays audio guides, text, and images, allowing the user to learn about the history and culture of the area.

[0881] Input: On-site guidance data Output: Displayed / played guidance data

[0882] Step 4:

[0883] The user selects the "Virtual Travel" mode in the application. The terminal sends this selection to the server. The server generates virtual sightseeing tour content based on the selected travel destinations and sends it to the terminal.

[0884] Input: Virtual travel mode selection information Output: Virtual sightseeing tour content

[0885] Step 5:

[0886] The device displays and plays the received virtual sightseeing tour content to the user. When using a head-mounted display, 3D visuals are displayed, allowing the user to experience traveling in a virtual environment.

[0887] Input: Virtual sightseeing tour content Output: Displayed / played tour content

[0888] Step 6:

[0889] The user selects the quiz mode. The terminal transmits this selection information to the server. The server generates questions about the local history and culture and transmits them to the terminal.

[0890] Input: Quiz mode selection information Output: Quiz question data

[0891] Step 7:

[0892] The terminal presents the received quiz question data to the user, collects the user's answers, and sends them to the server. The server evaluates the user's answers, generates guidance data for the next spot, and sends it to the terminal. The terminal displays it to the user.

[0893] Input: User response data Output: Next spot guidance data

[0894] Step 8:

[0895] Users input memories of past trips or upload photos and notes. The device sends this data to the server, which analyzes the uploaded data, generates content that recreates the virtual trip, and sends it to the device.

[0896] Input: Past travel data Output: Virtual travel reenactment content

[0897] Step 9:

[0898] The device displays and plays the received virtual travel recreation content to the user, allowing the user to enjoy memories of past trips from the comfort of their own home.

[0899] Input: Virtual travel content Output: Displayed / played travel content

[0900] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0901] This system makes it easier for travelers to enjoy the attractions of rural areas outside of cities, enriching the local tourist experience while also providing a similar experience in remote locations. It also provides the enjoyment of travel to elderly people who find it difficult to travel to the local area. The system supports the tourist experience by linking the user's smartphone with a server and an emotion engine.

[0902] User registration and information management

[0903] 1. The user installs a dedicated application on their smartphone.

[0904] 2. The user launches the application and enters profile information such as name, age, and historical topics of interest.

[0905] 3. The device checks the entered information and temporarily stores it.

[0906] 4. The device sends the profile information to the server.

[0907] 5. The server analyzes the received information and stores it in a database.

[0908] 6. The server sends a message to the device indicating that the information has been saved.

[0909] 7. The terminal displays a confirmation message to the user.

[0910] Automatic guidance upon arrival at destination

[0911] 1. The device periodically obtains the user's current location using GPS.

[0912] 2. The device sends the location data it has obtained to the server.

[0913] 3. The server analyzes the user's current location and confirms that it is the tourist spot that was set.

[0914] 4. If the server determines that the area is a tourist destination, it generates local guide data for the area.

[0915] 5. The server sends the local guide data to the terminal.

[0916] 6. The device displays the received local guidance data to the user, and the audio guide is automatically played.

[0917] Sightseeing with a Game-like Feel

[0918] 1. The user selects a puzzle game or quiz mode within the app.

[0919] 2. The terminal sends the user's selection information to the server.

[0920] 3. The server generates questions about local history and culture depending on the mode selected.

[0921] 4. The server sends the generated questions to the device.

[0922] 5. The device presents the problem to the user.

[0923] 6. The user answers the question.

[0924] 7. The device sends the user's answer to the server.

[0925] 8. The server evaluates the user's answer and determines whether it is correct.

[0926] 9. The server generates information about the next spot and sends it to the terminal.

[0927] 10. The device displays information about the next spot and directions to the location to the user.

[0928] Relive your travel memories

[0929] 1. Users enter memories of past trips or upload photos and notes within the app.

[0930] 2. The device sends the uploaded information to the server.

[0931] 3. The server analyzes the received data.

[0932] 4. The server generates content that recreates the virtual journey.

[0933] 5. The server sends the generated content to the device.

[0934] 6. The device displays and plays the virtual travel content to the user.

[0935] Virtual travel at home

[0936] 1. The user selects the "Virtual Travel" mode within the app.

[0937] 2. The device sends the selection information to the server.

[0938] 3. The server generates a virtual sightseeing tour based on information about the desired travel destination.

[0939] 4. The server sends the generated virtual tour data to the device.

[0940] 5. The device streams the virtual tour and provides it to the user.

[0941] Optimizing user experience with an emotion engine

[0942] 1. The device analyzes the user's facial expressions and voice using an emotion engine to recognize emotions.

[0943] 2. The server adjusts the local guidance data and the difficulty of the game questions based on the recognized emotion data.

[0944] 3. The server sends the data generated based on the emotion to the device.

[0945] 4. The terminal displays and plays the adjusted guidance data and questions to the user.

[0946] Specific examples

[0947] For example, if a user visits a historical spot in Nara, the smartphone will automatically display an audio guide and text about the Nara period. Furthermore, if the user selects the quiz mode, they will be asked, "What is a representative building from the Nara period?" If they answer correctly, directions to the next spot will be displayed.

[0948] By using the emotion engine, the difficulty of the questions can be adjusted even if the user is tired, for example. Also, if a user wants to relive a past trip to Kyoto, they can upload past photos to the app. The server analyzes the photos and generates a video that virtually recreates the Kyoto trip. The user can then enjoy the video at home.

[0949] This will enable people to gain a deeper understanding of local history and culture and experience the joy of travel without having to actually visit the area. Virtual travel will also be a new way to provide the joy of travel to elderly people and those with mobility difficulties.

[0950] The processing flow will be explained below.

[0951] User registration and information management

[0952] Step 1:

[0953] The user downloads and installs a dedicated application onto their smartphone.

[0954] Step 2:

[0955] A user launches the application and enters profile information such as name, age, and historical topics of interest.

[0956] Step 3:

[0957] The device checks the entered information and temporarily stores it.

[0958] Step 4:

[0959] The device sends the profile information to the server.

[0960] Step 5:

[0961] The server analyzes the received information and stores it in a database.

[0962] Step 6:

[0963] The server sends a message to the terminal indicating that the information has been saved.

[0964] Step 7:

[0965] The terminal displays a confirmation message to the user.

[0966] Automatic guidance upon arrival at destination

[0967] Step 1:

[0968] The device periodically obtains the user's current location using GPS.

[0969] Step 2:

[0970] The terminal transmits the obtained current location data to the server.

[0971] Step 3:

[0972] The server analyzes the user's current location and verifies that it is the tourist spot that was set.

[0973] Step 4:

[0974] If the server determines that the area is a tourist destination, it generates local guide data for the area.

[0975] Step 5:

[0976] The server transmits local guide data to the terminal.

[0977] Step 6:

[0978] The terminal displays the received local guidance data to the user, and the audio guide is automatically played.

[0979] Sightseeing with a Game-like Feel

[0980] Step 1:

[0981] The user selects a puzzle game or quiz mode within the app.

[0982] Step 2:

[0983] The terminal transmits the user's selection information to the server.

[0984] Step 3:

[0985] The server generates questions about local history and culture depending on the mode selected.

[0986] Step 4:

[0987] The server generates questions and sends them to the device.

[0988] Step 5:

[0989] The terminal presents the problem to the user.

[0990] Step 6:

[0991] The user answers the question.

[0992] Step 7:

[0993] The terminal sends the user's answer to the server.

[0994] Step 8:

[0995] The server evaluates the user's answer and determines whether it is correct.

[0996] Step 9:

[0997] The server generates information about the next spot and sends it to the terminal.

[0998] Step 10:

[0999] The device displays information about the next spot and provides directions to the location to the user.

[1000] Optimizing user experience with an emotion engine

[1001] Step 1:

[1002] The device analyzes the user's facial expressions and voice using an emotion engine to recognize emotions.

[1003] Step 2:

[1004] The device transmits the recognized emotion data to the server.

[1005] Step 3:

[1006] The server analyzes the emotional data and adjusts the local guide data and quiz questions.

[1007] Step 4:

[1008] The server transmits the adjusted data to the terminal.

[1009] Step 5:

[1010] The terminal displays and plays back the adjusted local guidance data and problems to the user.

[1011] Relive your travel memories

[1012] Step 1:

[1013] Users can enter memories of past trips within the app, or upload photos and notes.

[1014] Step 2:

[1015] The device sends the uploaded information to the server.

[1016] Step 3:

[1017] The server parses the data it receives.

[1018] Step 4:

[1019] The server generates content that recreates the virtual journey.

[1020] Step 5:

[1021] The server transmits the generated content to the terminal.

[1022] Step 6:

[1023] The terminal displays and plays the virtual travel content to the user.

[1024] Virtual travel at home

[1025] Step 1:

[1026] The user selects the "Virtual Travel" mode within the app.

[1027] Step 2:

[1028] The terminal transmits the selection information to the server.

[1029] Step 3:

[1030] The server generates a virtual sightseeing tour based on information about the desired travel destination.

[1031] Step 4:

[1032] The server transmits the generated virtual tour data to the terminal.

[1033] Step 5:

[1034] The terminal streams the virtual tour and provides it to the user.

[1035] Example 2

[1036] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1037] Conventional travel guide systems have the drawback of making it difficult for elderly people and those with mobility issues to use, as they require users to actually visit the destination to enjoy its attractions. Furthermore, the user experience is limited due to the lack of guidance that takes into account the user's emotions while traveling and the lack of functionality to recreate memories of past trips.

[1038] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1039] In this invention, the server includes means for inputting profile information through an application installed by the user and transmitting the information to the server, means for the server to confirm the user's current location based on GPS data, means for the server to generate local guidance data based on the current location confirmed by the server and transmit the data to the terminal, means for the terminal to display and play the received local guidance data to the user, means for the terminal to analyze the user's emotions and transmit the emotion data to the server, means for the server to adjust the local guidance data and the difficulty of the questions based on the emotion data and transmit the adjustment data to the terminal, means for the user to input past travel memories or upload photos and notes through the application and transmit the data to the server, means for the server to analyze the transmitted data, generate virtual travel content and transmit it to the terminal, and means for the terminal to display and play the virtual travel content to the user. This makes it possible to provide optimal guidance according to the user's emotions and to enhance the user experience through virtual travel.

[1040] "Profile Information" is information entered by a user that identifies them, such as their name, age, and historical topics of interest.

[1041] A "server" is a central computer device on a network that receives, analyzes, and stores data sent by users.

[1042] "GPS Data" means a user's current latitude and longitude information obtained from the Global Positioning System.

[1043] "Local guide data" is information and guide data about tourist spots that is generated based on the user's current location and interests.

[1044] "Terminal" refers to a mobile device such as a smartphone or tablet operated by a user.

[1045] "Emotion data" is information about the emotional state obtained by analyzing the user's facial expressions and voice.

[1046] "Virtual travel content" is digital content that provides a virtual travel experience, generated based on the user's travel records and uploaded photos and notes.

[1047] The "virtual travel mode" is a system mode that can be selected by the user to have a virtual travel experience.

[1048] "Difficulty of questions" refers to the level of difficulty of the questions in the generated quiz or puzzle game.

[1049] The present invention is a system for improving travel experiences through user-installed applications, providing appropriate guidance, quizzes, and virtual tours based on the user's emotions and interests. The system operates by combining the user's smartphone, a central server, and an emotion analysis engine.

[1050] User registration and information management

[1051] Users first install the application on their smartphone and enter their profile information, which includes their name, age, and historical topics of interest. The device temporarily stores this information and sends it to a server using the HTTPS protocol. The server analyzes the received data and stores it in a database, such as MySQL or PostgreSQL. The server then sends a message to the device indicating that the information has been saved, and the device notifies the user.

[1052] Automatic guidance upon arrival at destination

[1053] The device periodically obtains the user's current location using a GPS module and sends it to the server. The server analyzes the user's current location, and if that location is a tourist spot, it generates tourist spot information and sends it to the device. The device then displays the received tourist spot information to the user and plays an audio guide. This allows the user to immediately obtain information about the location when they arrive at a tourist spot.

[1054] Sightseeing with a Game-like Feel

[1055] Within the app, users can select a puzzle game or quiz mode. The device sends the user's selection information to the server, which then generates questions related to the local history and culture according to the selected mode. The questions are automatically generated using a generative AI model. The server sends the generated questions to the device, which then presents them to the user. When the user answers, the answer is sent to the server, which evaluates it and sends information and directions to the next spot to the device.

[1056] Relive your travel memories

[1057] Users can enter memories of past trips and upload photos and notes within the app. The device sends this data to the server, which analyzes the data and generates virtual travel content. The generated content is sent to the device, where users can view and play it to virtually relive past trips.

[1058] Virtual travel at home

[1059] Users select the "Virtual Travel" mode within the app and input their desired travel destination. The device sends the selected information to the server, which then generates a virtual sightseeing tour based on that information. The virtual sightseeing tour is created using a generative AI model. The generated tour data is sent to the device, allowing users to enjoy the virtual tour at home via streaming playback.

[1060] Optimizing user experience with an emotion engine

[1061] The device captures the user's facial expressions and voice in real time and uses an emotion analysis engine to obtain emotional data. The emotional data is sent to a server, which then adjusts the on-site guidance data and the difficulty of the questions based on that data. The adjusted data is then sent back to the device, allowing the user to receive customized guidance and quizzes tailored to their own state.

[1062] Specific examples

[1063] For example, if a user selects "Visit historical spots in Nara," the smartphone's GPS acquires the user's current location and sends it to the server. The server generates guide information about the "Nara period" and sends it to the device. The device then begins providing audio and text guidance. If the user selects quiz mode, a sample question such as "What is a representative building from the Nara period?" is displayed. If the user answers "Todaiji Temple," the server evaluates the answer and displays guidance to the next spot.

[1064] An example of a prompt sentence is "Please generate a prompt sentence for a system in which a user uses the emotion engine at tourist spots in Nara and is shown a quiz about the Nara period."

[1065] This will allow people to gain a deeper understanding of local history and culture and experience the joys of travel without having to actually visit the area. It will also make it possible to offer the joys of travel in a new way to elderly people and those with mobility difficulties.

[1066] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1067] Step 1:

[1068] The user installs the application on their smartphone.

[1069] The user downloads and installs the application from the smartphone's app store, then launches the application.

[1070] Input: App store download link

[1071] Output: Installed applications

[1072] Step 2:

[1073] The user launches the application and enters profile information.

[1074] Users launch the application and fill in fields with information such as their name, age, and historical topics of interest.

[1075] Input: Profile information entered by the user (name, age, topics of interest)

[1076] Output: Temporarily saved profile information

[1077] Step 3:

[1078] The device checks the entered information and temporarily stores it.

[1079] The terminal checks whether the input is in the correct format (for example, whether the name is not empty or the age is in numeric format), and then stores the information temporarily in memory.

[1080] Input: Profile information entered by the user

[1081] Output: Format-checked and temporarily saved profile information

[1082] Step 4:

[1083] The device sends the profile information to the server.

[1084] The device sends the checked profile information to the server using the HTTPS protocol.

[1085] Input: Format-checked and temporarily saved profile information

[1086] Output: Profile information sent to the server

[1087] Step 5:

[1088] The server analyzes the received information and stores it in a database.

[1089] The server parses the received profile information in JSON format and stores it in a database, for example, adding the user ID and timestamp.

[1090] Input: Profile information sent to the server

[1091] Output: Profile information stored in the database

[1092] Step 6:

[1093] The server sends a message to the terminal indicating that the information has been saved.

[1094] The server confirms that the data has been saved to the database, generates a message indicating that the data has been saved, and sends it to the terminal.

[1095] Input: Profile information stored in the database

[1096] Output: Message indicating information has been saved

[1097] Step 7:

[1098] The terminal displays a confirmation message to the user.

[1099] The terminal displays a message indicating that the information has been saved as received from the server as a pop-up notification or on the screen.

[1100] Input: Message that information has been saved

[1101] Output: User notification

[1102] Step 8:

[1103] The device periodically obtains the user's current location using GPS.

[1104] The device uses the smartphone's GPS module to obtain the user's current location at regular intervals (e.g., every 5 minutes).

[1105] Input: None (periodic acquisition of GPS module)

[1106] Output: User's current location data (latitude, longitude)

[1107] Step 9:

[1108] The terminal transmits the obtained current location data to the server.

[1109] The device converts the acquired location data into JSON format and sends it to the server using the HTTPS protocol.

[1110] Input: User's current location data (latitude, longitude)

[1111] Output: Location data sent to the server

[1112] Step 10:

[1113] The server analyzes the user's current location and confirms that it is a tourist spot.

[1114] The server compares the received current location data with a list of pre-registered tourist spots and analyzes whether the user's location is within a tourist spot.

[1115] Input: Location data sent to the server

[1116] Output: Tourist destination determination result

[1117] Step 11:

[1118] The server generates tourist destination information.

[1119] If the server confirms that the destination is a tourist spot, it generates information about the historical background and attractions of the destination.

[1120] Input: Tourist destination determination result

[1121] Output: Generated tourist destination information

[1122] Step 12:

[1123] The server transmits local guide data to the terminal.

[1124] The server sends the generated tourist destination information to the terminal in JSON format.

[1125] Input: Generated tourist destination information

[1126] Output: Local guide data sent to the terminal

[1127] Step 13:

[1128] The terminal displays the received local guidance data to the user and plays back the audio guide.

[1129] The device displays the received guidance data on the display, plays the audio guide file, and may notify the user using push notifications or alerts.

[1130] Input: Local guide data sent to the terminal

[1131] Output: Display to user, play audio guide

[1132] Step 14:

[1133] The user selects a puzzle game or quiz mode within the app.

[1134] Users select the "Play" tab from the in-app menu and choose their preferred mode.

[1135] Input: Mode selected by user

[1136] Output: Selected mode information

[1137] Step 15:

[1138] The terminal transmits the user's selection information to the server.

[1139] The device converts the selected mode information into JSON format and sends it to the server.

[1140] Input: Selected mode information

[1141] Output: Mode information sent to the server

[1142] Step 16:

[1143] The server generates questions related to local history and culture.

[1144] Based on the received mode information, the server uses a generative AI model to generate quizzes and questions about local history and culture.

[1145] Input: Mode information sent to the server

[1146] Output: Generated quizzes and questions

[1147] Step 17:

[1148] The server generates questions and sends them to the device.

[1149] The server sends the generated questions to the terminal in JSON format.

[1150] Input: Generated quizzes and questions

[1151] Output: Problems sent to terminal

[1152] Step 18:

[1153] The terminal presents the problem to the user.

[1154] The terminal displays the received questions on the screen so that the user can answer them.

[1155] Input: Question sent to terminal

[1156] Output: The problem presented to the user

[1157] Step 19:

[1158] The user answers the question.

[1159] Users can answer the questions by tapping on the appropriate answer option or by writing their own answer.

[1160] Input: User response

[1161] Output: User response data

[1162] Step 20:

[1163] The terminal sends the user's answer to the server.

[1164] The device converts the user's response into JSON format and sends it to the server.

[1165] Input: User response data

[1166] Output: Response data sent to the server

[1167] Step 21:

[1168] The server evaluates the user's answer and determines whether it is correct.

[1169] Based on the received answers, the server runs an algorithm to evaluate whether the answers are correct or incorrect and generates a judgment result.

[1170] Input: Response data sent to the server

[1171] Output: Judgment result (correct or incorrect)

[1172] Step 22:

[1173] The server generates information about the next spot and sends it to the terminal.

[1174] Based on the results of the determination, the server generates information about the next tourist spot and guidance data to that location, and sends it to the terminal.

[1175] Input: Judgment result

[1176] Output: Next spot information and guidance data

[1177] Step 23:

[1178] The terminal displays information and directions to the next spot to the user.

[1179] The device displays the received next spot information and provides the user with information such as maps and the shortest route.

[1180] Input: Next spot information and guidance data

[1181] Output: Next stop information displayed to the user

[1182] (Application example 2)

[1183] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1184] Conventional travel support systems have the problem that users cannot experience sightseeing unless they go to the destination, making it difficult for elderly people and those with mobility difficulties to enjoy the pleasure of traveling. In addition, the user experience is not personalized enough, and tourist information is provided uniformly. This makes it difficult to sufficiently increase user satisfaction.

[1185] Furthermore, virtual stores do not provide services that take into account the user's emotional state, making it difficult to provide a personalized shopping experience.

[1186] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1187] In this invention, the server includes means for inputting profile information through an application installed by the user and transmitting the information to the server, means for confirming the user's current location based on GPS data, means for generating local guidance data based on the confirmed current location and transmitting the data to the terminal, means for the terminal to display and play back the received local guidance data to the user, means for exploring a virtual store selected by the user, means for analyzing the user's emotions using an emotion engine and suggesting products and services based on the analysis results, and means for the terminal to display the suggested products and services to the user. This allows even elderly people and those with mobility difficulties to enjoy virtual travel, and enables personalized tourist information and shopping experiences tailored to the user's emotional state.

[1188] "Profile Information" means personal data such as a User's name, age, and interests.

[1189] "Server" refers to a device that receives, stores, and analyzes data from users, and generates and transmits information as instructed.

[1190] "GPS data" refers to data that includes location information for identifying a user's current location.

[1191] "Local guide data" means data including tourist information and guide information about the place the user is visiting.

[1192] A "terminal" is a device that is directly operated by a user, and includes, for example, a smartphone or a tablet.

[1193] A "virtual store" refers to a store that does not have a physical presence, but operates in a virtual space.

[1194] An "emotion engine" refers to software or hardware that analyzes a user's facial expressions, voice, etc. and recognizes their emotional state.

[1195] "Product and service suggestions" refers to the act of recommending the most suitable products and services to users based on the analysis results of the emotion engine.

[1196] "Quiz and mini-games" refers to entertainment elements in which users can participate and receive rewards or discounts based on their scores or results.

[1197] "Benefits and discounts" refers to benefits and discounts offered to users depending on the results of quizzes and mini-games.

[1198] "Virtual travel content" means content that recreates past travel data or uploaded information and allows users to virtually experience travel.

[1199] "Facial and voice analysis" refers to the process of analyzing facial and voice data to recognize the user's emotional state.

[1200] The present invention is a system implemented in the following specific steps: First, a user installs an application on a smartphone or head-mounted display and enters profile information. The application acquires information such as the user's name, age, and interests, and sends the information to a server. The server then stores the profile information in a database and manages the user's basic information.

[1201] Next, the user launches the application and can explore the virtual store. The application continuously acquires the user's current location using the GPS function and sends the data to the server. The server analyzes the acquired current location data and identifies the location within the virtual store selected by the user. Based on the user's location, the server then generates local guide data and product information for the virtual store and sends it to the device. The device then displays the received data to the user, providing intuitive navigation.

[1202] Furthermore, an emotion engine is used to analyze the user's emotional state. Specifically, the device acquires the user's facial expressions and voice data and sends that data to the server. The server then uses the emotion engine to analyze the user's emotions. Based on the analysis results, the server suggests the most suitable products and services for the user and sends that information to the device. This allows the device to display the suggested products and services to the user. For example, if the user has a happy expression, the server can provide special campaign and discount information tailored to that emotion.

[1203] Users can participate in quizzes and mini-games within the virtual store, which adds an entertainment element and improves the user experience. Based on the results of the quizzes and mini-games, rewards and discounts are offered, allowing users to enjoy shopping and earn rewards.

[1204] On the other hand, users can input memories of past trips or upload photos and notes. The server then analyzes the uploaded data and generates content that recreates a virtual trip. The content is sent to the device, allowing users to virtually re-experience past trips. This function is particularly useful for elderly people and those with mobility difficulties, who can enjoy virtual travel.

[1205] Specific examples

[1206] For example, consider a scenario where a 35-year-old user installs an application and explores a virtual store. The user's profile information includes their name, age, and categories of interest (e.g., electronics, books, and fashion). Using an emotion engine, the system suggests products that the user is interested in. For example, new smartphones, best-selling books, and trending fashion items may be suggested. The user can also participate in a quiz and earn discount coupons based on their score.

[1207] Example prompt for a generative AI model:

[1208] "A 35-year-old user is interested in electronics, books, and fashion. He enters a virtual store using his smartphone. Based on sentiment analysis, recommend the following products to him:

[1209] 1. The latest smartphone

[1210] 2. Bestselling books

[1211] 3. Trendy fashion items

[1212] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1213] Step 1:

[1214] A user installs an application on a smartphone or head-mounted display, which involves a download and installation process from an application store. The input is the application installation file, and the output is the installed application.

[1215] Step 2:

[1216] The user launches the application and enters profile information. Information such as name, age, and interest categories is received as input from the user, and the device temporarily stores that information. The input is the profile information entered directly by the user, and the output is the temporarily stored profile information.

[1217] Step 3:

[1218] The device sends profile information to the server. The server receives this information and stores it in a database. The input is the profile information sent from the device, and the output is the profile information stored in the database.

[1219] Step 4:

[1220] The user selects a virtual store and begins searching. The device uses its GPS function to obtain current location data and sends it to the server. The input is the device's GPS data, and the output is the current location data sent to the server.

[1221] Step 5:

[1222] The server analyzes the acquired current location data and identifies the location within the virtual store selected by the user. The server generates local guidance data and product information based on that location and sends it to the terminal. The input is the current location data, and the output is the generated local guidance data and product information.

[1223] Step 6:

[1224] The terminal displays the received local guide data and product information to the user. The display can be in the form of text, images, audio, etc. The input is the local guide data and product information sent from the server, and the output is the information displayed to the user.

[1225] Step 7:

[1226] The device acquires the user's facial expression and voice data and sends it to the server. The server uses an emotion engine to analyze the user's emotional state. The input is facial expression and voice data, and the output is the emotion analysis results.

[1227] Step 8:

[1228] The server generates data to suggest optimal products and services to the user based on the results of the emotion analysis and sends it to the terminal. The input is the emotion analysis results, and the output is the generated product and service suggestion data.

[1229] Step 9:

[1230] The terminal displays the suggested product and service data to the user, and the user can proceed with shopping based on that information. The input is the suggested data sent from the server, and the output is the product and service information displayed to the user.

[1231] Step 10:

[1232] Users participate in quizzes and mini-games in a virtual store. The terminal sends the game results to the server, which then generates rewards and discount data based on the results. The input is the user's game results, and the output is the rewards and discount data.

[1233] Step 11:

[1234] The server sends the generated reward and discount data to the terminal, which then displays it to the user. The input is the reward and discount data, and the output is the reward information displayed to the user.

[1235] Step 12:

[1236] Users input memories of past travels or upload photos and notes. The device sends the data to the server, which then generates content that recreates the virtual trip. The input is the data uploaded by the user, and the output is the virtual trip content.

[1237] Step 13:

[1238] The server sends the generated virtual travel content to the terminal, which then displays and plays it for the user. The input is the virtual travel content, and the output is the virtual travel content that is displayed and played for the user.

[1239] Examples:

[1240] For example, if a user is looking to buy a fashion item in a virtual store, sentiment analysis will suggest new items that the user is interested in. Users can also participate in quizzes and earn discount coupons depending on their scores.

[1241] Example prompt for a generative AI model:

[1242] "A 35-year-old user is interested in electronics, books, and fashion. He enters a virtual store using his smartphone. Based on sentiment analysis, recommend the following products to him:

[1243] 1. The latest smartphone

[1244] 2. Bestselling books

[1245] 3. Trendy fashion items

[1246] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1247] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1248] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1249] [Third embodiment]

[1250] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1251] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1252] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1253] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1254] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1255] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1256] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1257] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1258] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1259] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1260] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1261] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1262] Overall flow

[1263] This invention is a system that makes it easier for travelers to enjoy the attractions of rural areas outside of cities, enriching the local tourist experience while also providing a similar experience in remote areas. Furthermore, it also provides the enjoyment of travel to elderly people who find it difficult to travel to the local area. The system supports the tourist experience by linking the user's smartphone with a server.

[1264] User registration and information management

[1265] 1. The user installs a dedicated application on their smartphone.

[1266] 2. The user launches the application and enters profile information such as name, age, and historical topics of interest.

[1267] 3. The device sends the entered information to the server.

[1268] 4. The server stores the received information in a database and sends a confirmation message to the user.

[1269] Automatic guidance upon arrival at destination

[1270] 1. The device periodically obtains the user's current location using GPS.

[1271] 2. The device sends the acquired location data to the server.

[1272] 3. The server analyzes the user's current location and verifies that it is one of the tourist destinations set.

[1273] 4. The server generates local guidance data and sends it to the terminal.

[1274] 5. The terminal displays and plays the received guidance data (audio guide, text, images) to the user.

[1275] Sightseeing with a Game-like Feel

[1276] 1. The user selects a puzzle game or quiz mode within the app.

[1277] 2. The server generates questions about local history and culture according to the selected mode and sends them to the terminal.

[1278] 3. The device presents the user with a question, collects the user's answers, and sends them to the server.

[1279] 4. The server evaluates the user's answer, generates information about the next spot, and sends it to the device.

[1280] 5. The device displays information about the next spot and provides directions to the location to the user.

[1281] Relive your travel memories

[1282] 1. Users enter memories of past trips or upload photos and notes within the app.

[1283] 2. The server parses the uploaded data.

[1284] 3. The server generates content (in the form of videos or slideshows) that recreates the virtual journey.

[1285] 4. The terminal displays and plays the generated virtual travel content to the user.

[1286] Virtual travel at home

[1287] 1. The user selects the "Virtual Travel" mode within the app.

[1288] 2. The server collects information on the desired travel destinations based on the user's selection and generates a virtual sightseeing tour.

[1289] 3. The device streams the generated virtual tour and provides it to the user.

[1290] Specific examples

[1291] For example, consider a case where a user visits a historical spot in Nara. When the user arrives at the location, the smartphone automatically displays an audio guide and text about the Nara period. Furthermore, if the user selects the quiz mode, the smartphone asks the question, "What is a representative building from the Nara period?" If the answer is correct, directions to the next spot will be displayed.

[1292] If a user wants to relive a past trip to Kyoto, they can upload past photos to the app. The server analyzes the photos and generates a video that virtually recreates the trip to Kyoto. The user can then enjoy the video at home.

[1293] This will enable people to gain a deeper understanding of local history and culture and experience the joy of travel without having to go to the actual place. Virtual travel will also be a new way to provide the joy of travel to elderly people and those with mobility difficulties.

[1294] The processing flow will be explained below.

[1295] User registration and information management

[1296] Step 1:

[1297] The user downloads and installs the application on their smartphone.

[1298] Step 2:

[1299] A user launches the application and enters profile information such as name, age, and historical topics of interest.

[1300] Step 3:

[1301] The device checks the entered information and temporarily stores it.

[1302] Step 4:

[1303] The device sends the profile information to the server.

[1304] Step 5:

[1305] The server analyzes the received information and stores it in a database.

[1306] Step 6:

[1307] The server sends a message to the terminal indicating that the information has been saved.

[1308] Step 7:

[1309] The terminal displays a confirmation message to the user.

[1310] Automatic guidance upon arrival at destination

[1311] Step 1:

[1312] The device periodically obtains the user's current location using GPS.

[1313] Step 2:

[1314] The terminal transmits the obtained current location data to the server.

[1315] Step 3:

[1316] The server analyzes the user's current location and checks whether it is the tourist spot you have set.

[1317] Step 4:

[1318] If the server determines that the area is a tourist destination, it generates local guide data for the area.

[1319] Step 5:

[1320] The server transmits local guide data to the terminal.

[1321] Step 6:

[1322] The terminal displays the received local guidance data to the user, and the audio guide is automatically played.

[1323] Sightseeing with a Game-like Feel

[1324] Step 1:

[1325] The user selects a puzzle game or quiz mode within the app.

[1326] Step 2:

[1327] The terminal transmits the user's selection information to the server.

[1328] Step 3:

[1329] The server generates questions about local history and culture depending on the mode selected.

[1330] Step 4:

[1331] The server generates questions and sends them to the device.

[1332] Step 5:

[1333] The terminal presents the problem to the user.

[1334] Step 6:

[1335] The user answers the question.

[1336] Step 7:

[1337] The terminal sends the user's answer to the server.

[1338] Step 8:

[1339] The server evaluates the user's answer and determines whether it is correct.

[1340] Step 9:

[1341] The server generates information about the next spot and sends it to the terminal.

[1342] Step 10:

[1343] The device displays information about the next spot and provides directions to the location to the user.

[1344] Relive your travel memories

[1345] Step 1:

[1346] Users can enter memories of past trips within the app, or upload photos and notes.

[1347] Step 2:

[1348] The device sends the uploaded information to the server.

[1349] Step 3:

[1350] The server parses the data it receives.

[1351] Step 4:

[1352] The server generates content that recreates the virtual journey.

[1353] Step 5:

[1354] The server transmits the generated content to the terminal.

[1355] Step 6:

[1356] The terminal displays and plays the virtual travel content to the user.

[1357] Virtual travel at home

[1358] Step 1:

[1359] The user selects the "Virtual Travel" mode within the app.

[1360] Step 2:

[1361] The terminal transmits the selection information to the server.

[1362] Step 3:

[1363] The server generates a virtual sightseeing tour based on information about the desired travel destination.

[1364] Step 4:

[1365] The server transmits the generated virtual tour data to the terminal.

[1366] Step 5:

[1367] The terminal streams the virtual tour and provides it to the user.

[1368] Example 1

[1369] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1370] Conventional tourism systems often lack tourist information when visiting rural areas outside of cities, making it difficult to fully enjoy the attractions of those areas. There is also the problem that tourism itself is difficult for elderly people and people with mobility restrictions who have difficulty traveling to those areas. Furthermore, there are limited ways to recreate and re-enjoy memories of past trips. The goal of this project is to solve these problems and make travel and tourism attractive and easy for everyone to enjoy.

[1371] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1372] In this invention, the server includes a means for users to send profile information, a means for checking current location based on GPS data, and a means for generating local guide data based on the current location. This allows users to easily enjoy the attractions of local areas. The server also includes a means for inputting memories of past travels or uploading photos and notes, a means for analyzing the uploaded data and generating content that recreates a virtual travel, and a means for displaying and playing the generated virtual travel content to the user, allowing users to relive past travels. Furthermore, the server includes a means for selecting a "virtual travel" mode, a means for collecting information on desired travel destinations and generating a virtual sightseeing tour, and a means for streaming and providing the generated virtual tour to the user, allowing users who have difficulty visiting the destinations to enjoy the fun of travel.

[1373] "User" refers to a human user who interfaces with the server and terminal.

[1374] "Device" refers to a smartphone, tablet, or other mobile information device on which a user installs and uses an application.

[1375] "Application" refers to a software program that a user installs on a terminal and uses.

[1376] "Profile Information" refers to personal information about you, such as your name, age, and topics of interest.

[1377] "Server" means a remote computer system that processes and stores data received from a client device and executes server-side actions.

[1378] "GPS Data" refers to your location information obtained using the Global Positioning System.

[1379] "Current Location" refers to a user's real-time geographic location as determined based on GPS data.

[1380] "Local guide data" refers to data containing information about a specific tourist destination (audio guide, text, images, etc.).

[1381] "Virtual travel content" refers to content (such as videos or slideshows) that is displayed and played back to users and is generated based on past travel memories and uploaded data.

[1382] "Virtual travel" refers to a sightseeing tour that allows you to collect information about your desired travel destination and experience it virtually without physically going there.

[1383] "Questions" refer to quizzes and puzzles about history and culture that are generated by the server and presented to the user.

[1384] A "mode" refers to a particular type of operation or function that a user selects within an application.

[1385] This invention is a system for enhancing the appeal of visiting regional tourist destinations outside of urban areas. This system links the user's smartphone with a server, aiming to enrich the tourist experience on-site while also providing a similar experience in remote locations. Furthermore, it is possible to provide the enjoyment of travel to elderly users and users with mobility difficulties.

[1386] 1. User registration and information management

[1387] Users first install a dedicated application on their smartphone. They then launch the application and enter their profile information, such as their name, age, and historical topics of interest. The information is then sent from the device to a server, which stores it in a database. This is how the user profile is managed.

[1388] 2. Automatic guidance when arriving at your destination

[1389] The user's device periodically obtains its current location using GPS and sends this data to the server. The server analyzes the current location data and confirms that the user has arrived at the tourist destination. The server generates local guide data about the tourist destination and sends it to the device. The device receives this data and displays and plays audio guides, text, images, etc. to the user. Technologies used include Google Maps API and Azure Cognitive Services.

[1390] 3. Sightseeing like a game

[1391] Within the app, users can select a puzzle game or quiz mode. Depending on the selected mode, the server generates questions about local history and culture and sends them to the device. The device presents the questions to the user and collects their answers. The server evaluates the answers, generates information about the next spot, and sends it to the device. The device then displays information about the next spot and directions to the location to the user. A specific example is the question, "What is a representative building from the Nara period?", and the user inputs the answer.

[1392] 4. Relive your travel memories

[1393] Users can input memories of past travels and upload photos and notes. The server analyzes the uploaded data and generates content that recreates the virtual trip. The device displays and plays the generated virtual trip content to the user. Technologies used include Amazon Rekognition and Adobe Premiere API. An example prompt is, "Users upload photos of travel destinations they have visited in the past, and a video recreating that trip is generated and displayed."

[1394] 5. Virtual travel at home

[1395] Users can select the "Virtual Travel" mode within the app. The server collects information on the desired travel destination based on the user's selection and generates a virtual sightseeing tour. The device streams the generated virtual tour and provides it to the user. The technology used includes the Google Earth API. An example of a specific prompt is, "The user selects the 'Virtual Travel' mode and a virtual tour of Paris begins."

[1396] This makes it possible for the present invention to increase the enjoyment of traveling to rural areas, providing a similar experience from remote locations, and bringing the joy of travel to elderly people and those with mobility difficulties.

[1397] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1398] Step 1:

[1399] The user installs the dedicated application on their smartphone. The user downloads the application from the app store and completes the installation procedure. The application can then be used on the smartphone.

[1400] Step 2:

[1401] A user launches the application and enters profile information (name, age, historical topics of interest). The information entered by the user is temporarily stored on the device. Profile information is constructed based on the input (e.g., name "Taro Tanaka," age "65," interest "ancient history").

[1402] Step 3:

[1403] The device sends the profile information to the server. The input data from the device is transferred to the server via the Internet and received by the server. The received profile information is saved in the server's database. The server generates a save completion message and sends it to the device. The device displays a "Registration Complete" message to the user.

[1404] Step 4:

[1405] The device periodically obtains the user's current location using GPS. The device uses the GPS sensor to collect location information and generate that data. The generated GPS data is stored on the device.

[1406] Step 5:

[1407] The device transmits the acquired current location data to the server. The location information data from the device is transferred to the server and received by the server. The received data is sent to the server's analysis module.

[1408] Step 6:

[1409] The server analyzes the current location data and confirms that the user has arrived at the designated tourist spot. The server compares the location data with tourist spot information in the database to determine whether the user's current location matches the tourist spot. If the location information matches the tourist spot, the server generates local guidance data.

[1410] Step 7:

[1411] The server generates local guidance data (audio guide, text, images) and sends it to the terminal. The generated guidance data is transferred from the server to the terminal. The terminal displays and plays the received local guidance data to the user. The user plays the audio guide and receives the information.

[1412] Step 8:

[1413] The user selects a puzzle game or quiz mode within the app. The user operates the mode selection screen and selects the desired game mode. The selected mode information is saved on the device.

[1414] Step 9:

[1415] The server generates questions about local history and culture according to the selected mode and sends them to the device. The server generates question data based on the game mode and transfers the data to the device. The device presents the questions to the user and collects the user's answers.

[1416] Step 10:

[1417] The terminal sends the user's answer data to the server. The answer data from the terminal is transferred to the server and received by the server. The received data is sent to the server's analysis module.

[1418] Step 11:

[1419] The server evaluates the user's answers, generates information about the next spot, and sends it to the device. The server analyzes the answer data and generates information about the next tourist spot based on the evaluation results. The generated spot information is then transferred to the device.

[1420] Step 12:

[1421] The device displays information about the next spot and guidance to the user. The device receives the spot information and displays it to the user. The user confirms the guidance to the next spot and begins moving.

[1422] Step 13:

[1423] The user inputs memories of past trips or uploads photos and notes. The user inputs past trip information within the app or selects and uploads photo files. The input and uploaded data is saved on the device.

[1424] Step 14:

[1425] The device sends the uploaded data to the server. The uploaded data from the device is transferred to the server and received by the server. The received data is sent to the server's analysis module.

[1426] Step 15:

[1427] The server analyzes the uploaded data and generates virtual travel content. The server processes the data using image analysis technology and generates content for recreating the virtual travel. The generated content is stored on the server.

[1428] Step 16:

[1429] The server sends the virtual travel content to the terminal. The generated content data is transferred from the server to the terminal. The terminal receives the virtual travel content and displays and plays it for the user. The user experiences the virtual travel.

[1430] Step 17:

[1431] The user selects the "Virtual Travel" mode within the app. The user operates the mode selection screen and selects the Virtual Travel mode. The selection information is saved on the device.

[1432] Step 18:

[1433] The server collects information on the desired travel destination based on the user's selection and generates a virtual sightseeing tour. The server extracts the travel destination information from the database and creates a virtual tour based on the collected data. The generated tour information is saved on the server.

[1434] Step 19:

[1435] The server sends the generated virtual tour to the terminal. The tour information data is transferred from the server to the terminal. The terminal receives the virtual tour and starts streaming playback. The user experiences the virtual trip.

[1436] (Application example 1)

[1437] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1438] There is a need to provide tourism experiences for elderly people and travel enthusiasts living in remote areas who find it difficult to visit local sites. However, current systems rely on on-site visits and lack ways to enhance tourism experiences in virtual environments. This invention aims to solve this problem by providing a virtual travel system that allows users to enjoy the charms of local areas from the comfort of their own homes.

[1439] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1440] In this invention, the server includes means for inputting profile information through an application installed by the user and transmitting the information to the server, means for confirming the user's current location based on GPS data, means for generating local guidance data based on the confirmed current location and transmitting it to the terminal, means for the user to select a mode within the application for enjoying a virtual trip at home, means for generating virtual sightseeing tour content based on the selected travel destination and transmitting it to the terminal, and means for displaying and playing the received virtual sightseeing tour content to the user, thereby enabling the user to enjoy an advanced virtual sightseeing experience from the comfort of their own home.

[1441] "Profile Information" refers to personal information such as a User's name, age, and topics of interest that is used by the Application to provide the User with an optimal tourism experience.

[1442] "GPS Data" means location information obtained using the Global Positioning System and is used to determine a user's current location.

[1443] "Local guide data" refers to data that includes information such as audio guides, text, and images about tourist spots that users visit, and is provided to support the user's sightseeing experience.

[1444] "Device" refers to an electronic device used by a user, such as a smartphone, tablet, or head-mounted display, which is used to display and play data sent from the server.

[1445] "Virtual travel" refers to the process by which users virtually visit tourist spots through an application and experience their culture and history without actually traveling.

[1446] "Virtual sightseeing tour content" refers to content such as 3D visuals of tourist spots, audio guides, and text that is generated by the server and sent to the user's terminal, and is the main information that allows the user to experience a virtual trip.

[1447] "Quiz mode" refers to a function that, when selected by the user, causes the application to ask questions about tourist spots, and allows the user to enjoy sightseeing in a game-like manner by answering the questions.

[1448] A "head-mounted display" is a display device worn by the user on the head to display virtual reality (VR) and augmented reality (AR) content, and is used to provide an advanced virtual experience.

[1449] The present invention provides a virtual travel system that allows users to experience the attractions of local areas from the comfort of their own homes. Detailed embodiments are described below.

[1450] System Configuration

[1451] This system consists of an application installed by the user, a terminal (such as a smartphone or a head-mounted display), and a server.

[1452] Registering profile information and acquiring GPS data

[1453] Through the application, users enter profile information such as their name, age, and topics of interest, and submit that information to the server, which stores the user's profile information in a database.

[1454] The device periodically acquires the user's current location using GPS and sends that data to the server, which then analyzes the GPS data to confirm the user's current location.

[1455] Generation and display of on-site guidance data

[1456] The server generates local guidance data based on the user's current location and sends it to the device. The device then displays and plays the received local guidance data to the user as audio guides, text, and images, allowing the user to obtain a wealth of information about the local history and culture.

[1457] Providing virtual travel modes

[1458] Within the application, users can select a mode to enjoy a virtual trip at home. The server generates virtual sightseeing tour content based on the selected travel destination and sends it to the device. The device then displays and plays the received virtual sightseeing tour content to the user as an audio guide or 3D visuals. Specifically, users can experience the virtual trip through a smartphone or head-mounted display.

[1459] Quiz mode available

[1460] The application has a quiz mode. Based on the quiz mode selected by the user, the server generates questions about local history and culture and sends them to the device. The device presents the questions to the user, collects the user's answers, and sends them to the server. The server evaluates the user's answers, generates information about the next spot, and sends it to the device. This allows the user to enjoy sightseeing in a game-like manner.

[1461] Recreating a virtual journey

[1462] Users can input memories of past trips or upload photos and notes. The server analyzes the uploaded data and generates content that recreates a virtual trip. The device displays and plays the generated virtual trip content to the user.

[1463] Hardware and software used

[1464] This system uses the following hardware and software:

[1465] Hardware: Smartphone, Head-Mounted Display (HMD), Server

[1466] Software: Application, server program, HTTP request library (requests), JSON parser

[1467] Examples of concrete examples and prompts

[1468] As a concrete example, consider a scenario in which a user experiences historical spots in Nara from the comfort of their own home. The user launches the app and selects the "Virtual Travel" mode. The server generates an audio guide and text about Nara culture based on the user's profile data and selected travel destination, and sends them to the device along with 3D visuals. The user puts on a head-mounted display and experiences Nara's history in a virtual environment. In addition, in quiz mode, the user is asked the question, "What is a representative building from the Nara period?" and by answering the question, directions to the next spot are displayed.

[1469] Below is an example of a prompt sentence to input to the generative AI model.

[1470] Prompt Sentence Examples

[1471] ---

[1472] We are developing an application that allows users to enjoy virtual travel from the comfort of their own home. The application runs on smartphones and head-mounted displays (HMDs) and provides users with a virtual sightseeing experience related to local culture and history.

[1473] example:

[1474] The user uses the app to experience Nara's historical spots. The app retrieves audio guides and text about Nara based on the user's profile data, and displays them in 3D visuals through the HMD. The app also has a quiz mode, and after the user answers questions, the app guides the user to the next virtual spot.

[1475] Expected output:

[1476] Audio guide about historical spots from the Nara period

[1477] Coding audio files for broadcast

[1478] A quiz about representative buildings from the Nara period

[1479] Directions to the next virtual spot

[1480] Please explain in detail how this application works.

[1481] ---

[1482] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1483] Step 1:

[1484] A user installs the application, enters profile information (such as name, age, and topics of interest), and sends that information to the server, which stores the information in a database and sends a confirmation message to the user.

[1485] Input: User profile information Output: Confirmation message

[1486] Step 2:

[1487] The device periodically acquires GPS data and sends the user's current location to the server. The server analyzes the received GPS data and confirms the user's current location. When the user arrives at the location, the server generates local guidance data and sends it to the device.

[1488] Input: GPS data Output: Local guidance data

[1489] Step 3:

[1490] The device displays and plays the received local information to the user. Specifically, it displays audio guides, text, and images, allowing the user to learn about the history and culture of the area.

[1491] Input: On-site guidance data Output: Displayed / played guidance data

[1492] Step 4:

[1493] The user selects the "Virtual Travel" mode in the application. The terminal sends this selection to the server. The server generates virtual sightseeing tour content based on the selected travel destinations and sends it to the terminal.

[1494] Input: Virtual travel mode selection information Output: Virtual sightseeing tour content

[1495] Step 5:

[1496] The device displays and plays the received virtual sightseeing tour content to the user. When using a head-mounted display, 3D visuals are displayed, allowing the user to experience traveling in a virtual environment.

[1497] Input: Virtual sightseeing tour content Output: Displayed / played tour content

[1498] Step 6:

[1499] The user selects the quiz mode. The terminal transmits this selection information to the server. The server generates questions about the local history and culture and transmits them to the terminal.

[1500] Input: Quiz mode selection information Output: Quiz question data

[1501] Step 7:

[1502] The terminal presents the received quiz question data to the user, collects the user's answers, and sends them to the server. The server evaluates the user's answers, generates guidance data for the next spot, and sends it to the terminal. The terminal displays it to the user.

[1503] Input: User response data Output: Next spot guidance data

[1504] Step 8:

[1505] Users input memories of past trips or upload photos and notes. The device sends this data to the server, which analyzes the uploaded data, generates content that recreates the virtual trip, and sends it to the device.

[1506] Input: Past travel data Output: Virtual travel reenactment content

[1507] Step 9:

[1508] The device displays and plays the received virtual travel recreation content to the user, allowing the user to enjoy memories of past trips from the comfort of their own home.

[1509] Input: Virtual travel content Output: Displayed / played travel content

[1510] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1511] This system makes it easier for travelers to enjoy the attractions of rural areas outside of cities, enriching the local tourist experience while also providing a similar experience in remote locations. It also provides the enjoyment of travel to elderly people who find it difficult to travel to the local area. The system supports the tourist experience by linking the user's smartphone with a server and an emotion engine.

[1512] User registration and information management

[1513] 1. The user installs a dedicated application on their smartphone.

[1514] 2. The user launches the application and enters profile information such as name, age, and historical topics of interest.

[1515] 3. The device checks the entered information and temporarily stores it.

[1516] 4. The device sends the profile information to the server.

[1517] 5. The server analyzes the received information and stores it in a database.

[1518] 6. The server sends a message to the device indicating that the information has been saved.

[1519] 7. The terminal displays a confirmation message to the user.

[1520] Automatic guidance upon arrival at destination

[1521] 1. The device periodically obtains the user's current location using GPS.

[1522] 2. The device sends the location data it has obtained to the server.

[1523] 3. The server analyzes the user's current location and confirms that it is the tourist spot that was set.

[1524] 4. If the server determines that the area is a tourist destination, it generates local guide data for the area.

[1525] 5. The server sends the local guide data to the terminal.

[1526] 6. The device displays the received local guidance data to the user, and the audio guide is automatically played.

[1527] Sightseeing with a Game-like Feel

[1528] 1. The user selects a puzzle game or quiz mode within the app.

[1529] 2. The terminal sends the user's selection information to the server.

[1530] 3. The server generates questions about local history and culture depending on the mode selected.

[1531] 4. The server sends the generated questions to the device.

[1532] 5. The device presents the problem to the user.

[1533] 6. The user answers the question.

[1534] 7. The device sends the user's answer to the server.

[1535] 8. The server evaluates the user's answer and determines whether it is correct.

[1536] 9. The server generates information about the next spot and sends it to the terminal.

[1537] 10. The device displays information about the next spot and directions to the location to the user.

[1538] Relive your travel memories

[1539] 1. Users enter memories of past trips or upload photos and notes within the app.

[1540] 2. The device sends the uploaded information to the server.

[1541] 3. The server analyzes the received data.

[1542] 4. The server generates content that recreates the virtual journey.

[1543] 5. The server sends the generated content to the device.

[1544] 6. The device displays and plays the virtual travel content to the user.

[1545] Virtual travel at home

[1546] 1. The user selects the "Virtual Travel" mode within the app.

[1547] 2. The device sends the selection information to the server.

[1548] 3. The server generates a virtual sightseeing tour based on information about the desired travel destination.

[1549] 4. The server sends the generated virtual tour data to the device.

[1550] 5. The device streams the virtual tour and provides it to the user.

[1551] Optimizing user experience with an emotion engine

[1552] 1. The device analyzes the user's facial expressions and voice using an emotion engine to recognize emotions.

[1553] 2. The server adjusts the local guidance data and the difficulty of the game questions based on the recognized emotion data.

[1554] 3. The server sends the data generated based on the emotion to the device.

[1555] 4. The terminal displays and plays the adjusted guidance data and questions to the user.

[1556] Specific examples

[1557] For example, if a user visits a historical spot in Nara, the smartphone will automatically display an audio guide and text about the Nara period. Furthermore, if the user selects the quiz mode, they will be asked, "What is a representative building from the Nara period?" If they answer correctly, directions to the next spot will be displayed.

[1558] By using the emotion engine, the difficulty of the questions can be adjusted even if the user is tired, for example. Also, if a user wants to relive a past trip to Kyoto, they can upload past photos to the app. The server analyzes the photos and generates a video that virtually recreates the Kyoto trip. The user can then enjoy the video at home.

[1559] This will enable people to gain a deeper understanding of local history and culture and experience the joy of travel without having to actually visit the area. Virtual travel will also be a new way to provide the joy of travel to elderly people and those with mobility difficulties.

[1560] The processing flow will be explained below.

[1561] User registration and information management

[1562] Step 1:

[1563] The user downloads and installs a dedicated application onto their smartphone.

[1564] Step 2:

[1565] A user launches the application and enters profile information such as name, age, and historical topics of interest.

[1566] Step 3:

[1567] The device checks the entered information and temporarily stores it.

[1568] Step 4:

[1569] The device sends the profile information to the server.

[1570] Step 5:

[1571] The server analyzes the received information and stores it in a database.

[1572] Step 6:

[1573] The server sends a message to the terminal indicating that the information has been saved.

[1574] Step 7:

[1575] The terminal displays a confirmation message to the user.

[1576] Automatic guidance upon arrival at destination

[1577] Step 1:

[1578] The device periodically obtains the user's current location using GPS.

[1579] Step 2:

[1580] The terminal transmits the obtained current location data to the server.

[1581] Step 3:

[1582] The server analyzes the user's current location and verifies that it is the tourist spot that was set.

[1583] Step 4:

[1584] If the server determines that the area is a tourist destination, it generates local guide data for the area.

[1585] Step 5:

[1586] The server transmits local guide data to the terminal.

[1587] Step 6:

[1588] The terminal displays the received local guidance data to the user, and the audio guide is automatically played.

[1589] Sightseeing with a Game-like Feel

[1590] Step 1:

[1591] The user selects a puzzle game or quiz mode within the app.

[1592] Step 2:

[1593] The terminal transmits the user's selection information to the server.

[1594] Step 3:

[1595] The server generates questions about local history and culture depending on the mode selected.

[1596] Step 4:

[1597] The server generates questions and sends them to the device.

[1598] Step 5:

[1599] The terminal presents the problem to the user.

[1600] Step 6:

[1601] The user answers the question.

[1602] Step 7:

[1603] The terminal sends the user's answer to the server.

[1604] Step 8:

[1605] The server evaluates the user's answer and determines whether it is correct.

[1606] Step 9:

[1607] The server generates information about the next spot and sends it to the terminal.

[1608] Step 10:

[1609] The device displays information about the next spot and provides directions to the location to the user.

[1610] Optimizing user experience with an emotion engine

[1611] Step 1:

[1612] The device analyzes the user's facial expressions and voice using an emotion engine to recognize emotions.

[1613] Step 2:

[1614] The device transmits the recognized emotion data to the server.

[1615] Step 3:

[1616] The server analyzes the emotional data and adjusts the local guide data and quiz questions.

[1617] Step 4:

[1618] The server transmits the adjusted data to the terminal.

[1619] Step 5:

[1620] The terminal displays and plays back the adjusted local guidance data and problems to the user.

[1621] Relive your travel memories

[1622] Step 1:

[1623] Users can enter memories of past trips within the app, or upload photos and notes.

[1624] Step 2:

[1625] The device sends the uploaded information to the server.

[1626] Step 3:

[1627] The server parses the data it receives.

[1628] Step 4:

[1629] The server generates content that recreates the virtual journey.

[1630] Step 5:

[1631] The server transmits the generated content to the terminal.

[1632] Step 6:

[1633] The terminal displays and plays the virtual travel content to the user.

[1634] Virtual travel at home

[1635] Step 1:

[1636] The user selects the "Virtual Travel" mode within the app.

[1637] Step 2:

[1638] The terminal transmits the selection information to the server.

[1639] Step 3:

[1640] The server generates a virtual sightseeing tour based on information about the desired travel destination.

[1641] Step 4:

[1642] The server transmits the generated virtual tour data to the terminal.

[1643] Step 5:

[1644] The terminal streams the virtual tour and provides it to the user.

[1645] Example 2

[1646] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1647] Conventional travel guide systems have the drawback of making it difficult for elderly people and those with mobility issues to use, as they require users to actually visit the destination to enjoy its attractions. Furthermore, the user experience is limited due to the lack of guidance that takes into account the user's emotions while traveling and the lack of functionality to recreate memories of past trips.

[1648] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1649] In this invention, the server includes means for inputting profile information through an application installed by the user and transmitting the information to the server, means for the server to confirm the user's current location based on GPS data, means for the server to generate local guidance data based on the current location confirmed by the server and transmit the data to the terminal, means for the terminal to display and play the received local guidance data to the user, means for the terminal to analyze the user's emotions and transmit the emotion data to the server, means for the server to adjust the local guidance data and the difficulty of the questions based on the emotion data and transmit the adjustment data to the terminal, means for the user to input past travel memories or upload photos and notes through the application and transmit the data to the server, means for the server to analyze the transmitted data, generate virtual travel content and transmit it to the terminal, and means for the terminal to display and play the virtual travel content to the user. This makes it possible to provide optimal guidance according to the user's emotions and to enhance the user experience through virtual travel.

[1650] "Profile Information" is information entered by a user that identifies them, such as their name, age, and historical topics of interest.

[1651] A "server" is a central computer device on a network that receives, analyzes, and stores data sent by users.

[1652] "GPS Data" means a user's current latitude and longitude information obtained from the Global Positioning System.

[1653] "Local guide data" is information and guide data about tourist spots that is generated based on the user's current location and interests.

[1654] "Terminal" refers to a mobile device such as a smartphone or tablet operated by a user.

[1655] "Emotion data" is information about the emotional state obtained by analyzing the user's facial expressions and voice.

[1656] "Virtual travel content" is digital content that provides a virtual travel experience, generated based on the user's travel records and uploaded photos and notes.

[1657] The "virtual travel mode" is a system mode that can be selected by the user to have a virtual travel experience.

[1658] "Difficulty of questions" refers to the level of difficulty of the questions in the generated quiz or puzzle game.

[1659] The present invention is a system for improving travel experiences through user-installed applications, providing appropriate guidance, quizzes, and virtual tours based on the user's emotions and interests. The system operates by combining the user's smartphone, a central server, and an emotion analysis engine.

[1660] User registration and information management

[1661] Users first install the application on their smartphone and enter their profile information, which includes their name, age, and historical topics of interest. The device temporarily stores this information and sends it to a server using the HTTPS protocol. The server analyzes the received data and stores it in a database, such as MySQL or PostgreSQL. The server then sends a message to the device indicating that the information has been saved, and the device notifies the user.

[1662] Automatic guidance upon arrival at destination

[1663] The device periodically obtains the user's current location using a GPS module and sends it to the server. The server analyzes the user's current location, and if that location is a tourist spot, it generates tourist spot information and sends it to the device. The device then displays the received tourist spot information to the user and plays an audio guide. This allows the user to immediately obtain information about the location when they arrive at a tourist spot.

[1664] Sightseeing with a Game-like Feel

[1665] Within the app, users can select a puzzle game or quiz mode. The device sends the user's selection information to the server, which then generates questions related to the local history and culture according to the selected mode. The questions are automatically generated using a generative AI model. The server sends the generated questions to the device, which then presents them to the user. When the user answers, the answer is sent to the server, which evaluates it and sends information and directions to the next spot to the device.

[1666] Relive your travel memories

[1667] Users can enter memories of past trips and upload photos and notes within the app. The device sends this data to the server, which analyzes the data and generates virtual travel content. The generated content is sent to the device, where users can view and play it to virtually relive past trips.

[1668] Virtual travel at home

[1669] Users select the "Virtual Travel" mode within the app and input their desired travel destination. The device sends the selected information to the server, which then generates a virtual sightseeing tour based on that information. The virtual sightseeing tour is created using a generative AI model. The generated tour data is sent to the device, allowing users to enjoy the virtual tour at home via streaming playback.

[1670] Optimizing user experience with an emotion engine

[1671] The device captures the user's facial expressions and voice in real time and uses an emotion analysis engine to obtain emotional data. The emotional data is sent to a server, which then adjusts the on-site guidance data and the difficulty of the questions based on that data. The adjusted data is then sent back to the device, allowing the user to receive customized guidance and quizzes tailored to their own state.

[1672] Specific examples

[1673] For example, if a user selects "Visit historical spots in Nara," the smartphone's GPS acquires the user's current location and sends it to the server. The server generates guide information about the "Nara period" and sends it to the device. The device then begins providing audio and text guidance. If the user selects quiz mode, a sample question such as "What is a representative building from the Nara period?" is displayed. If the user answers "Todaiji Temple," the server evaluates the answer and displays guidance to the next spot.

[1674] An example of a prompt sentence is "Please generate a prompt sentence for a system in which a user uses the emotion engine at tourist spots in Nara and is shown a quiz about the Nara period."

[1675] This will allow people to gain a deeper understanding of local history and culture and experience the joys of travel without having to actually visit the area. It will also make it possible to offer the joys of travel in a new way to elderly people and those with mobility difficulties.

[1676] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1677] Step 1:

[1678] The user installs the application on their smartphone.

[1679] The user downloads and installs the application from the smartphone's app store, then launches the application.

[1680] Input: App store download link

[1681] Output: Installed applications

[1682] Step 2:

[1683] The user launches the application and enters profile information.

[1684] Users launch the application and fill in fields with information such as their name, age, and historical topics of interest.

[1685] Input: Profile information entered by the user (name, age, topics of interest)

[1686] Output: Temporarily saved profile information

[1687] Step 3:

[1688] The device checks the entered information and temporarily stores it.

[1689] The terminal checks whether the input is in the correct format (for example, whether the name is not empty or the age is in numeric format), and then stores the information temporarily in memory.

[1690] Input: Profile information entered by the user

[1691] Output: Format-checked and temporarily saved profile information

[1692] Step 4:

[1693] The device sends the profile information to the server.

[1694] The device sends the checked profile information to the server using the HTTPS protocol.

[1695] Input: Format-checked and temporarily saved profile information

[1696] Output: Profile information sent to the server

[1697] Step 5:

[1698] The server analyzes the received information and stores it in a database.

[1699] The server parses the received profile information in JSON format and stores it in a database, for example, adding the user ID and timestamp.

[1700] Input: Profile information sent to the server

[1701] Output: Profile information stored in the database

[1702] Step 6:

[1703] The server sends a message to the terminal indicating that the information has been saved.

[1704] The server confirms that the data has been saved to the database, generates a message indicating that the data has been saved, and sends it to the terminal.

[1705] Input: Profile information stored in the database

[1706] Output: Message indicating information has been saved

[1707] Step 7:

[1708] The terminal displays a confirmation message to the user.

[1709] The terminal displays a message indicating that the information has been saved as received from the server as a pop-up notification or on the screen.

[1710] Input: Message that information has been saved

[1711] Output: User notification

[1712] Step 8:

[1713] The device periodically obtains the user's current location using GPS.

[1714] The device uses the smartphone's GPS module to obtain the user's current location at regular intervals (e.g., every 5 minutes).

[1715] Input: None (periodic acquisition of GPS module)

[1716] Output: User's current location data (latitude, longitude)

[1717] Step 9:

[1718] The terminal transmits the obtained current location data to the server.

[1719] The device converts the acquired location data into JSON format and sends it to the server using the HTTPS protocol.

[1720] Input: User's current location data (latitude, longitude)

[1721] Output: Location data sent to the server

[1722] Step 10:

[1723] The server analyzes the user's current location and confirms that it is a tourist spot.

[1724] The server compares the received current location data with a list of pre-registered tourist spots and analyzes whether the user's location is within a tourist spot.

[1725] Input: Location data sent to the server

[1726] Output: Tourist destination determination result

[1727] Step 11:

[1728] The server generates tourist destination information.

[1729] If the server confirms that the destination is a tourist spot, it generates information about the historical background and attractions of the destination.

[1730] Input: Tourist destination determination result

[1731] Output: Generated tourist destination information

[1732] Step 12:

[1733] The server transmits local guide data to the terminal.

[1734] The server sends the generated tourist destination information to the terminal in JSON format.

[1735] Input: Generated tourist destination information

[1736] Output: Local guide data sent to the terminal

[1737] Step 13:

[1738] The terminal displays the received local guidance data to the user and plays back the audio guide.

[1739] The device displays the received guidance data on the display, plays the audio guide file, and may notify the user using push notifications or alerts.

[1740] Input: Local guide data sent to the terminal

[1741] Output: Display to user, play audio guide

[1742] Step 14:

[1743] The user selects a puzzle game or quiz mode within the app.

[1744] Users select the "Play" tab from the in-app menu and choose their preferred mode.

[1745] Input: Mode selected by user

[1746] Output: Selected mode information

[1747] Step 15:

[1748] The terminal transmits the user's selection information to the server.

[1749] The device converts the selected mode information into JSON format and sends it to the server.

[1750] Input: Selected mode information

[1751] Output: Mode information sent to the server

[1752] Step 16:

[1753] The server generates questions related to local history and culture.

[1754] Based on the received mode information, the server uses a generative AI model to generate quizzes and questions about local history and culture.

[1755] Input: Mode information sent to the server

[1756] Output: Generated quizzes and questions

[1757] Step 17:

[1758] The server generates questions and sends them to the device.

[1759] The server sends the generated questions to the terminal in JSON format.

[1760] Input: Generated quizzes and questions

[1761] Output: Problems sent to terminal

[1762] Step 18:

[1763] The terminal presents the problem to the user.

[1764] The terminal displays the received questions on the screen so that the user can answer them.

[1765] Input: Question sent to terminal

[1766] Output: The problem presented to the user

[1767] Step 19:

[1768] The user answers the question.

[1769] Users can answer the questions by tapping on the appropriate answer option or by writing their own answer.

[1770] Input: User response

[1771] Output: User response data

[1772] Step 20:

[1773] The terminal sends the user's answer to the server.

[1774] The device converts the user's response into JSON format and sends it to the server.

[1775] Input: User response data

[1776] Output: Response data sent to the server

[1777] Step 21:

[1778] The server evaluates the user's answer and determines whether it is correct.

[1779] Based on the received answers, the server runs an algorithm to evaluate whether the answers are correct or incorrect and generates a judgment result.

[1780] Input: Response data sent to the server

[1781] Output: Judgment result (correct or incorrect)

[1782] Step 22:

[1783] The server generates information about the next spot and sends it to the terminal.

[1784] Based on the results of the determination, the server generates information about the next tourist spot and guidance data to that location, and sends it to the terminal.

[1785] Input: Judgment result

[1786] Output: Next spot information and guidance data

[1787] Step 23:

[1788] The terminal displays information and directions to the next spot to the user.

[1789] The device displays the received next spot information and provides the user with information such as maps and the shortest route.

[1790] Input: Next spot information and guidance data

[1791] Output: Next stop information displayed to the user

[1792] (Application example 2)

[1793] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1794] Conventional travel support systems have the problem that users cannot experience sightseeing unless they go to the destination, making it difficult for elderly people and those with mobility difficulties to enjoy the pleasure of traveling. In addition, the user experience is not personalized enough, and tourist information is provided uniformly. This makes it difficult to sufficiently increase user satisfaction.

[1795] Furthermore, virtual stores do not provide services that take into account the user's emotional state, making it difficult to provide a personalized shopping experience.

[1796] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1797] In this invention, the server includes means for inputting profile information through an application installed by the user and transmitting the information to the server, means for confirming the user's current location based on GPS data, means for generating local guidance data based on the confirmed current location and transmitting the data to the terminal, means for the terminal to display and play back the received local guidance data to the user, means for exploring a virtual store selected by the user, means for analyzing the user's emotions using an emotion engine and suggesting products and services based on the analysis results, and means for the terminal to display the suggested products and services to the user. This allows even elderly people and those with mobility difficulties to enjoy virtual travel, and enables personalized tourist information and shopping experiences tailored to the user's emotional state.

[1798] "Profile Information" means personal data such as a User's name, age, and interests.

[1799] "Server" refers to a device that receives, stores, and analyzes data from users, and generates and transmits information as instructed.

[1800] "GPS data" refers to data that includes location information for identifying a user's current location.

[1801] "Local guide data" means data including tourist information and guide information about the place the user is visiting.

[1802] A "terminal" is a device that is directly operated by a user, and includes, for example, a smartphone or a tablet.

[1803] A "virtual store" refers to a store that does not have a physical presence, but operates in a virtual space.

[1804] An "emotion engine" refers to software or hardware that analyzes a user's facial expressions, voice, etc. and recognizes their emotional state.

[1805] "Product and service suggestions" refers to the act of recommending the most suitable products and services to users based on the analysis results of the emotion engine.

[1806] "Quiz and mini-games" refers to entertainment elements in which users can participate and receive rewards or discounts based on their scores or results.

[1807] "Benefits and discounts" refers to benefits and discounts offered to users depending on the results of quizzes and mini-games.

[1808] "Virtual travel content" means content that recreates past travel data or uploaded information and allows users to virtually experience travel.

[1809] "Facial and voice analysis" refers to the process of analyzing facial and voice data to recognize the user's emotional state.

[1810] The present invention is a system implemented in the following specific steps: First, a user installs an application on a smartphone or head-mounted display and enters profile information. The application acquires information such as the user's name, age, and interests, and sends the information to a server. The server then stores the profile information in a database and manages the user's basic information.

[1811] Next, the user launches the application and can explore the virtual store. The application continuously acquires the user's current location using the GPS function and sends the data to the server. The server analyzes the acquired current location data and identifies the location within the virtual store selected by the user. Based on the user's location, the server then generates local guide data and product information for the virtual store and sends it to the device. The device then displays the received data to the user, providing intuitive navigation.

[1812] Furthermore, an emotion engine is used to analyze the user's emotional state. Specifically, the device acquires the user's facial expressions and voice data and sends that data to the server. The server then uses the emotion engine to analyze the user's emotions. Based on the analysis results, the server suggests the most suitable products and services for the user and sends that information to the device. This allows the device to display the suggested products and services to the user. For example, if the user has a happy expression, the server can provide special campaign and discount information tailored to that emotion.

[1813] Users can participate in quizzes and mini-games within the virtual store, which adds an entertainment element and improves the user experience. Based on the results of the quizzes and mini-games, rewards and discounts are offered, allowing users to enjoy shopping and earn rewards.

[1814] On the other hand, users can input memories of past trips or upload photos and notes. The server then analyzes the uploaded data and generates content that recreates a virtual trip. The content is sent to the device, allowing users to virtually re-experience past trips. This function is particularly useful for elderly people and those with mobility difficulties, who can enjoy virtual travel.

[1815] Specific examples

[1816] For example, consider a scenario where a 35-year-old user installs an application and explores a virtual store. The user's profile information includes their name, age, and categories of interest (e.g., electronics, books, and fashion). Using an emotion engine, the system suggests products that the user is interested in. For example, new smartphones, best-selling books, and trending fashion items may be suggested. The user can also participate in a quiz and earn discount coupons based on their score.

[1817] Example prompt for a generative AI model:

[1818] "A 35-year-old user is interested in electronics, books, and fashion. He enters a virtual store using his smartphone. Based on sentiment analysis, recommend the following products to him:

[1819] 1. The latest smartphone

[1820] 2. Bestselling books

[1821] 3. Trendy fashion items

[1822] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1823] Step 1:

[1824] A user installs an application on a smartphone or head-mounted display, which involves a download and installation process from an application store. The input is the application installation file, and the output is the installed application.

[1825] Step 2:

[1826] The user launches the application and enters profile information. Information such as name, age, and interest categories is received as input from the user, and the device temporarily stores that information. The input is the profile information entered directly by the user, and the output is the temporarily stored profile information.

[1827] Step 3:

[1828] The device sends profile information to the server. The server receives this information and stores it in a database. The input is the profile information sent from the device, and the output is the profile information stored in the database.

[1829] Step 4:

[1830] The user selects a virtual store and begins searching. The device uses its GPS function to obtain current location data and sends it to the server. The input is the device's GPS data, and the output is the current location data sent to the server.

[1831] Step 5:

[1832] The server analyzes the acquired current location data and identifies the location within the virtual store selected by the user. The server generates local guidance data and product information based on that location and sends it to the terminal. The input is the current location data, and the output is the generated local guidance data and product information.

[1833] Step 6:

[1834] The terminal displays the received local guide data and product information to the user. The display can be in the form of text, images, audio, etc. The input is the local guide data and product information sent from the server, and the output is the information displayed to the user.

[1835] Step 7:

[1836] The device acquires the user's facial expression and voice data and sends it to the server. The server uses an emotion engine to analyze the user's emotional state. The input is facial expression and voice data, and the output is the emotion analysis results.

[1837] Step 8:

[1838] The server generates data to suggest optimal products and services to the user based on the results of the emotion analysis and sends it to the terminal. The input is the emotion analysis results, and the output is the generated product and service suggestion data.

[1839] Step 9:

[1840] The terminal displays the suggested product and service data to the user, and the user can proceed with shopping based on that information. The input is the suggested data sent from the server, and the output is the product and service information displayed to the user.

[1841] Step 10:

[1842] Users participate in quizzes and mini-games in a virtual store. The terminal sends the game results to the server, which then generates rewards and discount data based on the results. The input is the user's game results, and the output is the rewards and discount data.

[1843] Step 11:

[1844] The server sends the generated reward and discount data to the terminal, which then displays it to the user. The input is the reward and discount data, and the output is the reward information displayed to the user.

[1845] Step 12:

[1846] Users input memories of past travels or upload photos and notes. The device sends the data to the server, which then generates content that recreates the virtual trip. The input is the data uploaded by the user, and the output is the virtual trip content.

[1847] Step 13:

[1848] The server sends the generated virtual travel content to the terminal, which then displays and plays it for the user. The input is the virtual travel content, and the output is the virtual travel content that is displayed and played for the user.

[1849] Examples:

[1850] For example, if a user is looking to buy a fashion item in a virtual store, sentiment analysis will suggest new items that the user is interested in. Users can also participate in quizzes and earn discount coupons depending on their scores.

[1851] Example prompt for a generative AI model:

[1852] "A 35-year-old user is interested in electronics, books, and fashion. He enters a virtual store using his smartphone. Based on sentiment analysis, recommend the following products to him:

[1853] 1. The latest smartphone

[1854] 2. Bestselling books

[1855] 3. Trendy fashion items

[1856] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1857] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1858] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1859] [Fourth embodiment]

[1860] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1861] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1862] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1863] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1864] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1865] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1866] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1867] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1868] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1869] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1870] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1871] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1872] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1873] Overall flow

[1874] This invention is a system that makes it easier for travelers to enjoy the attractions of rural areas outside of cities, enriching the local tourist experience while also providing a similar experience in remote areas. Furthermore, it also provides the enjoyment of travel to elderly people who find it difficult to travel to the local area. The system supports the tourist experience by linking the user's smartphone with a server.

[1875] User registration and information management

[1876] 1. The user installs a dedicated application on their smartphone.

[1877] 2. The user launches the application and enters profile information such as name, age, and historical topics of interest.

[1878] 3. The device sends the entered information to the server.

[1879] 4. The server stores the received information in a database and sends a confirmation message to the user.

[1880] Automatic guidance upon arrival at destination

[1881] 1. The device periodically obtains the user's current location using GPS.

[1882] 2. The device sends the acquired location data to the server.

[1883] 3. The server analyzes the user's current location and verifies that it is one of the tourist destinations set.

[1884] 4. The server generates local guidance data and sends it to the terminal.

[1885] 5. The terminal displays and plays the received guidance data (audio guide, text, images) to the user.

[1886] Sightseeing with a Game-like Feel

[1887] 1. The user selects a puzzle game or quiz mode within the app.

[1888] 2. The server generates questions about local history and culture according to the selected mode and sends them to the terminal.

[1889] 3. The device presents the user with a question, collects the user's answers, and sends them to the server.

[1890] 4. The server evaluates the user's answer, generates information about the next spot, and sends it to the device.

[1891] 5. The device displays information about the next spot and provides directions to the location to the user.

[1892] Relive your travel memories

[1893] 1. Users enter memories of past trips or upload photos and notes within the app.

[1894] 2. The server parses the uploaded data.

[1895] 3. The server generates content (in the form of videos or slideshows) that recreates the virtual journey.

[1896] 4. The terminal displays and plays the generated virtual travel content to the user.

[1897] Virtual travel at home

[1898] 1. The user selects the "Virtual Travel" mode within the app.

[1899] 2. The server collects information on the desired travel destinations based on the user's selection and generates a virtual sightseeing tour.

[1900] 3. The device streams the generated virtual tour and provides it to the user.

[1901] Specific examples

[1902] For example, consider a case where a user visits a historical spot in Nara. When the user arrives at the location, the smartphone automatically displays an audio guide and text about the Nara period. Furthermore, if the user selects the quiz mode, the smartphone asks the question, "What is a representative building from the Nara period?" If the answer is correct, directions to the next spot will be displayed.

[1903] If a user wants to relive a past trip to Kyoto, they can upload past photos to the app. The server analyzes the photos and generates a video that virtually recreates the trip to Kyoto. The user can then enjoy the video at home.

[1904] This will enable people to gain a deeper understanding of local history and culture and experience the joy of travel without having to go to the actual place. Virtual travel will also be a new way to provide the joy of travel to elderly people and those with mobility difficulties.

[1905] The processing flow will be explained below.

[1906] User registration and information management

[1907] Step 1:

[1908] The user downloads and installs the application on their smartphone.

[1909] Step 2:

[1910] A user launches the application and enters profile information such as name, age, and historical topics of interest.

[1911] Step 3:

[1912] The device checks the entered information and temporarily stores it.

[1913] Step 4:

[1914] The device sends the profile information to the server.

[1915] Step 5:

[1916] The server analyzes the received information and stores it in a database.

[1917] Step 6:

[1918] The server sends a message to the terminal indicating that the information has been saved.

[1919] Step 7:

[1920] The terminal displays a confirmation message to the user.

[1921] Automatic guidance upon arrival at destination

[1922] Step 1:

[1923] The device periodically obtains the user's current location using GPS.

[1924] Step 2:

[1925] The terminal transmits the obtained current location data to the server.

[1926] Step 3:

[1927] The server analyzes the user's current location and checks whether it is the tourist spot you have set.

[1928] Step 4:

[1929] If the server determines that the area is a tourist destination, it generates local guide data for the area.

[1930] Step 5:

[1931] The server transmits local guide data to the terminal.

[1932] Step 6:

[1933] The terminal displays the received local guidance data to the user, and the audio guide is automatically played.

[1934] Sightseeing with a Game-like Feel

[1935] Step 1:

[1936] The user selects a puzzle game or quiz mode within the app.

[1937] Step 2:

[1938] The terminal transmits the user's selection information to the server.

[1939] Step 3:

[1940] The server generates questions about local history and culture depending on the mode selected.

[1941] Step 4:

[1942] The server generates questions and sends them to the device.

[1943] Step 5:

[1944] The terminal presents the problem to the user.

[1945] Step 6:

[1946] The user answers the question.

[1947] Step 7:

[1948] The terminal sends the user's answer to the server.

[1949] Step 8:

[1950] The server evaluates the user's answer and determines whether it is correct.

[1951] Step 9:

[1952] The server generates information about the next spot and sends it to the terminal.

[1953] Step 10:

[1954] The device displays information about the next spot and provides directions to the location to the user.

[1955] Relive your travel memories

[1956] Step 1:

[1957] Users can enter memories of past trips within the app, or upload photos and notes.

[1958] Step 2:

[1959] The device sends the uploaded information to the server.

[1960] Step 3:

[1961] The server parses the data it receives.

[1962] Step 4:

[1963] The server generates content that recreates the virtual journey.

[1964] Step 5:

[1965] The server transmits the generated content to the terminal.

[1966] Step 6:

[1967] The terminal displays and plays the virtual travel content to the user.

[1968] Virtual travel at home

[1969] Step 1:

[1970] The user selects the "Virtual Travel" mode within the app.

[1971] Step 2:

[1972] The terminal transmits the selection information to the server.

[1973] Step 3:

[1974] The server generates a virtual sightseeing tour based on information about the desired travel destination.

[1975] Step 4:

[1976] The server transmits the generated virtual tour data to the terminal.

[1977] Step 5:

[1978] The terminal streams the virtual tour and provides it to the user.

[1979] Example 1

[1980] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1981] Conventional tourism systems often lack tourist information when visiting rural areas outside of cities, making it difficult to fully enjoy the attractions of those areas. There is also the problem that tourism itself is difficult for elderly people and people with mobility restrictions who have difficulty traveling to those areas. Furthermore, there are limited ways to recreate and re-enjoy memories of past trips. The goal of this project is to solve these problems and make travel and tourism attractive and easy for everyone to enjoy.

[1982] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1983] In this invention, the server includes a means for users to send profile information, a means for checking current location based on GPS data, and a means for generating local guide data based on the current location. This allows users to easily enjoy the attractions of local areas. The server also includes a means for inputting memories of past travels or uploading photos and notes, a means for analyzing the uploaded data and generating content that recreates a virtual travel, and a means for displaying and playing the generated virtual travel content to the user, allowing users to relive past travels. Furthermore, the server includes a means for selecting a "virtual travel" mode, a means for collecting information on desired travel destinations and generating a virtual sightseeing tour, and a means for streaming and providing the generated virtual tour to the user, allowing users who have difficulty visiting the destinations to enjoy the fun of travel.

[1984] "User" refers to a human user who interfaces with the server and terminal.

[1985] "Device" refers to a smartphone, tablet, or other mobile information device on which a user installs and uses an application.

[1986] "Application" refers to a software program that a user installs on a terminal and uses.

[1987] "Profile Information" refers to personal information about you, such as your name, age, and topics of interest.

[1988] "Server" means a remote computer system that processes and stores data received from a client device and executes server-side actions.

[1989] "GPS Data" refers to your location information obtained using the Global Positioning System.

[1990] "Current Location" refers to a user's real-time geographic location as determined based on GPS data.

[1991] "Local guide data" refers to data containing information about a specific tourist destination (audio guide, text, images, etc.).

[1992] "Virtual travel content" refers to content (such as videos or slideshows) that is displayed and played back to users and is generated based on past travel memories and uploaded data.

[1993] "Virtual travel" refers to a sightseeing tour that allows you to collect information about your desired travel destination and experience it virtually without physically going there.

[1994] "Questions" refer to quizzes and puzzles about history and culture that are generated by the server and presented to the user.

[1995] A "mode" refers to a particular type of operation or function that a user selects within an application.

[1996] This invention is a system for enhancing the appeal of visiting regional tourist destinations outside of urban areas. This system links the user's smartphone with a server, aiming to enrich the tourist experience on-site while also providing a similar experience in remote locations. Furthermore, it is possible to provide the enjoyment of travel to elderly users and users with mobility difficulties.

[1997] 1. User registration and information management

[1998] Users first install a dedicated application on their smartphone. They then launch the application and enter their profile information, such as their name, age, and historical topics of interest. The information is then sent from the device to a server, which stores it in a database. This is how the user profile is managed.

[1999] 2. Automatic guidance when arriving at your destination

[2000] The user's device periodically obtains its current location using GPS and sends this data to the server. The server analyzes the current location data and confirms that the user has arrived at the tourist destination. The server generates local guide data about the tourist destination and sends it to the device. The device receives this data and displays and plays audio guides, text, images, etc. to the user. Technologies used include Google Maps API and Azure Cognitive Services.

[2001] 3. Sightseeing like a game

[2002] Within the app, users can select a puzzle game or quiz mode. Depending on the selected mode, the server generates questions about local history and culture and sends them to the device. The device presents the questions to the user and collects their answers. The server evaluates the answers, generates information about the next spot, and sends it to the device. The device then displays information about the next spot and directions to the location to the user. A specific example is the question, "What is a representative building from the Nara period?", and the user inputs the answer.

[2003] 4. Relive your travel memories

[2004] Users can input memories of past travels and upload photos and notes. The server analyzes the uploaded data and generates content that recreates the virtual trip. The device displays and plays the generated virtual trip content to the user. Technologies used include Amazon Rekognition and Adobe Premiere API. An example prompt is, "Users upload photos of travel destinations they have visited in the past, and a video recreating that trip is generated and displayed."

[2005] 5. Virtual travel at home

[2006] Users can select the "Virtual Travel" mode within the app. The server collects information on the desired travel destination based on the user's selection and generates a virtual sightseeing tour. The device streams the generated virtual tour and provides it to the user. The technology used includes the Google Earth API. An example of a specific prompt is, "The user selects the 'Virtual Travel' mode and a virtual tour of Paris begins."

[2007] This makes it possible for the present invention to increase the enjoyment of traveling to rural areas, providing a similar experience from remote locations, and bringing the joy of travel to elderly people and those with mobility difficulties.

[2008] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2009] Step 1:

[2010] The user installs the dedicated application on their smartphone. The user downloads the application from the app store and completes the installation procedure. The application can then be used on the smartphone.

[2011] Step 2:

[2012] A user launches the application and enters profile information (name, age, historical topics of interest). The information entered by the user is temporarily stored on the device. Profile information is constructed based on the input (e.g., name "Taro Tanaka," age "65," interest "ancient history").

[2013] Step 3:

[2014] The device sends the profile information to the server. The input data from the device is transferred to the server via the Internet and received by the server. The received profile information is saved in the server's database. The server generates a save completion message and sends it to the device. The device displays a "Registration Complete" message to the user.

[2015] Step 4:

[2016] The device periodically obtains the user's current location using GPS. The device uses the GPS sensor to collect location information and generate that data. The generated GPS data is stored on the device.

[2017] Step 5:

[2018] The device transmits the acquired current location data to the server. The location information data from the device is transferred to the server and received by the server. The received data is sent to the server's analysis module.

[2019] Step 6:

[2020] The server analyzes the current location data and confirms that the user has arrived at the designated tourist spot. The server compares the location data with tourist spot information in the database to determine whether the user's current location matches the tourist spot. If the location information matches the tourist spot, the server generates local guidance data.

[2021] Step 7:

[2022] The server generates local guidance data (audio guide, text, images) and sends it to the terminal. The generated guidance data is transferred from the server to the terminal. The terminal displays and plays the received local guidance data to the user. The user plays the audio guide and receives the information.

[2023] Step 8:

[2024] The user selects a puzzle game or quiz mode within the app. The user operates the mode selection screen and selects the desired game mode. The selected mode information is saved on the device.

[2025] Step 9:

[2026] The server generates questions about local history and culture according to the selected mode and sends them to the device. The server generates question data based on the game mode and transfers the data to the device. The device presents the questions to the user and collects the user's answers.

[2027] Step 10:

[2028] The terminal sends the user's answer data to the server. The answer data from the terminal is transferred to the server and received by the server. The received data is sent to the server's analysis module.

[2029] Step 11:

[2030] The server evaluates the user's answers, generates information about the next spot, and sends it to the device. The server analyzes the answer data and generates information about the next tourist spot based on the evaluation results. The generated spot information is then transferred to the device.

[2031] Step 12:

[2032] The device displays information about the next spot and guidance to the user. The device receives the spot information and displays it to the user. The user confirms the guidance to the next spot and begins moving.

[2033] Step 13:

[2034] The user inputs memories of past trips or uploads photos and notes. The user inputs past trip information within the app or selects and uploads photo files. The input and uploaded data is saved on the device.

[2035] Step 14:

[2036] The device sends the uploaded data to the server. The uploaded data from the device is transferred to the server and received by the server. The received data is sent to the server's analysis module.

[2037] Step 15:

[2038] The server analyzes the uploaded data and generates virtual travel content. The server processes the data using image analysis technology and generates content for recreating the virtual travel. The generated content is stored on the server.

[2039] Step 16:

[2040] The server sends the virtual travel content to the terminal. The generated content data is transferred from the server to the terminal. The terminal receives the virtual travel content and displays and plays it for the user. The user experiences the virtual travel.

[2041] Step 17:

[2042] The user selects the "Virtual Travel" mode within the app. The user operates the mode selection screen and selects the Virtual Travel mode. The selection information is saved on the device.

[2043] Step 18:

[2044] The server collects information on the desired travel destination based on the user's selection and generates a virtual sightseeing tour. The server extracts the travel destination information from the database and creates a virtual tour based on the collected data. The generated tour information is saved on the server.

[2045] Step 19:

[2046] The server sends the generated virtual tour to the terminal. The tour information data is transferred from the server to the terminal. The terminal receives the virtual tour and starts streaming playback. The user experiences the virtual trip.

[2047] (Application example 1)

[2048] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2049] There is a need to provide tourism experiences for elderly people and travel enthusiasts living in remote areas who find it difficult to visit local sites. However, current systems rely on on-site visits and lack ways to enhance tourism experiences in virtual environments. This invention aims to solve this problem by providing a virtual travel system that allows users to enjoy the charms of local areas from the comfort of their own homes.

[2050] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2051] In this invention, the server includes means for inputting profile information through an application installed by the user and transmitting the information to the server, means for confirming the user's current location based on GPS data, means for generating local guidance data based on the confirmed current location and transmitting it to the terminal, means for the user to select a mode within the application for enjoying a virtual trip at home, means for generating virtual sightseeing tour content based on the selected travel destination and transmitting it to the terminal, and means for displaying and playing the received virtual sightseeing tour content to the user, thereby enabling the user to enjoy an advanced virtual sightseeing experience from the comfort of their own home.

[2052] "Profile Information" refers to personal information such as a User's name, age, and topics of interest that is used by the Application to provide the User with an optimal tourism experience.

[2053] "GPS Data" means location information obtained using the Global Positioning System and is used to determine a user's current location.

[2054] "Local guide data" refers to data that includes information such as audio guides, text, and images about tourist spots that users visit, and is provided to support the user's sightseeing experience.

[2055] "Device" refers to an electronic device used by a user, such as a smartphone, tablet, or head-mounted display, which is used to display and play data sent from the server.

[2056] "Virtual travel" refers to the process by which users virtually visit tourist spots through an application and experience their culture and history without actually traveling.

[2057] "Virtual sightseeing tour content" refers to content such as 3D visuals of tourist spots, audio guides, and text that is generated by the server and sent to the user's terminal, and is the main information that allows the user to experience a virtual trip.

[2058] "Quiz mode" refers to a function that, when selected by the user, causes the application to ask questions about tourist spots, and allows the user to enjoy sightseeing in a game-like manner by answering the questions.

[2059] A "head-mounted display" is a display device worn by the user on the head to display virtual reality (VR) and augmented reality (AR) content, and is used to provide an advanced virtual experience.

[2060] The present invention provides a virtual travel system that allows users to experience the attractions of local areas from the comfort of their own homes. Detailed embodiments are described below.

[2061] System Configuration

[2062] This system consists of an application installed by the user, a terminal (such as a smartphone or a head-mounted display), and a server.

[2063] Registering profile information and acquiring GPS data

[2064] Through the application, users enter profile information such as their name, age, and topics of interest, and submit that information to the server, which stores the user's profile information in a database.

[2065] The device periodically acquires the user's current location using GPS and sends that data to the server, which then analyzes the GPS data to confirm the user's current location.

[2066] Generation and display of on-site guidance data

[2067] The server generates local guidance data based on the user's current location and sends it to the device. The device then displays and plays the received local guidance data to the user as audio guides, text, and images, allowing the user to obtain a wealth of information about the local history and culture.

[2068] Providing virtual travel modes

[2069] Within the application, users can select a mode to enjoy a virtual trip at home. The server generates virtual sightseeing tour content based on the selected travel destination and sends it to the device. The device then displays and plays the received virtual sightseeing tour content to the user as an audio guide or 3D visuals. Specifically, users can experience the virtual trip through a smartphone or head-mounted display.

[2070] Quiz mode available

[2071] The application has a quiz mode. Based on the quiz mode selected by the user, the server generates questions about local history and culture and sends them to the device. The device presents the questions to the user, collects the user's answers, and sends them to the server. The server evaluates the user's answers, generates information about the next spot, and sends it to the device. This allows the user to enjoy sightseeing in a game-like manner.

[2072] Recreating a virtual journey

[2073] Users can input memories of past trips or upload photos and notes. The server analyzes the uploaded data and generates content that recreates a virtual trip. The device displays and plays the generated virtual trip content to the user.

[2074] Hardware and software used

[2075] This system uses the following hardware and software:

[2076] Hardware: Smartphone, Head-Mounted Display (HMD), Server

[2077] Software: Application, server program, HTTP request library (requests), JSON parser

[2078] Examples of concrete examples and prompts

[2079] As a concrete example, consider a scenario in which a user experiences historical spots in Nara from the comfort of their own home. The user launches the app and selects the "Virtual Travel" mode. The server generates an audio guide and text about Nara culture based on the user's profile data and selected travel destination, and sends them to the device along with 3D visuals. The user puts on a head-mounted display and experiences Nara's history in a virtual environment. In addition, in quiz mode, the user is asked the question, "What is a representative building from the Nara period?" and by answering the question, directions to the next spot are displayed.

[2080] Below is an example of a prompt sentence to input to the generative AI model.

[2081] Prompt Sentence Examples

[2082] ---

[2083] We are developing an application that allows users to enjoy virtual travel from the comfort of their own home. The application runs on smartphones and head-mounted displays (HMDs) and provides users with a virtual sightseeing experience related to local culture and history.

[2084] example:

[2085] The user uses the app to experience Nara's historical spots. The app retrieves audio guides and text about Nara based on the user's profile data, and displays them in 3D visuals through the HMD. The app also has a quiz mode, and after the user answers questions, the app guides the user to the next virtual spot.

[2086] Expected output:

[2087] Audio guide about historical spots from the Nara period

[2088] Coding audio files for broadcast

[2089] A quiz about representative buildings from the Nara period

[2090] Directions to the next virtual spot

[2091] Please explain in detail how this application works.

[2092] ---

[2093] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2094] Step 1:

[2095] A user installs the application, enters profile information (such as name, age, and topics of interest), and sends that information to the server, which stores the information in a database and sends a confirmation message to the user.

[2096] Input: User profile information Output: Confirmation message

[2097] Step 2:

[2098] The device periodically acquires GPS data and sends the user's current location to the server. The server analyzes the received GPS data and confirms the user's current location. When the user arrives at the location, the server generates local guidance data and sends it to the device.

[2099] Input: GPS data Output: Local guidance data

[2100] Step 3:

[2101] The device displays and plays the received local information to the user. Specifically, it displays audio guides, text, and images, allowing the user to learn about the history and culture of the area.

[2102] Input: On-site guidance data Output: Displayed / played guidance data

[2103] Step 4:

[2104] The user selects the "Virtual Travel" mode in the application. The terminal sends this selection to the server. The server generates virtual sightseeing tour content based on the selected travel destinations and sends it to the terminal.

[2105] Input: Virtual travel mode selection information Output: Virtual sightseeing tour content

[2106] Step 5:

[2107] The device displays and plays the received virtual sightseeing tour content to the user. When using a head-mounted display, 3D visuals are displayed, allowing the user to experience traveling in a virtual environment.

[2108] Input: Virtual sightseeing tour content Output: Displayed / played tour content

[2109] Step 6:

[2110] The user selects the quiz mode. The terminal transmits this selection information to the server. The server generates questions about the local history and culture and transmits them to the terminal.

[2111] Input: Quiz mode selection information Output: Quiz question data

[2112] Step 7:

[2113] The terminal presents the received quiz question data to the user, collects the user's answers, and sends them to the server. The server evaluates the user's answers, generates guidance data for the next spot, and sends it to the terminal. The terminal displays it to the user.

[2114] Input: User response data Output: Next spot guidance data

[2115] Step 8:

[2116] Users input memories of past trips or upload photos and notes. The device sends this data to the server, which analyzes the uploaded data, generates content that recreates the virtual trip, and sends it to the device.

[2117] Input: Past travel data Output: Virtual travel reenactment content

[2118] Step 9:

[2119] The device displays and plays the received virtual travel recreation content to the user, allowing the user to enjoy memories of past trips from the comfort of their own home.

[2120] Input: Virtual travel content Output: Displayed / played travel content

[2121] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2122] This system makes it easier for travelers to enjoy the attractions of rural areas outside of cities, enriching the local tourist experience while also providing a similar experience in remote locations. It also provides the enjoyment of travel to elderly people who find it difficult to travel to the local area. The system supports the tourist experience by linking the user's smartphone with a server and an emotion engine.

[2123] User registration and information management

[2124] 1. The user installs a dedicated application on their smartphone.

[2125] 2. The user launches the application and enters profile information such as name, age, and historical topics of interest.

[2126] 3. The device checks the entered information and temporarily stores it.

[2127] 4. The device sends the profile information to the server.

[2128] 5. The server analyzes the received information and stores it in a database.

[2129] 6. The server sends a message to the device indicating that the information has been saved.

[2130] 7. The terminal displays a confirmation message to the user.

[2131] Automatic guidance upon arrival at destination

[2132] 1. The device periodically obtains the user's current location using GPS.

[2133] 2. The device sends the location data it has obtained to the server.

[2134] 3. The server analyzes the user's current location and confirms that it is the tourist spot that was set.

[2135] 4. If the server determines that the area is a tourist destination, it generates local guide data for the area.

[2136] 5. The server sends the local guide data to the terminal.

[2137] 6. The device displays the received local guidance data to the user, and the audio guide is automatically played.

[2138] Sightseeing with a Game-like Feel

[2139] 1. The user selects a puzzle game or quiz mode within the app.

[2140] 2. The terminal sends the user's selection information to the server.

[2141] 3. The server generates questions about local history and culture depending on the mode selected.

[2142] 4. The server sends the generated questions to the device.

[2143] 5. The device presents the problem to the user.

[2144] 6. The user answers the question.

[2145] 7. The device sends the user's answer to the server.

[2146] 8. The server evaluates the user's answer and determines whether it is correct.

[2147] 9. The server generates information about the next spot and sends it to the terminal.

[2148] 10. The device displays information about the next spot and directions to the location to the user.

[2149] Relive your travel memories

[2150] 1. Users enter memories of past trips or upload photos and notes within the app.

[2151] 2. The device sends the uploaded information to the server.

[2152] 3. The server analyzes the received data.

[2153] 4. The server generates content that recreates the virtual journey.

[2154] 5. The server sends the generated content to the device.

[2155] 6. The device displays and plays the virtual travel content to the user.

[2156] Virtual travel at home

[2157] 1. The user selects the "Virtual Travel" mode within the app.

[2158] 2. The device sends the selection information to the server.

[2159] 3. The server generates a virtual sightseeing tour based on information about the desired travel destination.

[2160] 4. The server sends the generated virtual tour data to the device.

[2161] 5. The device streams the virtual tour and provides it to the user.

[2162] Optimizing user experience with an emotion engine

[2163] 1. The device analyzes the user's facial expressions and voice using an emotion engine to recognize emotions.

[2164] 2. The server adjusts the local guidance data and the difficulty of the game questions based on the recognized emotion data.

[2165] 3. The server sends the data generated based on the emotion to the device.

[2166] 4. The terminal displays and plays the adjusted guidance data and questions to the user.

[2167] Specific examples

[2168] For example, if a user visits a historical spot in Nara, the smartphone will automatically display an audio guide and text about the Nara period. Furthermore, if the user selects the quiz mode, they will be asked, "What is a representative building from the Nara period?" If they answer correctly, directions to the next spot will be displayed.

[2169] By using the emotion engine, the difficulty of the questions can be adjusted even if the user is tired, for example. Also, if a user wants to relive a past trip to Kyoto, they can upload past photos to the app. The server analyzes the photos and generates a video that virtually recreates the Kyoto trip. The user can then enjoy the video at home.

[2170] This will enable people to gain a deeper understanding of local history and culture and experience the joy of travel without having to actually visit the area. Virtual travel will also be a new way to provide the joy of travel to elderly people and those with mobility difficulties.

[2171] The processing flow will be explained below.

[2172] User registration and information management

[2173] Step 1:

[2174] The user downloads and installs a dedicated application onto their smartphone.

[2175] Step 2:

[2176] A user launches the application and enters profile information such as name, age, and historical topics of interest.

[2177] Step 3:

[2178] The device checks the entered information and temporarily stores it.

[2179] Step 4:

[2180] The device sends the profile information to the server.

[2181] Step 5:

[2182] The server analyzes the received information and stores it in a database.

[2183] Step 6:

[2184] The server sends a message to the terminal indicating that the information has been saved.

[2185] Step 7:

[2186] The terminal displays a confirmation message to the user.

[2187] Automatic guidance upon arrival at destination

[2188] Step 1:

[2189] The device periodically obtains the user's current location using GPS.

[2190] Step 2:

[2191] The terminal transmits the obtained current location data to the server.

[2192] Step 3:

[2193] The server analyzes the user's current location and verifies that it is the tourist spot that was set.

[2194] Step 4:

[2195] If the server determines that the area is a tourist destination, it generates local guide data for the area.

[2196] Step 5:

[2197] The server transmits local guide data to the terminal.

[2198] Step 6:

[2199] The terminal displays the received local guidance data to the user, and the audio guide is automatically played.

[2200] Sightseeing with a Game-like Feel

[2201] Step 1:

[2202] The user selects a puzzle game or quiz mode within the app.

[2203] Step 2:

[2204] The terminal transmits the user's selection information to the server.

[2205] Step 3:

[2206] The server generates questions about local history and culture depending on the mode selected.

[2207] Step 4:

[2208] The server generates questions and sends them to the device.

[2209] Step 5:

[2210] The terminal presents the problem to the user.

[2211] Step 6:

[2212] The user answers the question.

[2213] Step 7:

[2214] The terminal sends the user's answer to the server.

[2215] Step 8:

[2216] The server evaluates the user's answer and determines whether it is correct.

[2217] Step 9:

[2218] The server generates information about the next spot and sends it to the terminal.

[2219] Step 10:

[2220] The device displays information about the next spot and provides directions to the location to the user.

[2221] Optimizing user experience with an emotion engine

[2222] Step 1:

[2223] The device analyzes the user's facial expressions and voice using an emotion engine to recognize emotions.

[2224] Step 2:

[2225] The device transmits the recognized emotion data to the server.

[2226] Step 3:

[2227] The server analyzes the emotional data and adjusts the local guide data and quiz questions.

[2228] Step 4:

[2229] The server transmits the adjusted data to the terminal.

[2230] Step 5:

[2231] The terminal displays and plays back the adjusted local guidance data and problems to the user.

[2232] Relive your travel memories

[2233] Step 1:

[2234] Users can enter memories of past trips within the app, or upload photos and notes.

[2235] Step 2:

[2236] The device sends the uploaded information to the server.

[2237] Step 3:

[2238] The server parses the data it receives.

[2239] Step 4:

[2240] The server generates content that recreates the virtual journey.

[2241] Step 5:

[2242] The server transmits the generated content to the terminal.

[2243] Step 6:

[2244] The terminal displays and plays the virtual travel content to the user.

[2245] Virtual travel at home

[2246] Step 1:

[2247] The user selects the "Virtual Travel" mode within the app.

[2248] Step 2:

[2249] The terminal transmits the selection information to the server.

[2250] Step 3:

[2251] The server generates a virtual sightseeing tour based on information about the desired travel destination.

[2252] Step 4:

[2253] The server transmits the generated virtual tour data to the terminal.

[2254] Step 5:

[2255] The terminal streams the virtual tour and provides it to the user.

[2256] Example 2

[2257] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2258] Conventional travel guide systems have the drawback of making it difficult for elderly people and those with mobility issues to use, as they require users to actually visit the destination to enjoy its attractions. Furthermore, the user experience is limited due to the lack of guidance that takes into account the user's emotions while traveling and the lack of functionality to recreate memories of past trips.

[2259] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2260] In this invention, the server includes means for inputting profile information through an application installed by the user and transmitting the information to the server, means for the server to confirm the user's current location based on GPS data, means for the server to generate local guidance data based on the current location confirmed by the server and transmit the data to the terminal, means for the terminal to display and play the received local guidance data to the user, means for the terminal to analyze the user's emotions and transmit the emotion data to the server, means for the server to adjust the local guidance data and the difficulty of the questions based on the emotion data and transmit the adjustment data to the terminal, means for the user to input past travel memories or upload photos and notes through the application and transmit the data to the server, means for the server to analyze the transmitted data, generate virtual travel content and transmit it to the terminal, and means for the terminal to display and play the virtual travel content to the user. This makes it possible to provide optimal guidance according to the user's emotions and to enhance the user experience through virtual travel.

[2261] "Profile Information" is information entered by a user that identifies them, such as their name, age, and historical topics of interest.

[2262] A "server" is a central computer device on a network that receives, analyzes, and stores data sent by users.

[2263] "GPS Data" means a user's current latitude and longitude information obtained from the Global Positioning System.

[2264] "Local guide data" is information and guide data about tourist spots that is generated based on the user's current location and interests.

[2265] "Terminal" refers to a mobile device such as a smartphone or tablet operated by a user.

[2266] "Emotion data" is information about the emotional state obtained by analyzing the user's facial expressions and voice.

[2267] "Virtual travel content" is digital content that provides a virtual travel experience, generated based on the user's travel records and uploaded photos and notes.

[2268] The "virtual travel mode" is a system mode that can be selected by the user to have a virtual travel experience.

[2269] "Difficulty of questions" refers to the level of difficulty of the questions in the generated quiz or puzzle game.

[2270] The present invention is a system for improving travel experiences through user-installed applications, providing appropriate guidance, quizzes, and virtual tours based on the user's emotions and interests. The system operates by combining the user's smartphone, a central server, and an emotion analysis engine.

[2271] User registration and information management

[2272] Users first install the application on their smartphone and enter their profile information, which includes their name, age, and historical topics of interest. The device temporarily stores this information and sends it to a server using the HTTPS protocol. The server analyzes the received data and stores it in a database, such as MySQL or PostgreSQL. The server then sends a message to the device indicating that the information has been saved, and the device notifies the user.

[2273] Automatic guidance upon arrival at destination

[2274] The device periodically obtains the user's current location using a GPS module and sends it to the server. The server analyzes the user's current location, and if that location is a tourist spot, it generates tourist spot information and sends it to the device. The device then displays the received tourist spot information to the user and plays an audio guide. This allows the user to immediately obtain information about the location when they arrive at a tourist spot.

[2275] Sightseeing with a Game-like Feel

[2276] Within the app, users can select a puzzle game or quiz mode. The device sends the user's selection information to the server, which then generates questions related to the local history and culture according to the selected mode. The questions are automatically generated using a generative AI model. The server sends the generated questions to the device, which then presents them to the user. When the user answers, the answer is sent to the server, which evaluates it and sends information and directions to the next spot to the device.

[2277] Relive your travel memories

[2278] Users can enter memories of past trips and upload photos and notes within the app. The device sends this data to the server, which analyzes the data and generates virtual travel content. The generated content is sent to the device, where users can view and play it to virtually relive past trips.

[2279] Virtual travel at home

[2280] Users select the "Virtual Travel" mode within the app and input their desired travel destination. The device sends the selected information to the server, which then generates a virtual sightseeing tour based on that information. The virtual sightseeing tour is created using a generative AI model. The generated tour data is sent to the device, allowing users to enjoy the virtual tour at home via streaming playback.

[2281] Optimizing user experience with an emotion engine

[2282] The device captures the user's facial expressions and voice in real time and uses an emotion analysis engine to obtain emotional data. The emotional data is sent to a server, which then adjusts the on-site guidance data and the difficulty of the questions based on that data. The adjusted data is then sent back to the device, allowing the user to receive customized guidance and quizzes tailored to their own state.

[2283] Specific examples

[2284] For example, if a user selects "Visit historical spots in Nara," the smartphone's GPS acquires the user's current location and sends it to the server. The server generates guide information about the "Nara period" and sends it to the device. The device then begins providing audio and text guidance. If the user selects quiz mode, a sample question such as "What is a representative building from the Nara period?" is displayed. If the user answers "Todaiji Temple," the server evaluates the answer and displays guidance to the next spot.

[2285] An example of a prompt sentence is "Please generate a prompt sentence for a system in which a user uses the emotion engine at tourist spots in Nara and is shown a quiz about the Nara period."

[2286] This will allow people to gain a deeper understanding of local history and culture and experience the joys of travel without having to actually visit the area. It will also make it possible to offer the joys of travel in a new way to elderly people and those with mobility difficulties.

[2287] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2288] Step 1:

[2289] The user installs the application on their smartphone.

[2290] The user downloads and installs the application from the smartphone's app store, then launches the application.

[2291] Input: App store download link

[2292] Output: Installed applications

[2293] Step 2:

[2294] The user launches the application and enters profile information.

[2295] Users launch the application and fill in fields with information such as their name, age, and historical topics of interest.

[2296] Input: Profile information entered by the user (name, age, topics of interest)

[2297] Output: Temporarily saved profile information

[2298] Step 3:

[2299] The device checks the entered information and temporarily stores it.

[2300] The terminal checks whether the input is in the correct format (for example, whether the name is not empty or the age is in numeric format), and then stores the information temporarily in memory.

[2301] Input: Profile information entered by the user

[2302] Output: Format-checked and temporarily saved profile information

[2303] Step 4:

[2304] The device sends the profile information to the server.

[2305] The device sends the checked profile information to the server using the HTTPS protocol.

[2306] Input: Format-checked and temporarily saved profile information

[2307] Output: Profile information sent to the server

[2308] Step 5:

[2309] The server analyzes the received information and stores it in a database.

[2310] The server parses the received profile information in JSON format and stores it in a database, for example, adding the user ID and timestamp.

[2311] Input: Profile information sent to the server

[2312] Output: Profile information stored in the database

[2313] Step 6:

[2314] The server sends a message to the terminal indicating that the information has been saved.

[2315] The server confirms that the data has been saved to the database, generates a message indicating that the data has been saved, and sends it to the terminal.

[2316] Input: Profile information stored in the database

[2317] Output: Message indicating information has been saved

[2318] Step 7:

[2319] The terminal displays a confirmation message to the user.

[2320] The terminal displays a message indicating that the information has been saved as received from the server as a pop-up notification or on the screen.

[2321] Input: Message that information has been saved

[2322] Output: User notification

[2323] Step 8:

[2324] The device periodically obtains the user's current location using GPS.

[2325] The device uses the smartphone's GPS module to obtain the user's current location at regular intervals (e.g., every 5 minutes).

[2326] Input: None (periodic acquisition of GPS module)

[2327] Output: User's current location data (latitude, longitude)

[2328] Step 9:

[2329] The terminal transmits the obtained current location data to the server.

[2330] The device converts the acquired location data into JSON format and sends it to the server using the HTTPS protocol.

[2331] Input: User's current location data (latitude, longitude)

[2332] Output: Location data sent to the server

[2333] Step 10:

[2334] The server analyzes the user's current location and confirms that it is a tourist spot.

[2335] The server compares the received current location data with a list of pre-registered tourist spots and analyzes whether the user's location is within a tourist spot.

[2336] Input: Location data sent to the server

[2337] Output: Tourist destination determination result

[2338] Step 11:

[2339] The server generates tourist destination information.

[2340] If the server confirms that the destination is a tourist spot, it generates information about the historical background and attractions of the destination.

[2341] Input: Tourist destination determination result

[2342] Output: Generated tourist destination information

[2343] Step 12:

[2344] The server transmits local guide data to the terminal.

[2345] The server sends the generated tourist destination information to the terminal in JSON format.

[2346] Input: Generated tourist destination information

[2347] Output: Local guide data sent to the terminal

[2348] Step 13:

[2349] The terminal displays the received local guidance data to the user and plays back the audio guide.

[2350] The device displays the received guidance data on the display, plays the audio guide file, and may notify the user using push notifications or alerts.

[2351] Input: Local guide data sent to the terminal

[2352] Output: Display to user, play audio guide

[2353] Step 14:

[2354] The user selects a puzzle game or quiz mode within the app.

[2355] Users select the "Play" tab from the in-app menu and choose their preferred mode.

[2356] Input: Mode selected by user

[2357] Output: Selected mode information

[2358] Step 15:

[2359] The terminal transmits the user's selection information to the server.

[2360] The device converts the selected mode information into JSON format and sends it to the server.

[2361] Input: Selected mode information

[2362] Output: Mode information sent to the server

[2363] Step 16:

[2364] The server generates questions related to local history and culture.

[2365] Based on the received mode information, the server uses a generative AI model to generate quizzes and questions about local history and culture.

[2366] Input: Mode information sent to the server

[2367] Output: Generated quizzes and questions

[2368] Step 17:

[2369] The server generates questions and sends them to the device.

[2370] The server sends the generated questions to the terminal in JSON format.

[2371] Input: Generated quizzes and questions

[2372] Output: Problems sent to terminal

[2373] Step 18:

[2374] The terminal presents the problem to the user.

[2375] The terminal displays the received questions on the screen so that the user can answer them.

[2376] Input: Question sent to terminal

[2377] Output: The problem presented to the user

[2378] Step 19:

[2379] The user answers the question.

[2380] Users can answer the questions by tapping on the appropriate answer option or by writing their own answer.

[2381] Input: User response

[2382] Output: User response data

[2383] Step 20:

[2384] The terminal sends the user's answer to the server.

[2385] The device converts the user's response into JSON format and sends it to the server.

[2386] Input: User response data

[2387] Output: Response data sent to the server

[2388] Step 21:

[2389] The server evaluates the user's answer and determines whether it is correct.

[2390] Based on the received answers, the server runs an algorithm to evaluate whether the answers are correct or incorrect and generates a judgment result.

[2391] Input: Response data sent to the server

[2392] Output: Judgment result (correct or incorrect)

[2393] Step 22:

[2394] The server generates information about the next spot and sends it to the terminal.

[2395] Based on the results of the determination, the server generates information about the next tourist spot and guidance data to that location, and sends it to the terminal.

[2396] Input: Judgment result

[2397] Output: Next spot information and guidance data

[2398] Step 23:

[2399] The terminal displays information and directions to the next spot to the user.

[2400] The device displays the received next s...

Claims

1. means for inputting profile information through a user-installed application and transmitting the information to a server; A means for the server to check the user's current location based on GPS data; means for generating local guidance data based on the current location confirmed by the server and transmitting the data to the terminal; means for displaying and reproducing the local guidance data received by the terminal to the user; A system including:

2. A means for the server to generate questions about local history and culture based on the mode selected by the user and transmit the questions to the terminal; a means for the terminal to present questions to the user, collect answers from the user, and transmit the answers to the server; A means for the server to evaluate the user's answer, generate the following information, and transmit it to the terminal; a means for the terminal to display information about the next spot and guidance to the location to the user; The system of claim 1 , comprising:

3. A means for users to input memories of past trips or upload photos and notes; A means for the server to analyze the uploaded data and generate content that recreates the virtual journey; A means for displaying and playing the generated virtual travel content to the user on the terminal; The system of claim 1 , comprising:

Images