system

Abstract

We provide the system. [Solution] A terminal means for acquiring the user's location information and history information, A computing device means that generates the optimal sightseeing route for the user using a generative model based on acquired location information and historical information, A terminal device that provides guidance information to users based on the generated tourist route, A system that includes means for detecting user behavior and enabling the acquisition of stamps.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional tourist guidance systems, it has been difficult to propose an optimal tourist route based on the preferences and visit histories of individual users, and there has been a problem that the attractions of the region cannot be effectively conveyed. Also, even if the number of visitors can be increased in the short term, there has been a problem that it is difficult to provide an attraction that makes users visit repeatedly.

Means for Solving the Problems

[0005] This invention acquires the user's location and history information, analyzes it using a generative model, and generates individually optimized sightseeing routes. By providing these sightseeing routes to the terminal, a system is built that provides users with specific guidance information. Furthermore, when the user acquires a stamp, additional entertainment content is provided, and new guidance information for the next visit is generated, allowing users to experience the charm of the region over the long term.

[0006] "Users" refer to individuals who use this system to visit a region and engage in tourism experiences.

[0007] "Location information" refers to data that identifies the geographical location where a user is currently or has been located in the past.

[0008] "History information" refers to recorded data about the routes and places visited by users during their daily lives or while traveling.

[0009] A "terminal" is an electronic device owned by a user, on which the system's application is installed, and which is used to display information and perform operations.

[0010] A "generative model" refers to an algorithm or technology used to analyze collected data and create optimized suggestions for users.

[0011] "Computing device" refers to electronic devices or servers that process and analyze data, generate routes, and manage records.

[0012] A "tour route" refers to the route that a user plans to take while sightseeing, and includes the order and locations of places to visit suggested by the system.

[0013] "Information" refers to directional information provided to users, including descriptions of tourist spots, directions, opening hours, and event information.

[0014] "Stamp" refers to digital or physical authentication that proves a visit to a specific location.

[0015] "Entertainment content" refers to elements of entertainment such as quizzes, games, videos, etc. that users can enjoy.

Brief Explanation of Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment.. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. ) [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), etc.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disk (e.g., hard disk), or magnetic tape, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] The system of this invention is implemented by combining various technologies to optimize the tourist experience for users.

[0038] The system configuration involves a mobile device carried by the user, with a tourism application installed. This device continuously collects location and historical information and transmits it to the server. Location information is obtained in real time using GPS functionality, and historical information includes past visit data.

[0039] The server receives the transmitted data and uses a generative model for analysis. The generative model generates the most suitable travel route for the user's preferences, referencing local tourist attractions and event information stored in the database. The generated route data consists of detailed guide information, including the order of tourist attractions and recommended visiting times.

[0040] This information is sent to the device and displayed to the user in an accessible format. Specifically, tourist routes are visualized on a map, and detailed information, local products, and cultural information for each location are shown.

[0041] Users follow the instructions on the device, and upon reaching a stamp point, the system confirms their location and prompts them to collect a stamp. After collecting the stamp, the device provides additional entertainment content, including location-related entertainment elements such as quiz games and videos about the history of the visited area.

[0042] Furthermore, the server records this activity history and uses it to generate more customized information for subsequent visits. This allows users to enjoy a more engaging and personalized sightseeing experience over the long term.

[0043] For example, if a user enjoys historical sites, the system will create a route based on their history, focusing on historical locations they haven't visited yet. Furthermore, mini-quizzes related to history are provided at each destination, deepening their knowledge and enhancing the experience. In this way, the system can individually optimize sightseeing according to each user's interests, allowing them to fully experience the charm of the region.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] The user launches the device with the tourism application installed and allows location sharing.

[0047] Step 2:

[0048] The device uses GPS to obtain real-time location information and sends it to the server along with the user's history information.

[0049] Step 3:

[0050] The server stores the received location and history information in a database. Based on this information, a generative model is used to analyze the user's interests and preferences.

[0051] Step 4:

[0052] Based on the analysis results, the server determines recommended tourist spots for the user. This allows it to calculate the optimal sightseeing route and create guide information.

[0053] Step 5:

[0054] The server sends guidance information to the terminal. This guidance information includes details of the sightseeing route and an overview of the places to visit.

[0055] Step 6:

[0056] The terminal displays the received guidance information to the user. It visually presents tourist routes and information about each spot on a map to guide the user.

[0057] Step 7:

[0058] Users follow the instructions on their device to navigate the sightseeing route and visit each stamp point.

[0059] Step 8:

[0060] The device detects when the user arrives at a stamp point and notifies them that they can obtain a stamp.

[0061] Step 9:

[0062] Users operate their devices to collect stamps and enjoy additional entertainment content (quizzes, mini-games, etc.).

[0063] Step 10:

[0064] The server then saves the acquired stamps and user activity history to prepare an optimized visit plan for the next visit.

[0065] (Example 1)

[0066] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0067] When providing tourism experiences, there is a lack of systems that generate optimal travel routes based on individual users' preferences and behavioral history, and furthermore, provide entertainment content to enrich the experience. The present invention aims to solve these problems and provide users with personalized tourism experiences.

[0068] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0069] In this invention, the server includes an information processing device means for acquiring the user's location information and history information, a computing device means for generating an optimal visit route for the user using a generation AI model based on the acquired location information and history information, and an information processing device means for providing guidance information to the user based on the generated visit route. This makes it possible to generate personalized visit routes according to the user's preferences and behavioral history, and to provide relevant entertainment content.

[0070] An "information processing device" is a device that acquires the user's location information and history information and transmits this information to a server.

[0071] A "generative AI model" is an algorithm that uses acquired user data to generate the optimal route for visits.

[0072] A "computational device" is a device that uses a generative AI model to calculate the optimal route for a user to visit.

[0073] "Guidance information" refers to the order of visits and information about each spot presented to the user based on the generated visit route.

[0074] A "digital certificate" is a digital certificate or piece of information that can be obtained by a user performing a specified action.

[0075] "Entertainment content" refers to information with entertainment elements, such as games and videos, that is provided to users when they obtain a digital certificate.

[0076] A description of embodiments for carrying out this invention will be given.

[0077] First, the terminal is a mobile device carried by the user, and a tourism application is installed on this device. The terminal uses GPS functionality to acquire the user's location information in real time and also records tourism history data. This information is continuously transmitted to the server.

[0078] The server uses a generative AI model to analyze location and historical information received from the terminal. This generative AI model refers to information on tourist spots and events stored in a database and is used to generate the most suitable sightseeing route for the user. For example, the generative AI model is given instructions such as "Based on the user's past visit history, suggest the next tourist spot to visit" as a prompt to perform the analysis.

[0079] The generated tourist route data includes the order of visits and recommended visiting times, which are compiled into detailed guide information. This guide information is transmitted to the terminal, which visualizes the tourist route on a map and displays it visually to the user. Information on local products and cultural background at each tourist spot is also presented.

[0080] Users navigate tourist destinations following the instructions on their device. Upon reaching specific points along the sightseeing route, known as "stamp points," the device prompts them to collect a stamp. This allows them to confirm their location and collect the stamp digitally. Furthermore, after collecting the stamp, the device offers additional entertainment content such as quiz games and videos about the visited locations, aiming to enhance the user's knowledge and enrich their entertainment experience.

[0081] For example, if a user enjoys historical sites, the system will suggest a sightseeing route based on unvisited historical spots that match their preferences. Through history quizzes at the visited locations, users can deepen their historical knowledge in an enjoyable way. In this way, the sightseeing experience is optimized according to the individual user's interests, making it possible to fully experience the charm of the region.

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] The device activates its GPS function to acquire current location information as soon as the user starts moving. This information is acquired at pre-set time intervals. The input is real-time location information, and the output is a data packet containing this location information. This data packet is sent to the server.

[0085] Step 2:

[0086] The device also collects past visit history information. This history information includes past visited locations and visit times retrieved from a history database. The input is history information extracted from the history database, and the output is comprehensive user data integrated with current location information. This integrated data is sent to the server.

[0087] Step 3:

[0088] The server receives comprehensive user data sent from the terminal. The input is current location and history data sent from the terminal, which is analyzed to understand the user's preferences and current state. Using a generative AI model, it receives a prompt message—for example, "Based on the user's preferences, please suggest the next tourist spot to visit"—and generates a suitable route. The output is this optimal route data.

[0089] Step 4:

[0090] The server sends the generated visit route data to the terminal. The input is the visit route data, and the output is the guidance information delivered to the terminal. The guidance information includes the order of visits and the recommended time to stay at each spot.

[0091] Step 5:

[0092] The terminal provides the user with received guidance information and visualizes the sightseeing route on a map. The input is guidance information from the server, and the output is a visualized sightseeing map and point-of-interest information. The user navigates based on the map.

[0093] Step 6:

[0094] The user visits tourist spots based on the terminal's directions. Upon reaching a stamp point, the terminal prompts the user to obtain a digital certificate. The input is the user's location information and the matching of the stamp point, and the output is the obtained digital certificate.

[0095] Step 7:

[0096] After obtaining a stamp, the terminal provides users with additional entertainment content. Input is the acquired digital certificate information, and output is entertainment content such as related quizzes and videos. This allows users to enjoy their sightseeing even more.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] Modern tourism experiences typically involve physical travel and are subject to time and cost constraints. Furthermore, even upon reaching a destination, providing personalized information and experiences tailored to each user is challenging, often resulting in standardized services. In particular, users living in remote locations face limitations in deeply experiencing foreign cultures and histories. Therefore, there is a need to develop systems that allow users to enjoy individually customized tourism experiences through virtual spaces.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes a device means for acquiring user coordinate information and history information, a computing device means for generating an optimal spatial path to the user using a generative model based on the acquired coordinate information and history information, and means for providing three-dimensional spatial information to the user through a visual device. This makes it possible for users to enjoy a individually tailored sightseeing experience in a virtual space while staying at home.

[0102] "Users" refers to individuals who seek to enjoy a tourism experience in a virtual space.

[0103] "Coordinate information" refers to geographical data used to indicate a user's current location, and is obtained using technologies such as GPS.

[0104] "History information" refers to records of places a user has visited and experiences they have had in the past, and is data used to provide personalized experiences.

[0105] "Device" refers to hardware or its components designed to perform a specific function.

[0106] A "generative model" is an algorithm that uses techniques such as machine learning to generate new outputs based on input data.

[0107] A "spatial route" refers to a travel path that a user can follow within a virtual space, and includes tourist information related to that route.

[0108] The term "arithmetic unit" refers to a computer or its hardware configuration that performs calculations to make decisions or predictions based on input data.

[0109] A "virtual stamp" is a digital proof that a user has reached a specific location through virtual tourism.

[0110] "Visual devices" refer to hardware used to present virtual reality or augmented reality to users, and include smart glasses and head-mounted displays.

[0111] In the system that realizes this invention, first, a terminal collects the user's coordinate information and history information. A mobile device such as a smartphone utilizing GPS functionality is used for this purpose. The terminal's role is to transmit the collected data to a server. Upon receiving this data, the server uses a generative AI model to generate an optimal spatial route based on the user's preferences and past visit history. In this process, user-specific data is analyzed using a machine learning framework such as TENSORFLOW®.

[0112] The generated spatial paths are calculated by a computing device and then provided to the user via a terminal. Three-dimensional virtual space information is displayed to the user wearing a visual device such as smart glasses or a head-mounted display. This allows users to virtually experience foreign lands and cultures without physically traveling. The display on the visual device is built using a platform such as Unity.

[0113] Furthermore, during the process of users acquiring virtual stamps, the system detects the user's movements and recognizes their arrival at a specific location. Upon confirmation of arrival, the server provides additional entertainment content, such as quizzes about the history and culture of the visited virtual location. This allows users to have an experience that goes beyond mere sightseeing and deepens their knowledge.

[0114] As a concrete example, if a user wishes to deepen their knowledge of Florence, Italy, the system will create a virtual tour focusing on Renaissance landmarks based on the user's interests. The user will visit famous museums and historical buildings in the virtual space and take related quizzes. An example of a prompt to the generative AI model used in this process would be: "Create a virtual tour route themed on Renaissance landmarks in Florence and provide an interactive quiz that includes relevant historical background information."

[0115] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0116] Step 1:

[0117] The device acquires the user's coordinate information. Using GPS functionality, it obtains the user's location in real time and records this information on the device. The output is the user's current coordinate data.

[0118] Step 2:

[0119] The device collects user history information. It extracts previously visited tourist spots and user activity history from a database and constructs history information based on this. The output provides information about the user's interests and past visit history.

[0120] Step 3:

[0121] The terminal transmits acquired coordinate and historical information to the server. This input data serves as foundational data to facilitate further analysis on the server.

[0122] Step 4:

[0123] The server processes the user's coordinate and history information using a generative AI model. It utilizes a machine learning framework (such as TensorFlow) for data analysis to calculate the optimal sightseeing route based on the input data. The output is a customized spatial route tailored to the user's interests.

[0124] Step 5:

[0125] The server sends the generated spatial route data to the terminal. This route includes related tourist spots and associated information.

[0126] Step 6:

[0127] The device uses visual devices to display three-dimensional spatial information to the user. Leveraging the Unity platform, it visually represents tourist information and routes. This allows users to begin a visual sightseeing experience within a virtual space.

[0128] Step 7:

[0129] The user explores a virtual space, and the server detects the user's location and movements. When the user reaches a specific tourist spot, the terminal recognizes their arrival and prompts them to obtain a virtual marker.

[0130] Step 8:

[0131] Once the server confirms that a virtual stamp has been obtained, it provides additional entertainment content. For example, it might generate a quiz about the relevant history or culture. This would use the prompt message, "If a virtual stamp has been obtained, please provide an interactive quiz that reflects the corresponding historical information."

[0132] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0133] This invention provides a system for personalizing the tourism experience by taking into account the user's emotional state. This system is implemented by combining the acquisition of standard location and historical information with an emotion engine.

[0134] The terminal is a mobile device that users carry with them daily, and it has a tourism application installed. This terminal provides tourist information based on the collection of the user's location and history information. In addition, the terminal uses sensors such as a camera and microphone to detect the user's emotional state (facial expressions and voice) and transmits the acquired data to a server. This enables real-time sentiment analysis.

[0135] The server analyzes received location information, historical information, and sentiment data. Using a generative model, it generates optimized sightseeing routes based on each user's preferences and emotions at the time. Through the sentiment engine, it enables dynamic sightseeing guidance that takes into account the user's current emotional state.

[0136] For example, if the user is perceived as active, the system will suggest routes that emphasize sports and activity-related tourist spots. On the other hand, if the user is judged to be relaxed, it will recommend quiet and calming spots such as art museums or cafes. In this way, the device provides guidance information accordingly and continuously monitors the user's emotions in real time.

[0137] Furthermore, when a user reaches a stamp point and obtains a stamp, the system presents entertainment content tailored to the individual user's emotions based on feedback from the emotion engine. For example, if a user is excited, it provides an active game; conversely, if they are calm, it delivers an informative documentary video, thereby improving user satisfaction.

[0138] This system allows tourism experiences to be combined with the user's temporary emotional state, providing individually optimized routes and content, thereby improving the quality of tourism.

[0139] The following describes the processing flow.

[0140] Step 1:

[0141] Users install a tourism application on their device and allow the collection of location and sentiment data.

[0142] Step 2:

[0143] The device obtains the user's location information using GPS and sends it to the server along with the user's current history data.

[0144] Step 3:

[0145] The device uses its built-in camera and microphone to periodically collect the user's facial expressions and voice, and sends this data to a server as emotion data.

[0146] Step 4:

[0147] The server integrates received location information, historical information, and sentiment data, and uses a generative model to analyze the user's interests and current sentiment state.

[0148] Step 5:

[0149] Based on the analysis results, the server calculates the most suitable sightseeing route for the user and generates real-time, emotion-responsive guidance information.

[0150] Step 6:

[0151] The server sends the generated information to the terminal, including specific tourist spots and recommended activities.

[0152] Step 7:

[0153] The terminal displays navigation information to the user. Depending on the user's status, it presents an interactive map and detailed information.

[0154] Step 8:

[0155] Users follow the on-screen guide to travel along the sightseeing route and visit various stamp points.

[0156] Step 9:

[0157] The device detects when the user reaches a stamp point and, along with obtaining the stamp, provides entertainment content tailored to the user's emotions.

[0158] Step 10:

[0159] The server stores the user's behavior history, acquired stamp data, and sentiment feedback, and uses this information for further customization during subsequent visits.

[0160] (Example 2)

[0161] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0162] Conventional tourist information systems only provide sightseeing routes based on the user's location and history information, and have the limitation of not being able to personalize the guidance by taking into account the user's emotional state. As a result, it was difficult to provide a sightseeing experience that was more suited to the user's temporary emotional state, and there was room to improve the quality and satisfaction of sightseeing.

[0163] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0164] In this invention, the server includes terminal means for sensing and acquiring the user's emotional state in addition to the user's location information and history information; information processing means for generating an optimal sightseeing route that takes the user's emotional state into consideration using a generative model based on the acquired location information, history information, and emotional data; and terminal means for providing dynamic guidance information to the user based on the generated sightseeing route. This makes it possible to provide sightseeing routes and entertainment content optimized for the user's current emotional state.

[0165] "User location information" refers to geographical data that indicates the user's current location.

[0166] "History information" refers to data based on records of places and activities that users have visited in the past.

[0167] "Emotional state" refers to data that represents the user's psychological state, and is obtained from facial expressions and voice.

[0168] A "terminal" is a mobile device carried by a user, which is a device that collects and transmits various types of data.

[0169] A "generative model" is an algorithm and program used to derive personalized conclusions based on various data.

[0170] An "information processing device" is a machine or system that analyzes received data and generates and distributes useful information.

[0171] "Dynamic guidance information" refers to guide information that changes based on the user's current and past data.

[0172] "Entertainment content" refers to digital resources, including entertainment, that are provided to suit the user's emotions and preferences.

[0173] A "stamp point" is a digital mark that indicates a user has achieved a specific action or location.

[0174] This invention relates to a system that personalizes the tourist experience by taking into account the emotional state of the user.

[0175] The terminal is a personal information terminal that users carry with them daily, and it has a tourism application installed to implement this system. This application collects the user's location and history information using GPS and a history database. The terminal also has a built-in camera (image sensor) and microphone (audio sensor), which are used to analyze the user's facial expressions and voice in real time and acquire their emotional state. The data obtained in this way is transmitted to a server via the internet.

[0176] The server is an information processing device located in the cloud that integrates received location information, historical information, and emotional data, and analyzes them using a generative AI model. The server utilizes this generative model to generate optimized sightseeing routes based on the user's individual emotional state and preferences. The generated sightseeing routes are dynamically adjusted by the emotional engine and provided to the terminal in real time. For example, if the user is active, a route including sightseeing spots that emphasize sports and activities will be suggested, taking this into account.

[0177] Furthermore, when a user obtains a stamp at a location they visit, the device receives feedback from the server, suggesting entertainment content based on the user's emotional state (e.g., active games or quiet documentary videos) to further engage the user.

[0178] For example, if a user sightseeing in Kyoto enjoys a lively festival in the morning, the server might determine that the user is looking for an active experience in the afternoon and suggest a route that includes a geisha experience in Gion or nearby activity facilities. In response, an example of a prompt message entered into the system might be, "The user is currently active. Please generate a recommended afternoon sightseeing route in Kyoto. Avoid quiet spots and prioritize energetic experiences."

[0179] In this way, this invention dynamically provides a tourism experience tailored to the emotional state of the user, thereby improving the quality and satisfaction of tourism.

[0180] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0181] Step 1:

[0182] The device is the user's everyday mobile device, and first, it obtains the user's current location using location services. In parallel, it retrieves the user's past movement history from a history database. This uses GPS signals, network data, and data from user interactions within the application. The retrieved data is temporarily stored on the device and prepared for subsequent processing.

[0183] Step 2:

[0184] The device uses its built-in camera and microphone to capture the user's current facial expressions and voice. By analyzing this sensor data, the device determines the user's emotional state. For example, an AI-based algorithm analyzes changes in voice tone and facial expressions in real time to generate emotional data such as "excited" or "relaxed." This generated emotional data, along with location and historical information, is packetized for use in later steps.

[0185] Step 3:

[0186] The device transmits the location information, history information, and sentiment data collected in steps 1 and 2 to the server via the internet connection. The data is encrypted to protect user privacy. After transmitting the data, the device waits for a response from the server.

[0187] Step 4:

[0188] The server receives the data, performs initial processing, and then analyzes this data using a generative AI model. Specifically, it identifies the current location from location information, extracts past behavioral patterns from historical information, and understands the current emotional state from sentiment data. Using this data, the server generates the optimal sightseeing route based on prompts. The generative model, based on its learned algorithms, selects the most appropriate personalized route from multiple candidate routes.

[0189] Step 5:

[0190] The server sends generated tourist routes and related information to the device. This includes dynamically adjusted tourist information and recommended entertainment content. This provides users with real-time, personalized guidance.

[0191] Step 6:

[0192] Users can view sightseeing suggestions and entertainment content received from the server via their devices and use them to help plan their activities at their destination. For example, they can enjoy sightseeing while viewing recommended routes and spots, and enrich their travel experience by using content that suits their emotional state.

[0193] Step 7:

[0194] The device continuously monitors emotional data while the user continues sightseeing, sending new data to the server whenever it becomes available. This continuous data provision and analysis ensures that the sightseeing experience is always up-to-date and that the user is continuously provided with information optimized for them.

[0195] (Application Example 2)

[0196] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0197] Current tourist information systems struggle to provide personalized experiences that take into account the emotional state of users. Furthermore, existing systems are unable to implement dynamic purchase suggestions based on emotions, resulting in limited suggestions for products and services that are best suited to the user.

[0198] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0199] In this invention, the server includes information terminal means for acquiring the user's location information, history information, and emotional state; computing device means for generating an optimal route according to the user's emotional state using a generative model based on the acquired data; and information terminal means for dynamically customizing guidance information and purchase suggestions based on the emotional state. This enables a personalized sightseeing experience and optimal product suggestions based on the user's real-time emotional state.

[0200] An "information terminal" is a portable device used to acquire a user's location information, history information, and emotional state, and to provide tourist information and purchase suggestions.

[0201] A "computational device" is a device that uses a generative model based on acquired data to dynamically generate paths and suggestions that correspond to the user's emotional state.

[0202] A "generative model" is a model that predicts users' preferences and emotional states through data analysis, and uses that information to create optimal tourism experiences and product recommendations.

[0203] "Emotional state" refers to the psychological state expressed by the user during the experience, and is analyzed from facial expressions, voice, and other factors.

[0204] "Purchase recommendations" are the process of recommending relevant products and services based on the user's emotional state.

[0205] This system consists of user-owned information terminals and a server. The information terminals are portable devices such as smartphones and smart glasses, equipped with sensors (cameras, microphones, etc.) to acquire the user's location information, history information, and emotional state. The information terminals transmit the acquired data to the server. The server has a generative AI model for analyzing the received data, which is used to generate optimal sightseeing routes and purchase suggestions that take the user's emotional state into consideration.

[0206] The server analyzes data using sentiment analysis libraries such as Google® Cloud Vision API and IBM Watson® Tone Analyzer. For example, if a particular user is analyzed to be in a relaxed emotional state, the server will suggest tourist spots such as art museums or quiet cafes. At the same time, it will also present products and services suitable for that emotional state as purchase suggestions. The information terminal plays the role of guiding the user through the information fed back from the server.

[0207] For example, if a camera captures a user smiling while sightseeing, the system can determine that the user is enjoying themselves and suggest entertainment-related products and services. An example of a prompt message to the generating AI would be, "Analyze the user's smiling photo, and if it determines they are excited, recommend the latest entertainment products."

[0208] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0209] Step 1:

[0210] The user activates the information terminal and starts the tourist information application. The terminal uses its camera and microphone to capture the user's facial expressions and voice in real time. This collects information about the user's emotional state as input data.

[0211] Step 2:

[0212] The device transmits acquired emotional state information, location information, and user history information to the server. The server analyzes this input data using a generating AI model. It classifies emotional states using an emotional analysis library and narrows down potential tourist spots and purchase suggestions based on the results.

[0213] Step 3:

[0214] Based on the analysis results it generates, the server produces sightseeing routes and purchase suggestions best suited to the user's emotional state. For example, it selects active tourist spots for a user in an excited state. As output, user-optimized information is generated, and feedback is prepared.

[0215] Step 4:

[0216] The server sends optimized sightseeing routes and shopping suggestions back to the information terminal. The terminal receives this information and presents it to the user as visual or auditory information. For example, it might display a map of the sightseeing route on the screen and provide shopping suggestions via voice guidance.

[0217] Step 5:

[0218] Users can enjoy sightseeing based on the information presented. If a new emotional state is detected, the process restarts from step 1, and guidance and suggestions are updated in real time.

[0219] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0220] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0221] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0222] [Second Embodiment]

[0223] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0224] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0225] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0226] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0227] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0228] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0229] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0230] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0231] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0232] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0233] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0234] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0235] The system of this invention is implemented by combining various technologies to optimize the tourist experience for users.

[0236] The system configuration involves a mobile device carried by the user, with a tourism application installed. This device continuously collects location and historical information and transmits it to the server. Location information is obtained in real time using GPS functionality, and historical information includes past visit data.

[0237] The server receives the transmitted data and uses a generative model for analysis. The generative model generates the most suitable travel route for the user's preferences, referencing local tourist attractions and event information stored in the database. The generated route data consists of detailed guide information, including the order of tourist attractions and recommended visiting times.

[0238] This information is sent to the device and displayed to the user in an accessible format. Specifically, tourist routes are visualized on a map, and detailed information, local products, and cultural information for each location are shown.

[0239] Users follow the instructions on the device, and upon reaching a stamp point, the system confirms their location and prompts them to collect a stamp. After collecting the stamp, the device provides additional entertainment content, including location-related entertainment elements such as quiz games and videos about the history of the visited area.

[0240] Furthermore, the server records this activity history and uses it to generate more customized information for subsequent visits. This allows users to enjoy a more engaging and personalized sightseeing experience over the long term.

[0241] For example, if a user enjoys historical sites, the system will create a route based on their history, focusing on historical locations they haven't visited yet. Furthermore, mini-quizzes related to history are provided at each destination, deepening their knowledge and enhancing the experience. In this way, the system can individually optimize sightseeing according to each user's interests, allowing them to fully experience the charm of the region.

[0242] The following describes the processing flow.

[0243] Step 1:

[0244] The user launches the device with the tourism application installed and allows location sharing.

[0245] Step 2:

[0246] The device uses GPS to obtain real-time location information and sends it to the server along with the user's history information.

[0247] Step 3:

[0248] The server stores the received location and history information in a database. Based on this information, a generative model is used to analyze the user's interests and preferences.

[0249] Step 4:

[0250] Based on the analysis results, the server determines recommended tourist spots for the user. This allows it to calculate the optimal sightseeing route and create guide information.

[0251] Step 5:

[0252] The server sends guidance information to the terminal. This guidance information includes details of the sightseeing route and an overview of the places to visit.

[0253] Step 6:

[0254] The terminal displays the received guidance information to the user. It visually presents tourist routes and information about each spot on a map to guide the user.

[0255] Step 7:

[0256] Users follow the instructions on their device to navigate the sightseeing route and visit each stamp point.

[0257] Step 8:

[0258] The device detects when the user arrives at a stamp point and notifies them that they can obtain a stamp.

[0259] Step 9:

[0260] Users operate their devices to collect stamps and enjoy additional entertainment content (quizzes, mini-games, etc.).

[0261] Step 10:

[0262] The server then saves the acquired stamps and user activity history to prepare an optimized visit plan for the next visit.

[0263] (Example 1)

[0264] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0265] When providing tourism experiences, there is a lack of systems that generate optimal travel routes based on individual users' preferences and behavioral history, and furthermore, provide entertainment content to enrich the experience. The present invention aims to solve these problems and provide users with personalized tourism experiences.

[0266] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0267] In this invention, the server includes an information processing device means for acquiring the user's location information and history information, a computing device means for generating an optimal visit route for the user using a generation AI model based on the acquired location information and history information, and an information processing device means for providing guidance information to the user based on the generated visit route. This makes it possible to generate personalized visit routes according to the user's preferences and behavioral history, and to provide relevant entertainment content.

[0268] An "information processing device" is a device that acquires the user's location information and history information and transmits this information to a server.

[0269] A "generative AI model" is an algorithm that uses acquired user data to generate the optimal route for visits.

[0270] A "computational device" is a device that uses a generative AI model to calculate the optimal route for a user to visit.

[0271] "Guidance information" refers to the order of visits and information about each spot presented to the user based on the generated visit route.

[0272] A "digital certificate" is a digital certificate or piece of information that can be obtained by a user performing a specified action.

[0273] "Entertainment content" refers to information with entertainment elements, such as games and videos, that is provided to users when they obtain a digital certificate.

[0274] A description of embodiments for carrying out this invention will be given.

[0275] First, the terminal is a mobile device carried by the user, and a tourism application is installed on this device. The terminal uses GPS functionality to acquire the user's location information in real time and also records tourism history data. This information is continuously transmitted to the server.

[0276] The server uses a generative AI model to analyze location and historical information received from the terminal. This generative AI model refers to information on tourist spots and events stored in a database and is used to generate the most suitable sightseeing route for the user. For example, the generative AI model is given instructions such as "Based on the user's past visit history, suggest the next tourist spot to visit" as a prompt to perform the analysis.

[0277] The generated tourist route data includes the order of visits and recommended visiting times, which are compiled into detailed guide information. This guide information is transmitted to the terminal, which visualizes the tourist route on a map and displays it visually to the user. Information on local products and cultural background at each tourist spot is also presented.

[0278] Users navigate tourist destinations following the instructions on their device. Upon reaching specific points along the sightseeing route, known as "stamp points," the device prompts them to collect a stamp. This allows them to confirm their location and collect the stamp digitally. Furthermore, after collecting the stamp, the device offers additional entertainment content such as quiz games and videos about the visited locations, aiming to enhance the user's knowledge and enrich their entertainment experience.

[0279] For example, if a user enjoys historical sites, the system will suggest a sightseeing route based on unvisited historical spots that match their preferences. Through history quizzes at the visited locations, users can deepen their historical knowledge in an enjoyable way. In this way, the sightseeing experience is optimized according to the individual user's interests, making it possible to fully experience the charm of the region.

[0280] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0281] Step 1:

[0282] The device activates its GPS function to acquire current location information as soon as the user starts moving. This information is acquired at pre-set time intervals. The input is real-time location information, and the output is a data packet containing this location information. This data packet is sent to the server.

[0283] Step 2:

[0284] The terminal further collects past visit history information. This history information includes past visited locations and visit times obtained from the history database. The input is the history information extracted from the history database, and the output is comprehensive user data integrated with the current location information. This integrated data is sent to the server.

[0285] Step 3:

[0286] The server receives the comprehensive user data sent from the terminal. The input is the data of the current location and history information sent from the terminal, and this is analyzed to understand the user's preferences and current state. Using a generated AI model, a prompt sentence - for example, "Please propose the tourist spots to visit next based on the user's preferences" - is input to generate a suitable visit route. The output is this optimal visit route data.

[0287] Step 4:

[0288] The server sends the generated visit route data to the terminal. The input is the visit route data, and the output is the guidance information distributed to the terminal. The guidance information includes the visit order and the recommended stay time at each spot.

[0289] Step 5:

[0290] The terminal provides the received guidance information to the user and visualizes the tourist route on the map. The input is the guidance information from the server, and the output is the visualized tourist map and spot information. The user moves based on the map.

[0291] Step 6: <00(...) The user visits tourist spots based on the guidance of the terminal. When reaching the stamp point, the terminal prompts the acquisition of a digital certificate. The input is the match between the user's location information and the stamp point, and the output is the acquired digital certificate. <0(...)

[0293] Step 7:

[0294] After obtaining a stamp, the terminal provides users with additional entertainment content. Input is the acquired digital certificate information, and output is entertainment content such as related quizzes and videos. This allows users to enjoy their sightseeing even more.

[0295] (Application Example 1)

[0296] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0297] Modern tourism experiences typically involve physical travel and are subject to time and cost constraints. Furthermore, even upon reaching a destination, providing personalized information and experiences tailored to each user is challenging, often resulting in standardized services. In particular, users living in remote locations face limitations in deeply experiencing foreign cultures and histories. Therefore, there is a need to develop systems that allow users to enjoy individually customized tourism experiences through virtual spaces.

[0298] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0299] In this invention, the server includes a device means for acquiring user coordinate information and history information, a computing device means for generating an optimal spatial path to the user using a generative model based on the acquired coordinate information and history information, and means for providing three-dimensional spatial information to the user through a visual device. This makes it possible for users to enjoy a individually tailored sightseeing experience in a virtual space while staying at home.

[0300] "Users" refers to individuals who seek to enjoy a tourism experience in a virtual space.

[0301] "Coordinate information" refers to geographical data used to indicate a user's current location, and is obtained using technologies such as GPS.

[0302] "Historical information" refers to records of places visited by users in the past and experiences had, and is data used to provide individualized experiences.

[0303] "Device" refers to hardware designed to perform specific functions and its components.

[0304] "Generative model" is an algorithm for generating new outputs based on input data using techniques such as machine learning.

[0305] "Spatial path" refers to a movement path that a user can follow within a virtual space and includes tourist destination information related to that path.

[0306] "Arithmetic device" refers to a computer or its hardware configuration that performs calculations for making decisions or inferences based on input data.

[0307] "Virtual stamp" is a digital proof indicating that a user has reached a specific location through virtual tourism.

[0308] "Visual device" is hardware for presenting virtual reality or augmented reality to users, and smart glasses and head-mounted displays fall under this category.

[0309] In the system that realizes this invention, first, the terminal collects the user's coordinate information and historical information. For this, mobile devices such as smartphones that utilize the GPS function are used. The terminal serves to transmit the collected data to the server. When the server receives this data, it uses the generative AI model to generate an optimal spatial path based on the user's preferences and past visit history. In this process, user-specific data is analyzed using a machine learning framework such as TensorFlow.

[0310] The generated spatial paths are calculated by a computing device and then provided to the user via a terminal. Three-dimensional virtual space information is displayed to the user wearing a visual device such as smart glasses or a head-mounted display. This allows users to virtually experience foreign lands and cultures without physically traveling. The display on the visual device is built using a platform such as Unity.

[0311] Furthermore, during the process of users acquiring virtual stamps, the system detects the user's movements and recognizes their arrival at a specific location. Upon confirmation of arrival, the server provides additional entertainment content, such as quizzes about the history and culture of the visited virtual location. This allows users to have an experience that goes beyond mere sightseeing and deepens their knowledge.

[0312] As a concrete example, if a user wishes to deepen their knowledge of Florence, Italy, the system will create a virtual tour focusing on Renaissance landmarks based on the user's interests. The user will visit famous museums and historical buildings in the virtual space and take related quizzes. An example of a prompt to the generative AI model used in this process would be: "Create a virtual tour route themed on Renaissance landmarks in Florence and provide an interactive quiz that includes relevant historical background information."

[0313] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0314] Step 1:

[0315] The device acquires the user's coordinate information. Using GPS functionality, it obtains the user's location in real time and records this information on the device. The output is the user's current coordinate data.

[0316] Step 2:

[0317] The device collects user history information. It extracts previously visited tourist spots and user activity history from a database and constructs history information based on this. The output provides information about the user's interests and past visit history.

[0318] Step 3:

[0319] The terminal transmits acquired coordinate and historical information to the server. This input data serves as foundational data to facilitate further analysis on the server.

[0320] Step 4:

[0321] The server processes the user's coordinate and history information using a generative AI model. It utilizes a machine learning framework (such as TensorFlow) for data analysis to calculate the optimal sightseeing route based on the input data. The output is a customized spatial route tailored to the user's interests.

[0322] Step 5:

[0323] The server sends the generated spatial route data to the terminal. This route includes related tourist spots and associated information.

[0324] Step 6:

[0325] The device uses visual devices to display three-dimensional spatial information to the user. Leveraging the Unity platform, it visually represents tourist information and routes. This allows users to begin a visual sightseeing experience within a virtual space.

[0326] Step 7:

[0327] The user explores a virtual space, and the server detects the user's location and movements. When the user reaches a specific tourist spot, the terminal recognizes their arrival and prompts them to obtain a virtual marker.

[0328] Step 8:

[0329] Once the server confirms that a virtual stamp has been obtained, it provides additional entertainment content. For example, it might generate a quiz about the relevant history or culture. This would use the prompt message, "If a virtual stamp has been obtained, please provide an interactive quiz that reflects the corresponding historical information."

[0330] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0331] This invention provides a system for personalizing the tourism experience by taking into account the user's emotional state. This system is implemented by combining the acquisition of standard location and historical information with an emotion engine.

[0332] The terminal is a mobile device that users carry with them daily, and it has a tourism application installed. This terminal provides tourist information based on the collection of the user's location and history information. In addition, the terminal uses sensors such as a camera and microphone to detect the user's emotional state (facial expressions and voice) and transmits the acquired data to a server. This enables real-time sentiment analysis.

[0333] The server analyzes received location information, historical information, and sentiment data. Using a generative model, it generates optimized sightseeing routes based on each user's preferences and emotions at the time. Through the sentiment engine, it enables dynamic sightseeing guidance that takes into account the user's current emotional state.

[0334] For example, if the user is perceived as active, the system will suggest routes that emphasize sports and activity-related tourist spots. On the other hand, if the user is judged to be relaxed, it will recommend quiet and calming spots such as art museums or cafes. In this way, the device provides guidance information accordingly and continuously monitors the user's emotions in real time.

[0335] Furthermore, when a user reaches a stamp point and obtains a stamp, the system presents entertainment content tailored to the individual user's emotions based on feedback from the emotion engine. For example, if a user is excited, it provides an active game; conversely, if they are calm, it delivers an informative documentary video, thereby improving user satisfaction.

[0336] This system allows tourism experiences to be combined with the user's temporary emotional state, providing individually optimized routes and content, thereby improving the quality of tourism.

[0337] The following describes the processing flow.

[0338] Step 1:

[0339] Users install a tourism application on their device and allow the collection of location and sentiment data.

[0340] Step 2:

[0341] The device obtains the user's location information using GPS and sends it to the server along with the user's current history data.

[0342] Step 3:

[0343] The device uses its built-in camera and microphone to periodically collect the user's facial expressions and voice, and sends this data to a server as emotion data.

[0344] Step 4:

[0345] The server integrates received location information, historical information, and sentiment data, and uses a generative model to analyze the user's interests and current sentiment state.

[0346] Step 5:

[0347] Based on the analysis results, the server calculates the most suitable sightseeing route for the user and generates real-time, emotion-responsive guidance information.

[0348] Step 6:

[0349] The server sends the generated information to the terminal, including specific tourist spots and recommended activities.

[0350] Step 7:

[0351] The terminal displays navigation information to the user. Depending on the user's status, it presents an interactive map and detailed information.

[0352] Step 8:

[0353] Users follow the on-screen guide to travel along the sightseeing route and visit various stamp points.

[0354] Step 9:

[0355] The device detects when the user reaches a stamp point and, along with obtaining the stamp, provides entertainment content tailored to the user's emotions.

[0356] Step 10:

[0357] The server stores the user's behavior history, acquired stamp data, and sentiment feedback, and uses this information for further customization during subsequent visits.

[0358] (Example 2)

[0359] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0360] Conventional tourist information systems only provide sightseeing routes based on the user's location and history information, and have the limitation of not being able to personalize the guidance by taking into account the user's emotional state. As a result, it was difficult to provide a sightseeing experience that was more suited to the user's temporary emotional state, and there was room to improve the quality and satisfaction of sightseeing.

[0361] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0362] In this invention, the server includes terminal means for sensing and acquiring the user's emotional state in addition to the user's location information and history information; information processing means for generating an optimal sightseeing route that takes the user's emotional state into consideration using a generative model based on the acquired location information, history information, and emotional data; and terminal means for providing dynamic guidance information to the user based on the generated sightseeing route. This makes it possible to provide sightseeing routes and entertainment content optimized for the user's current emotional state.

[0363] "User location information" refers to geographical data that indicates the user's current location.

[0364] "History information" refers to data based on records of places and activities that users have visited in the past.

[0365] "Emotional state" refers to data that represents the user's psychological state, and is obtained from facial expressions and voice.

[0366] A "terminal" is a mobile device carried by a user, which is a device that collects and transmits various types of data.

[0367] A "generative model" is an algorithm and program used to derive personalized conclusions based on various data.

[0368] An "information processing device" is a machine or system that analyzes received data and generates and distributes useful information.

[0369] "Dynamic guidance information" refers to guide information that changes based on the user's current and past data.

[0370] "Entertainment content" refers to digital resources, including entertainment, that are provided to suit the user's emotions and preferences.

[0371] A "stamp point" is a digital mark that indicates a user has achieved a specific action or location.

[0372] This invention relates to a system that personalizes the tourist experience by taking into account the emotional state of the user.

[0373] The terminal is a personal information terminal that users carry with them daily, and it has a tourism application installed to implement this system. This application collects the user's location and history information using GPS and a history database. The terminal also has a built-in camera (image sensor) and microphone (audio sensor), which are used to analyze the user's facial expressions and voice in real time and acquire their emotional state. The data obtained in this way is transmitted to a server via the internet.

[0374] The server is an information processing device located in the cloud that integrates received location information, historical information, and emotional data, and analyzes them using a generative AI model. The server utilizes this generative model to generate optimized sightseeing routes based on the user's individual emotional state and preferences. The generated sightseeing routes are dynamically adjusted by the emotional engine and provided to the terminal in real time. For example, if the user is active, a route including sightseeing spots that emphasize sports and activities will be suggested, taking this into account.

[0375] Furthermore, when a user obtains a stamp at a location they visit, the device receives feedback from the server, suggesting entertainment content based on the user's emotional state (e.g., active games or quiet documentary videos) to further engage the user.

[0376] For example, if a user sightseeing in Kyoto enjoys a lively festival in the morning, the server might determine that the user is looking for an active experience in the afternoon and suggest a route that includes a geisha experience in Gion or nearby activity facilities. In response, an example of a prompt message entered into the system might be, "The user is currently active. Please generate a recommended afternoon sightseeing route in Kyoto. Avoid quiet spots and prioritize energetic experiences."

[0377] In this way, this invention dynamically provides a tourism experience tailored to the emotional state of the user, thereby improving the quality and satisfaction of tourism.

[0378] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0379] Step 1:

[0380] The device is the user's everyday mobile device, and first, it obtains the user's current location using location services. In parallel, it retrieves the user's past movement history from a history database. This uses GPS signals, network data, and data from user interactions within the application. The retrieved data is temporarily stored on the device and prepared for subsequent processing.

[0381] Step 2:

[0382] The device uses its built-in camera and microphone to capture the user's current facial expressions and voice. By analyzing this sensor data, the device determines the user's emotional state. For example, an AI-based algorithm analyzes changes in voice tone and facial expressions in real time to generate emotional data such as "excited" or "relaxed." This generated emotional data, along with location and historical information, is packetized for use in later steps.

[0383] Step 3:

[0384] The device transmits the location information, history information, and sentiment data collected in steps 1 and 2 to the server via the internet connection. The data is encrypted to protect user privacy. After transmitting the data, the device waits for a response from the server.

[0385] Step 4:

[0386] The server receives the data, performs initial processing, and then analyzes this data using a generative AI model. Specifically, it identifies the current location from location information, extracts past behavioral patterns from historical information, and understands the current emotional state from sentiment data. Using this data, the server generates the optimal sightseeing route based on prompts. The generative model, based on its learned algorithms, selects the most appropriate personalized route from multiple candidate routes.

[0387] Step 5:

[0388] The server sends generated tourist routes and related information to the device. This includes dynamically adjusted tourist information and recommended entertainment content. This provides users with real-time, personalized guidance.

[0389] Step 6:

[0390] Users can view sightseeing suggestions and entertainment content received from the server via their devices and use them to help plan their activities at their destination. For example, they can enjoy sightseeing while viewing recommended routes and spots, and enrich their travel experience by using content that suits their emotional state.

[0391] Step 7:

[0392] The device continuously monitors emotional data while the user continues sightseeing, sending new data to the server whenever it becomes available. This continuous data provision and analysis ensures that the sightseeing experience is always up-to-date and that the user is continuously provided with information optimized for them.

[0393] (Application Example 2)

[0394] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0395] Current tourist information systems struggle to provide personalized experiences that take into account the emotional state of users. Furthermore, existing systems are unable to implement dynamic purchase suggestions based on emotions, resulting in limited suggestions for products and services that are best suited to the user.

[0396] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0397] In this invention, the server includes information terminal means for acquiring the user's location information, history information, and emotional state; computing device means for generating an optimal route according to the user's emotional state using a generative model based on the acquired data; and information terminal means for dynamically customizing guidance information and purchase suggestions based on the emotional state. This enables a personalized sightseeing experience and optimal product suggestions based on the user's real-time emotional state.

[0398] An "information terminal" is a portable device used to acquire a user's location information, history information, and emotional state, and to provide tourist information and purchase suggestions.

[0399] A "computational device" is a device that uses a generative model based on acquired data to dynamically generate paths and suggestions that correspond to the user's emotional state.

[0400] A "generative model" is a model that predicts users' preferences and emotional states through data analysis, and uses that information to create optimal tourism experiences and product recommendations.

[0401] "Emotional state" refers to the psychological state expressed by the user during the experience, and is analyzed from facial expressions, voice, and other factors.

[0402] "Purchase recommendations" are the process of recommending relevant products and services based on the user's emotional state.

[0403] This system consists of user-owned information terminals and a server. The information terminals are portable devices such as smartphones and smart glasses, equipped with sensors (cameras, microphones, etc.) to acquire the user's location information, history information, and emotional state. The information terminals transmit the acquired data to the server. The server has a generative AI model for analyzing the received data, which is used to generate optimal sightseeing routes and purchase suggestions that take the user's emotional state into consideration.

[0404] The server analyzes data using sentiment analysis libraries such as the Google Cloud Vision API and IBM Watson Tone Analyzer. For example, if a particular user is analyzed as being in a relaxed emotional state, the server will suggest tourist spots such as art museums or quiet cafes. At the same time, it will also present products and services suitable for that emotional state as purchase suggestions. The information terminal plays the role of guiding the user through the information fed back from the server.

[0405] For example, if a camera captures a user smiling while sightseeing, the system can determine that the user is enjoying themselves and suggest entertainment-related products and services. An example of a prompt message to the generating AI would be, "Analyze the user's smiling photo, and if it determines they are excited, recommend the latest entertainment products."

[0406] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0407] Step 1:

[0408] The user activates the information terminal and starts the tourist information application. The terminal uses its camera and microphone to capture the user's facial expressions and voice in real time. This collects information about the user's emotional state as input data.

[0409] Step 2:

[0410] The device transmits acquired emotional state information, location information, and user history information to the server. The server analyzes this input data using a generating AI model. It classifies emotional states using an emotional analysis library and narrows down potential tourist spots and purchase suggestions based on the results.

[0411] Step 3:

[0412] Based on the analysis results it generates, the server produces sightseeing routes and purchase suggestions best suited to the user's emotional state. For example, it selects active tourist spots for a user in an excited state. As output, user-optimized information is generated, and feedback is prepared.

[0413] Step 4:

[0414] The server sends optimized sightseeing routes and shopping suggestions back to the information terminal. The terminal receives this information and presents it to the user as visual or auditory information. For example, it might display a map of the sightseeing route on the screen and provide shopping suggestions via voice guidance.

[0415] Step 5:

[0416] Users can enjoy sightseeing based on the information presented. If a new emotional state is detected, the process restarts from step 1, and guidance and suggestions are updated in real time.

[0417] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0418] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0419] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0420] [Third Embodiment]

[0421] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0422] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0423] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0424] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0425] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0426] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0427] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0428] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0429] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0430] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0431] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0432] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0433] The system of this invention is implemented by combining various technologies to optimize the tourist experience for users.

[0434] The system configuration involves a mobile device carried by the user, with a tourism application installed. This device continuously collects location and historical information and transmits it to the server. Location information is obtained in real time using GPS functionality, and historical information includes past visit data.

[0435] The server receives the transmitted data and uses a generative model for analysis. The generative model generates the most suitable travel route for the user's preferences, referencing local tourist attractions and event information stored in the database. The generated route data consists of detailed guide information, including the order of tourist attractions and recommended visiting times.

[0436] This information is sent to the device and displayed to the user in an accessible format. Specifically, tourist routes are visualized on a map, and detailed information, local products, and cultural information for each location are shown.

[0437] Users follow the instructions on the device, and upon reaching a stamp point, the system confirms their location and prompts them to collect a stamp. After collecting the stamp, the device provides additional entertainment content, including location-related entertainment elements such as quiz games and videos about the history of the visited area.

[0438] Furthermore, the server records this activity history and uses it to generate more customized information for subsequent visits. This allows users to enjoy a more engaging and personalized sightseeing experience over the long term.

[0439] For example, if a user enjoys historical sites, the system will create a route based on their history, focusing on historical locations they haven't visited yet. Furthermore, mini-quizzes related to history are provided at each destination, deepening their knowledge and enhancing the experience. In this way, the system can individually optimize sightseeing according to each user's interests, allowing them to fully experience the charm of the region.

[0440] The following describes the processing flow.

[0441] Step 1:

[0442] The user launches the device with the tourism application installed and allows location sharing.

[0443] Step 2:

[0444] The device uses GPS to obtain real-time location information and sends it to the server along with the user's history information.

[0445] Step 3:

[0446] The server stores the received location and history information in a database. Based on this information, a generative model is used to analyze the user's interests and preferences.

[0447] Step 4:

[0448] Based on the analysis results, the server determines recommended tourist spots for the user. This allows it to calculate the optimal sightseeing route and create guide information.

[0449] Step 5:

[0450] The server sends guidance information to the terminal. This guidance information includes details of the sightseeing route and an overview of the places to visit.

[0451] Step 6:

[0452] The terminal displays the received guidance information to the user. It visually presents tourist routes and information about each spot on a map to guide the user.

[0453] Step 7:

[0454] Users follow the instructions on their device to navigate the sightseeing route and visit each stamp point.

[0455] Step 8:

[0456] The device detects when the user arrives at a stamp point and notifies them that they can obtain a stamp.

[0457] Step 9:

[0458] Users operate their devices to collect stamps and enjoy additional entertainment content (quizzes, mini-games, etc.).

[0459] Step 10:

[0460] The server then saves the acquired stamps and user activity history to prepare an optimized visit plan for the next visit.

[0461] (Example 1)

[0462] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0463] When providing tourism experiences, there is a lack of systems that generate optimal travel routes based on individual users' preferences and behavioral history, and furthermore, provide entertainment content to enrich the experience. The present invention aims to solve these problems and provide users with personalized tourism experiences.

[0464] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0465] In this invention, the server includes an information processing device means for acquiring the user's location information and history information, a computing device means for generating an optimal visit route for the user using a generation AI model based on the acquired location information and history information, and an information processing device means for providing guidance information to the user based on the generated visit route. This makes it possible to generate personalized visit routes according to the user's preferences and behavioral history, and to provide relevant entertainment content.

[0466] An "information processing device" is a device that acquires the user's location information and history information and transmits this information to a server.

[0467] A "generative AI model" is an algorithm that uses acquired user data to generate the optimal route for visits.

[0468] A "computational device" is a device that uses a generative AI model to calculate the optimal route for a user to visit.

[0469] "Guidance information" refers to the order of visits and information about each spot presented to the user based on the generated visit route.

[0470] A "digital certificate" is a digital certificate or piece of information that can be obtained by a user performing a specified action.

[0471] "Entertainment content" refers to information with entertainment elements, such as games and videos, that is provided to users when they obtain a digital certificate.

[0472] A description of embodiments for carrying out this invention will be given.

[0473] First, the terminal is a mobile device carried by the user, and a tourism application is installed on this device. The terminal uses GPS functionality to acquire the user's location information in real time and also records tourism history data. This information is continuously transmitted to the server.

[0474] The server uses a generative AI model to analyze location and historical information received from the terminal. This generative AI model refers to information on tourist spots and events stored in a database and is used to generate the most suitable sightseeing route for the user. For example, the generative AI model is given instructions such as "Based on the user's past visit history, suggest the next tourist spot to visit" as a prompt to perform the analysis.

[0475] The generated tourist route data includes the order of visits and recommended visiting times, which are compiled into detailed guide information. This guide information is transmitted to the terminal, which visualizes the tourist route on a map and displays it visually to the user. Information on local products and cultural background at each tourist spot is also presented.

[0476] Users navigate tourist destinations following the instructions on their device. Upon reaching specific points along the sightseeing route, known as "stamp points," the device prompts them to collect a stamp. This allows them to confirm their location and collect the stamp digitally. Furthermore, after collecting the stamp, the device offers additional entertainment content such as quiz games and videos about the visited locations, aiming to enhance the user's knowledge and enrich their entertainment experience.

[0477] For example, if a user enjoys historical sites, the system will suggest a sightseeing route based on unvisited historical spots that match their preferences. Through history quizzes at the visited locations, users can deepen their historical knowledge in an enjoyable way. In this way, the sightseeing experience is optimized according to the individual user's interests, making it possible to fully experience the charm of the region.

[0478] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0479] Step 1:

[0480] The device activates its GPS function to acquire current location information as soon as the user starts moving. This information is acquired at pre-set time intervals. The input is real-time location information, and the output is a data packet containing this location information. This data packet is sent to the server.

[0481] Step 2:

[0482] The device also collects past visit history information. This history information includes past visited locations and visit times retrieved from a history database. The input is history information extracted from the history database, and the output is comprehensive user data integrated with current location information. This integrated data is sent to the server.

[0483] Step 3:

[0484] The server receives comprehensive user data sent from the terminal. The input is current location and history data sent from the terminal, which is analyzed to understand the user's preferences and current state. Using a generative AI model, it receives a prompt message—for example, "Based on the user's preferences, please suggest the next tourist spot to visit"—and generates a suitable route. The output is this optimal route data.

[0485] Step 4:

[0486] The server sends the generated visit route data to the terminal. The input is the visit route data, and the output is the guidance information delivered to the terminal. The guidance information includes the order of visits and the recommended time to stay at each spot.

[0487] Step 5:

[0488] The terminal provides the user with received guidance information and visualizes the sightseeing route on a map. The input is guidance information from the server, and the output is a visualized sightseeing map and point-of-interest information. The user navigates based on the map.

[0489] Step 6:

[0490] The user visits tourist spots based on the terminal's directions. Upon reaching a stamp point, the terminal prompts the user to obtain a digital certificate. The input is the user's location information and the matching of the stamp point, and the output is the obtained digital certificate.

[0491] Step 7:

[0492] After obtaining a stamp, the terminal provides users with additional entertainment content. Input is the acquired digital certificate information, and output is entertainment content such as related quizzes and videos. This allows users to enjoy their sightseeing even more.

[0493] (Application Example 1)

[0494] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0495] Modern tourism experiences typically involve physical travel and are subject to time and cost constraints. Furthermore, even upon reaching a destination, providing personalized information and experiences tailored to each user is challenging, often resulting in standardized services. In particular, users living in remote locations face limitations in deeply experiencing foreign cultures and histories. Therefore, there is a need to develop systems that allow users to enjoy individually customized tourism experiences through virtual spaces.

[0496] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0497] In this invention, the server includes a device means for acquiring user coordinate information and history information, a computing device means for generating an optimal spatial path to the user using a generative model based on the acquired coordinate information and history information, and means for providing three-dimensional spatial information to the user through a visual device. This makes it possible for users to enjoy a individually tailored sightseeing experience in a virtual space while staying at home.

[0498] "Users" refers to individuals who seek to enjoy a tourism experience in a virtual space.

[0499] "Coordinate information" refers to geographical data used to indicate a user's current location, and is obtained using technologies such as GPS.

[0500] "History information" refers to records of places a user has visited and experiences they have had in the past, and is data used to provide personalized experiences.

[0501] "Device" refers to hardware or its components designed to perform a specific function.

[0502] A "generative model" is an algorithm that uses techniques such as machine learning to generate new outputs based on input data.

[0503] A "spatial route" refers to a travel path that a user can follow within a virtual space, and includes tourist information related to that route.

[0504] The term "arithmetic unit" refers to a computer or its hardware configuration that performs calculations to make decisions or predictions based on input data.

[0505] A "virtual stamp" is a digital proof that a user has reached a specific location through virtual tourism.

[0506] "Visual devices" refer to hardware used to present virtual reality or augmented reality to users, and include smart glasses and head-mounted displays.

[0507] In the system that realizes this invention, first, a terminal collects the user's coordinate information and history information. A mobile device such as a smartphone utilizing GPS functionality is used for this purpose. The terminal's role is to transmit the collected data to a server. Upon receiving this data, the server uses a generative AI model to generate an optimal spatial route based on the user's preferences and past visit history. In this process, user-specific data is analyzed using a machine learning framework such as TensorFlow.

[0508] The generated spatial paths are calculated by a computing device and then provided to the user via a terminal. Three-dimensional virtual space information is displayed to the user wearing a visual device such as smart glasses or a head-mounted display. This allows users to virtually experience foreign lands and cultures without physically traveling. The display on the visual device is built using a platform such as Unity.

[0509] Furthermore, during the process of users acquiring virtual stamps, the system detects the user's movements and recognizes their arrival at a specific location. Upon confirmation of arrival, the server provides additional entertainment content, such as quizzes about the history and culture of the visited virtual location. This allows users to have an experience that goes beyond mere sightseeing and deepens their knowledge.

[0510] As a concrete example, if a user wishes to deepen their knowledge of Florence, Italy, the system will create a virtual tour focusing on Renaissance landmarks based on the user's interests. The user will visit famous museums and historical buildings in the virtual space and take related quizzes. An example of a prompt to the generative AI model used in this process would be: "Create a virtual tour route themed on Renaissance landmarks in Florence and provide an interactive quiz that includes relevant historical background information."

[0511] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0512] Step 1:

[0513] The device acquires the user's coordinate information. Using GPS functionality, it obtains the user's location in real time and records this information on the device. The output is the user's current coordinate data.

[0514] Step 2:

[0515] The device collects user history information. It extracts previously visited tourist spots and user activity history from a database and constructs history information based on this. The output provides information about the user's interests and past visit history.

[0516] Step 3:

[0517] The terminal transmits acquired coordinate and historical information to the server. This input data serves as foundational data to facilitate further analysis on the server.

[0518] Step 4:

[0519] The server processes the user's coordinate and history information using a generative AI model. It utilizes a machine learning framework (such as TensorFlow) for data analysis to calculate the optimal sightseeing route based on the input data. The output is a customized spatial route tailored to the user's interests.

[0520] Step 5:

[0521] The server sends the generated spatial route data to the terminal. This route includes related tourist spots and associated information.

[0522] Step 6:

[0523] The device uses visual devices to display three-dimensional spatial information to the user. Leveraging the Unity platform, it visually represents tourist information and routes. This allows users to begin a visual sightseeing experience within a virtual space.

[0524] Step 7:

[0525] The user explores a virtual space, and the server detects the user's location and movements. When the user reaches a specific tourist spot, the terminal recognizes their arrival and prompts them to obtain a virtual marker.

[0526] Step 8:

[0527] Once the server confirms that a virtual stamp has been obtained, it provides additional entertainment content. For example, it might generate a quiz about the relevant history or culture. This would use the prompt message, "If a virtual stamp has been obtained, please provide an interactive quiz that reflects the corresponding historical information."

[0528] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0529] This invention provides a system for personalizing the tourism experience by taking into account the user's emotional state. This system is implemented by combining the acquisition of standard location and historical information with an emotion engine.

[0530] The terminal is a mobile device that users carry with them daily, and it has a tourism application installed. This terminal provides tourist information based on the collection of the user's location and history information. In addition, the terminal uses sensors such as a camera and microphone to detect the user's emotional state (facial expressions and voice) and transmits the acquired data to a server. This enables real-time sentiment analysis.

[0531] The server analyzes received location information, historical information, and sentiment data. Using a generative model, it generates optimized sightseeing routes based on each user's preferences and emotions at the time. Through the sentiment engine, it enables dynamic sightseeing guidance that takes into account the user's current emotional state.

[0532] For example, if the user is perceived as active, the system will suggest routes that emphasize sports and activity-related tourist spots. On the other hand, if the user is judged to be relaxed, it will recommend quiet and calming spots such as art museums or cafes. In this way, the device provides guidance information accordingly and continuously monitors the user's emotions in real time.

[0533] Furthermore, when a user reaches a stamp point and obtains a stamp, the system presents entertainment content tailored to the individual user's emotions based on feedback from the emotion engine. For example, if a user is excited, it provides an active game; conversely, if they are calm, it delivers an informative documentary video, thereby improving user satisfaction.

[0534] This system allows tourism experiences to be combined with the user's temporary emotional state, providing individually optimized routes and content, thereby improving the quality of tourism.

[0535] The following describes the processing flow.

[0536] Step 1:

[0537] Users install a tourism application on their device and allow the collection of location and sentiment data.

[0538] Step 2:

[0539] The device obtains the user's location information using GPS and sends it to the server along with the user's current history data.

[0540] Step 3:

[0541] The device uses its built-in camera and microphone to periodically collect the user's facial expressions and voice, and sends this data to a server as emotion data.

[0542] Step 4:

[0543] The server integrates received location information, historical information, and sentiment data, and uses a generative model to analyze the user's interests and current sentiment state.

[0544] Step 5:

[0545] Based on the analysis results, the server calculates the most suitable sightseeing route for the user and generates real-time, emotion-responsive guidance information.

[0546] Step 6:

[0547] The server sends the generated information to the terminal, including specific tourist spots and recommended activities.

[0548] Step 7:

[0549] The terminal displays navigation information to the user. Depending on the user's status, it presents an interactive map and detailed information.

[0550] Step 8:

[0551] Users follow the on-screen guide to travel along the sightseeing route and visit various stamp points.

[0552] Step 9:

[0553] The device detects when the user reaches a stamp point and, along with obtaining the stamp, provides entertainment content tailored to the user's emotions.

[0554] Step 10:

[0555] The server stores the user's behavior history, acquired stamp data, and sentiment feedback, and uses this information for further customization during subsequent visits.

[0556] (Example 2)

[0557] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0558] Conventional tourist information systems only provide sightseeing routes based on the user's location and history information, and have the limitation of not being able to personalize the guidance by taking into account the user's emotional state. As a result, it was difficult to provide a sightseeing experience that was more suited to the user's temporary emotional state, and there was room to improve the quality and satisfaction of sightseeing.

[0559] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0560] In this invention, the server includes terminal means for sensing and acquiring the user's emotional state in addition to the user's location information and history information; information processing means for generating an optimal sightseeing route that takes the user's emotional state into consideration using a generative model based on the acquired location information, history information, and emotional data; and terminal means for providing dynamic guidance information to the user based on the generated sightseeing route. This makes it possible to provide sightseeing routes and entertainment content optimized for the user's current emotional state.

[0561] "User location information" refers to geographical data that indicates the user's current location.

[0562] "History information" refers to data based on records of places and activities that users have visited in the past.

[0563] "Emotional state" refers to data that represents the user's psychological state, and is obtained from facial expressions and voice.

[0564] A "terminal" is a mobile device carried by a user, which is a device that collects and transmits various types of data.

[0565] A "generative model" is an algorithm and program used to derive personalized conclusions based on various data.

[0566] An "information processing device" is a machine or system that analyzes received data and generates and distributes useful information.

[0567] "Dynamic guidance information" refers to guide information that changes based on the user's current and past data.

[0568] "Entertainment content" refers to digital resources, including entertainment, that are provided to suit the user's emotions and preferences.

[0569] A "stamp point" is a digital mark that indicates a user has achieved a specific action or location.

[0570] This invention relates to a system that personalizes the tourist experience by taking into account the emotional state of the user.

[0571] The terminal is a personal information terminal that users carry with them daily, and it has a tourism application installed to implement this system. This application collects the user's location and history information using GPS and a history database. The terminal also has a built-in camera (image sensor) and microphone (audio sensor), which are used to analyze the user's facial expressions and voice in real time and acquire their emotional state. The data obtained in this way is transmitted to a server via the internet.

[0572] The server is an information processing device located in the cloud that integrates received location information, historical information, and emotional data, and analyzes them using a generative AI model. The server utilizes this generative model to generate optimized sightseeing routes based on the user's individual emotional state and preferences. The generated sightseeing routes are dynamically adjusted by the emotional engine and provided to the terminal in real time. For example, if the user is active, a route including sightseeing spots that emphasize sports and activities will be suggested, taking this into account.

[0573] Furthermore, when a user obtains a stamp at a location they visit, the device receives feedback from the server, suggesting entertainment content based on the user's emotional state (e.g., active games or quiet documentary videos) to further engage the user.

[0574] For example, if a user sightseeing in Kyoto enjoys a lively festival in the morning, the server might determine that the user is looking for an active experience in the afternoon and suggest a route that includes a geisha experience in Gion or nearby activity facilities. In response, an example of a prompt message entered into the system might be, "The user is currently active. Please generate a recommended afternoon sightseeing route in Kyoto. Avoid quiet spots and prioritize energetic experiences."

[0575] In this way, this invention dynamically provides a tourism experience tailored to the emotional state of the user, thereby improving the quality and satisfaction of tourism.

[0576] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0577] Step 1:

[0578] The device is the user's everyday mobile device, and first, it obtains the user's current location using location services. In parallel, it retrieves the user's past movement history from a history database. This uses GPS signals, network data, and data from user interactions within the application. The retrieved data is temporarily stored on the device and prepared for subsequent processing.

[0579] Step 2:

[0580] The device uses its built-in camera and microphone to capture the user's current facial expressions and voice. By analyzing this sensor data, the device determines the user's emotional state. For example, an AI-based algorithm analyzes changes in voice tone and facial expressions in real time to generate emotional data such as "excited" or "relaxed." This generated emotional data, along with location and historical information, is packetized for use in later steps.

[0581] Step 3:

[0582] The device transmits the location information, history information, and sentiment data collected in steps 1 and 2 to the server via the internet connection. The data is encrypted to protect user privacy. After transmitting the data, the device waits for a response from the server.

[0583] Step 4:

[0584] The server receives the data, performs initial processing, and then analyzes this data using a generative AI model. Specifically, it identifies the current location from location information, extracts past behavioral patterns from historical information, and understands the current emotional state from sentiment data. Using this data, the server generates the optimal sightseeing route based on prompts. The generative model, based on its learned algorithms, selects the most appropriate personalized route from multiple candidate routes.

[0585] Step 5:

[0586] The server sends generated tourist routes and related information to the device. This includes dynamically adjusted tourist information and recommended entertainment content. This provides users with real-time, personalized guidance.

[0587] Step 6:

[0588] Users can view sightseeing suggestions and entertainment content received from the server via their devices and use them to help plan their activities at their destination. For example, they can enjoy sightseeing while viewing recommended routes and spots, and enrich their travel experience by using content that suits their emotional state.

[0589] Step 7:

[0590] The device continuously monitors emotional data while the user continues sightseeing, sending new data to the server whenever it becomes available. This continuous data provision and analysis ensures that the sightseeing experience is always up-to-date and that the user is continuously provided with information optimized for them.

[0591] (Application Example 2)

[0592] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0593] Current tourist information systems struggle to provide personalized experiences that take into account the emotional state of users. Furthermore, existing systems are unable to implement dynamic purchase suggestions based on emotions, resulting in limited suggestions for products and services that are best suited to the user.

[0594] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0595] In this invention, the server includes information terminal means for acquiring the user's location information, history information, and emotional state; computing device means for generating an optimal route according to the user's emotional state using a generative model based on the acquired data; and information terminal means for dynamically customizing guidance information and purchase suggestions based on the emotional state. This enables a personalized sightseeing experience and optimal product suggestions based on the user's real-time emotional state.

[0596] An "information terminal" is a portable device used to acquire a user's location information, history information, and emotional state, and to provide tourist information and purchase suggestions.

[0597] A "computational device" is a device that uses a generative model based on acquired data to dynamically generate paths and suggestions that correspond to the user's emotional state.

[0598] A "generative model" is a model that predicts users' preferences and emotional states through data analysis, and uses that information to create optimal tourism experiences and product recommendations.

[0599] "Emotional state" refers to the psychological state expressed by the user during the experience, and is analyzed from facial expressions, voice, and other factors.

[0600] "Purchase recommendations" are the process of recommending relevant products and services based on the user's emotional state.

[0601] This system consists of user-owned information terminals and a server. The information terminals are portable devices such as smartphones and smart glasses, equipped with sensors (cameras, microphones, etc.) to acquire the user's location information, history information, and emotional state. The information terminals transmit the acquired data to the server. The server has a generative AI model for analyzing the received data, which is used to generate optimal sightseeing routes and purchase suggestions that take the user's emotional state into consideration.

[0602] The server analyzes data using sentiment analysis libraries such as the Google Cloud Vision API and IBM Watson Tone Analyzer. For example, if a particular user is analyzed as being in a relaxed emotional state, the server will suggest tourist spots such as art museums or quiet cafes. At the same time, it will also present products and services suitable for that emotional state as purchase suggestions. The information terminal plays the role of guiding the user through the information fed back from the server.

[0603] For example, if a camera captures a user smiling while sightseeing, the system can determine that the user is enjoying themselves and suggest entertainment-related products and services. An example of a prompt message to the generating AI would be, "Analyze the user's smiling photo, and if it determines they are excited, recommend the latest entertainment products."

[0604] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0605] Step 1:

[0606] The user activates the information terminal and starts the tourist information application. The terminal uses its camera and microphone to capture the user's facial expressions and voice in real time. This collects information about the user's emotional state as input data.

[0607] Step 2:

[0608] The device transmits acquired emotional state information, location information, and user history information to the server. The server analyzes this input data using a generating AI model. It classifies emotional states using an emotional analysis library and narrows down potential tourist spots and purchase suggestions based on the results.

[0609] Step 3:

[0610] Based on the analysis results it generates, the server produces sightseeing routes and purchase suggestions best suited to the user's emotional state. For example, it selects active tourist spots for a user in an excited state. As output, user-optimized information is generated, and feedback is prepared.

[0611] Step 4:

[0612] The server sends optimized sightseeing routes and shopping suggestions back to the information terminal. The terminal receives this information and presents it to the user as visual or auditory information. For example, it might display a map of the sightseeing route on the screen and provide shopping suggestions via voice guidance.

[0613] Step 5:

[0614] Users can enjoy sightseeing based on the information presented. If a new emotional state is detected, the process restarts from step 1, and guidance and suggestions are updated in real time.

[0615] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0616] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0617] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0618] [Fourth Embodiment]

[0619] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0620] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0621] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0622] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0623] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0624] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0625] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0626] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0627] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0628] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0629] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0630] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0631] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0632] The system of this invention is implemented by combining various technologies to optimize the tourist experience for users.

[0633] The system configuration involves a mobile device carried by the user, with a tourism application installed. This device continuously collects location and historical information and transmits it to the server. Location information is obtained in real time using GPS functionality, and historical information includes past visit data.

[0634] The server receives the transmitted data and uses a generative model for analysis. The generative model generates the most suitable travel route for the user's preferences, referencing local tourist attractions and event information stored in the database. The generated route data consists of detailed guide information, including the order of tourist attractions and recommended visiting times.

[0635] This information is sent to the device and displayed to the user in an accessible format. Specifically, tourist routes are visualized on a map, and detailed information, local products, and cultural information for each location are shown.

[0636] Users follow the instructions on the device, and upon reaching a stamp point, the system confirms their location and prompts them to collect a stamp. After collecting the stamp, the device provides additional entertainment content, including location-related entertainment elements such as quiz games and videos about the history of the visited area.

[0637] Furthermore, the server records this activity history and uses it to generate more customized information for subsequent visits. This allows users to enjoy a more engaging and personalized sightseeing experience over the long term.

[0638] For example, if a user enjoys historical sites, the system will create a route based on their history, focusing on historical locations they haven't visited yet. Furthermore, mini-quizzes related to history are provided at each destination, deepening their knowledge and enhancing the experience. In this way, the system can individually optimize sightseeing according to each user's interests, allowing them to fully experience the charm of the region.

[0639] The following describes the processing flow.

[0640] Step 1:

[0641] The user launches the device with the tourism application installed and allows location sharing.

[0642] Step 2:

[0643] The device uses GPS to obtain real-time location information and sends it to the server along with the user's history information.

[0644] Step 3:

[0645] The server stores the received location and history information in a database. Based on this information, a generative model is used to analyze the user's interests and preferences.

[0646] Step 4:

[0647] Based on the analysis results, the server determines recommended tourist spots for the user. This allows it to calculate the optimal sightseeing route and create guide information.

[0648] Step 5:

[0649] The server sends guidance information to the terminal. This guidance information includes details of the sightseeing route and an overview of the places to visit.

[0650] Step 6:

[0651] The terminal displays the received guidance information to the user. It visually presents tourist routes and information about each spot on a map to guide the user.

[0652] Step 7:

[0653] Users follow the instructions on their device to navigate the sightseeing route and visit each stamp point.

[0654] Step 8:

[0655] The device detects when the user arrives at a stamp point and notifies them that they can obtain a stamp.

[0656] Step 9:

[0657] Users operate their devices to collect stamps and enjoy additional entertainment content (quizzes, mini-games, etc.).

[0658] Step 10:

[0659] The server then saves the acquired stamps and user activity history to prepare an optimized visit plan for the next visit.

[0660] (Example 1)

[0661] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0662] When providing tourism experiences, there is a lack of systems that generate optimal travel routes based on individual users' preferences and behavioral history, and furthermore, provide entertainment content to enrich the experience. The present invention aims to solve these problems and provide users with personalized tourism experiences.

[0663] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0664] In this invention, the server includes an information processing device means for acquiring the user's location information and history information, a computing device means for generating an optimal visit route for the user using a generation AI model based on the acquired location information and history information, and an information processing device means for providing guidance information to the user based on the generated visit route. This makes it possible to generate personalized visit routes according to the user's preferences and behavioral history, and to provide relevant entertainment content.

[0665] An "information processing device" is a device that acquires the user's location information and history information and transmits this information to a server.

[0666] A "generative AI model" is an algorithm that uses acquired user data to generate the optimal route for visits.

[0667] A "computational device" is a device that uses a generative AI model to calculate the optimal route for a user to visit.

[0668] "Guidance information" refers to the order of visits and information about each spot presented to the user based on the generated visit route.

[0669] A "digital certificate" is a digital certificate or piece of information that can be obtained by a user performing a specified action.

[0670] "Entertainment content" refers to information with entertainment elements, such as games and videos, that is provided to users when they obtain a digital certificate.

[0671] A description of embodiments for carrying out this invention will be given.

[0672] First, the terminal is a mobile device carried by the user, and a tourism application is installed on this device. The terminal uses GPS functionality to acquire the user's location information in real time and also records tourism history data. This information is continuously transmitted to the server.

[0673] The server uses a generative AI model to analyze location and historical information received from the terminal. This generative AI model refers to information on tourist spots and events stored in a database and is used to generate the most suitable sightseeing route for the user. For example, the generative AI model is given instructions such as "Based on the user's past visit history, suggest the next tourist spot to visit" as a prompt to perform the analysis.

[0674] The generated tourist route data includes the order of visits and recommended visiting times, which are compiled into detailed guide information. This guide information is transmitted to the terminal, which visualizes the tourist route on a map and displays it visually to the user. Information on local products and cultural background at each tourist spot is also presented.

[0675] Users navigate tourist destinations following the instructions on their device. Upon reaching specific points along the sightseeing route, known as "stamp points," the device prompts them to collect a stamp. This allows them to confirm their location and collect the stamp digitally. Furthermore, after collecting the stamp, the device offers additional entertainment content such as quiz games and videos about the visited locations, aiming to enhance the user's knowledge and enrich their entertainment experience.

[0676] For example, if a user enjoys historical sites, the system will suggest a sightseeing route based on unvisited historical spots that match their preferences. Through history quizzes at the visited locations, users can deepen their historical knowledge in an enjoyable way. In this way, the sightseeing experience is optimized according to the individual user's interests, making it possible to fully experience the charm of the region.

[0677] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0678] Step 1:

[0679] The device activates its GPS function to acquire current location information as soon as the user starts moving. This information is acquired at pre-set time intervals. The input is real-time location information, and the output is a data packet containing this location information. This data packet is sent to the server.

[0680] Step 2:

[0681] The device also collects past visit history information. This history information includes past visited locations and visit times retrieved from a history database. The input is history information extracted from the history database, and the output is comprehensive user data integrated with current location information. This integrated data is sent to the server.

[0682] Step 3:

[0683] The server receives comprehensive user data sent from the terminal. The input is current location and history data sent from the terminal, which is analyzed to understand the user's preferences and current state. Using a generative AI model, it receives a prompt message—for example, "Based on the user's preferences, please suggest the next tourist spot to visit"—and generates a suitable route. The output is this optimal route data.

[0684] Step 4:

[0685] The server sends the generated visit route data to the terminal. The input is the visit route data, and the output is the guidance information delivered to the terminal. The guidance information includes the order of visits and the recommended time to stay at each spot.

[0686] Step 5:

[0687] The terminal provides the user with received guidance information and visualizes the sightseeing route on a map. The input is guidance information from the server, and the output is a visualized sightseeing map and point-of-interest information. The user navigates based on the map.

[0688] Step 6:

[0689] The user visits tourist spots based on the terminal's directions. Upon reaching a stamp point, the terminal prompts the user to obtain a digital certificate. The input is the user's location information and the matching of the stamp point, and the output is the obtained digital certificate.

[0690] Step 7:

[0691] After obtaining a stamp, the terminal provides users with additional entertainment content. Input is the acquired digital certificate information, and output is entertainment content such as related quizzes and videos. This allows users to enjoy their sightseeing even more.

[0692] (Application Example 1)

[0693] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0694] Modern tourism experiences typically involve physical travel and are subject to time and cost constraints. Furthermore, even upon reaching a destination, providing personalized information and experiences tailored to each user is challenging, often resulting in standardized services. In particular, users living in remote locations face limitations in deeply experiencing foreign cultures and histories. Therefore, there is a need to develop systems that allow users to enjoy individually customized tourism experiences through virtual spaces.

[0695] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0696] In this invention, the server includes a device means for acquiring user coordinate information and history information, a computing device means for generating an optimal spatial path to the user using a generative model based on the acquired coordinate information and history information, and means for providing three-dimensional spatial information to the user through a visual device. This makes it possible for users to enjoy a individually tailored sightseeing experience in a virtual space while staying at home.

[0697] "Users" refers to individuals who seek to enjoy a tourism experience in a virtual space.

[0698] "Coordinate information" refers to geographical data used to indicate a user's current location, and is obtained using technologies such as GPS.

[0699] "History information" refers to records of places a user has visited and experiences they have had in the past, and is data used to provide personalized experiences.

[0700] "Device" refers to hardware or its components designed to perform a specific function.

[0701] A "generative model" is an algorithm that uses techniques such as machine learning to generate new outputs based on input data.

[0702] A "spatial route" refers to a travel path that a user can follow within a virtual space, and includes tourist information related to that route.

[0703] The term "arithmetic unit" refers to a computer or its hardware configuration that performs calculations to make decisions or predictions based on input data.

[0704] A "virtual stamp" is a digital proof that a user has reached a specific location through virtual tourism.

[0705] "Visual devices" refer to hardware used to present virtual reality or augmented reality to users, and include smart glasses and head-mounted displays.

[0706] In the system that realizes this invention, first, a terminal collects the user's coordinate information and history information. A mobile device such as a smartphone utilizing GPS functionality is used for this purpose. The terminal's role is to transmit the collected data to a server. Upon receiving this data, the server uses a generative AI model to generate an optimal spatial route based on the user's preferences and past visit history. In this process, user-specific data is analyzed using a machine learning framework such as TensorFlow.

[0707] The generated spatial paths are calculated by a computing device and then provided to the user via a terminal. Three-dimensional virtual space information is displayed to the user wearing a visual device such as smart glasses or a head-mounted display. This allows users to virtually experience foreign lands and cultures without physically traveling. The display on the visual device is built using a platform such as Unity.

[0708] Furthermore, during the process of users acquiring virtual stamps, the system detects the user's movements and recognizes their arrival at a specific location. Upon confirmation of arrival, the server provides additional entertainment content, such as quizzes about the history and culture of the visited virtual location. This allows users to have an experience that goes beyond mere sightseeing and deepens their knowledge.

[0709] As a concrete example, if a user wishes to deepen their knowledge of Florence, Italy, the system will create a virtual tour focusing on Renaissance landmarks based on the user's interests. The user will visit famous museums and historical buildings in the virtual space and take related quizzes. An example of a prompt to the generative AI model used in this process would be: "Create a virtual tour route themed on Renaissance landmarks in Florence and provide an interactive quiz that includes relevant historical background information."

[0710] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0711] Step 1:

[0712] The device acquires the user's coordinate information. Using GPS functionality, it obtains the user's location in real time and records this information on the device. The output is the user's current coordinate data.

[0713] Step 2:

[0714] The device collects user history information. It extracts previously visited tourist spots and user activity history from a database and constructs history information based on this. The output provides information about the user's interests and past visit history.

[0715] Step 3:

[0716] The terminal transmits acquired coordinate and historical information to the server. This input data serves as foundational data to facilitate further analysis on the server.

[0717] Step 4:

[0718] The server processes the user's coordinate and history information using a generative AI model. It utilizes a machine learning framework (such as TensorFlow) for data analysis to calculate the optimal sightseeing route based on the input data. The output is a customized spatial route tailored to the user's interests.

[0719] Step 5:

[0720] The server sends the generated spatial route data to the terminal. This route includes related tourist spots and associated information.

[0721] Step 6:

[0722] The device uses visual devices to display three-dimensional spatial information to the user. Leveraging the Unity platform, it visually represents tourist information and routes. This allows users to begin a visual sightseeing experience within a virtual space.

[0723] Step 7:

[0724] The user explores a virtual space, and the server detects the user's location and movements. When the user reaches a specific tourist spot, the terminal recognizes their arrival and prompts them to obtain a virtual marker.

[0725] Step 8:

[0726] Once the server confirms that a virtual stamp has been obtained, it provides additional entertainment content. For example, it might generate a quiz about the relevant history or culture. This would use the prompt message, "If a virtual stamp has been obtained, please provide an interactive quiz that reflects the corresponding historical information."

[0727] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0728] This invention provides a system for personalizing the tourism experience by taking into account the user's emotional state. This system is implemented by combining the acquisition of standard location and historical information with an emotion engine.

[0729] The terminal is a mobile device that users carry with them daily, and it has a tourism application installed. This terminal provides tourist information based on the collection of the user's location and history information. In addition, the terminal uses sensors such as a camera and microphone to detect the user's emotional state (facial expressions and voice) and transmits the acquired data to a server. This enables real-time sentiment analysis.

[0730] The server analyzes received location information, historical information, and sentiment data. Using a generative model, it generates optimized sightseeing routes based on each user's preferences and emotions at the time. Through the sentiment engine, it enables dynamic sightseeing guidance that takes into account the user's current emotional state.

[0731] For example, if the user is perceived as active, the system will suggest routes that emphasize sports and activity-related tourist spots. On the other hand, if the user is judged to be relaxed, it will recommend quiet and calming spots such as art museums or cafes. In this way, the device provides guidance information accordingly and continuously monitors the user's emotions in real time.

[0732] Furthermore, when a user reaches a stamp point and obtains a stamp, the system presents entertainment content tailored to the individual user's emotions based on feedback from the emotion engine. For example, if a user is excited, it provides an active game; conversely, if they are calm, it delivers an informative documentary video, thereby improving user satisfaction.

[0733] This system allows tourism experiences to be combined with the user's temporary emotional state, providing individually optimized routes and content, thereby improving the quality of tourism.

[0734] The following describes the processing flow.

[0735] Step 1:

[0736] Users install a tourism application on their device and allow the collection of location and sentiment data.

[0737] Step 2:

[0738] The device obtains the user's location information using GPS and sends it to the server along with the user's current history data.

[0739] Step 3:

[0740] The device uses its built-in camera and microphone to periodically collect the user's facial expressions and voice, and sends this data to a server as emotion data.

[0741] Step 4:

[0742] The server integrates received location information, historical information, and sentiment data, and uses a generative model to analyze the user's interests and current sentiment state.

[0743] Step 5:

[0744] Based on the analysis results, the server calculates the most suitable sightseeing route for the user and generates real-time, emotion-responsive guidance information.

[0745] Step 6:

[0746] The server sends the generated information to the terminal, including specific tourist spots and recommended activities.

[0747] Step 7:

[0748] The terminal displays navigation information to the user. Depending on the user's status, it presents an interactive map and detailed information.

[0749] Step 8:

[0750] Users follow the on-screen guide to travel along the sightseeing route and visit various stamp points.

[0751] Step 9:

[0752] The device detects when the user reaches a stamp point and, along with obtaining the stamp, provides entertainment content tailored to the user's emotions.

[0753] Step 10:

[0754] The server stores the user's behavior history, acquired stamp data, and sentiment feedback, and uses this information for further customization during subsequent visits.

[0755] (Example 2)

[0756] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0757] Conventional tourist information systems only provide sightseeing routes based on the user's location and history information, and have the limitation of not being able to personalize the guidance by taking into account the user's emotional state. As a result, it was difficult to provide a sightseeing experience that was more suited to the user's temporary emotional state, and there was room to improve the quality and satisfaction of sightseeing.

[0758] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0759] In this invention, the server includes terminal means for sensing and acquiring the user's emotional state in addition to the user's location information and history information; information processing means for generating an optimal sightseeing route that takes the user's emotional state into consideration using a generative model based on the acquired location information, history information, and emotional data; and terminal means for providing dynamic guidance information to the user based on the generated sightseeing route. This makes it possible to provide sightseeing routes and entertainment content optimized for the user's current emotional state.

[0760] "User location information" refers to geographical data that indicates the user's current location.

[0761] "History information" refers to data based on records of places and activities that users have visited in the past.

[0762] "Emotional state" refers to data that represents the user's psychological state, and is obtained from facial expressions and voice.

[0763] A "terminal" is a mobile device carried by a user, which is a device that collects and transmits various types of data.

[0764] A "generative model" is an algorithm and program used to derive personalized conclusions based on various data.

[0765] An "information processing device" is a machine or system that analyzes received data and generates and distributes useful information.

[0766] "Dynamic guidance information" refers to guide information that changes based on the user's current and past data.

[0767] "Entertainment content" refers to digital resources, including entertainment, that are provided to suit the user's emotions and preferences.

[0768] A "stamp point" is a digital mark that indicates a user has achieved a specific action or location.

[0769] This invention relates to a system that personalizes the tourist experience by taking into account the emotional state of the user.

[0770] The terminal is a personal information terminal that users carry with them daily, and it has a tourism application installed to implement this system. This application collects the user's location and history information using GPS and a history database. The terminal also has a built-in camera (image sensor) and microphone (audio sensor), which are used to analyze the user's facial expressions and voice in real time and acquire their emotional state. The data obtained in this way is transmitted to a server via the internet.

[0771] The server is an information processing device located in the cloud that integrates received location information, historical information, and emotional data, and analyzes them using a generative AI model. The server utilizes this generative model to generate optimized sightseeing routes based on the user's individual emotional state and preferences. The generated sightseeing routes are dynamically adjusted by the emotional engine and provided to the terminal in real time. For example, if the user is active, a route including sightseeing spots that emphasize sports and activities will be suggested, taking this into account.

[0772] Furthermore, when a user obtains a stamp at a location they visit, the device receives feedback from the server, suggesting entertainment content based on the user's emotional state (e.g., active games or quiet documentary videos) to further engage the user.

[0773] For example, if a user sightseeing in Kyoto enjoys a lively festival in the morning, the server might determine that the user is looking for an active experience in the afternoon and suggest a route that includes a geisha experience in Gion or nearby activity facilities. In response, an example of a prompt message entered into the system might be, "The user is currently active. Please generate a recommended afternoon sightseeing route in Kyoto. Avoid quiet spots and prioritize energetic experiences."

[0774] In this way, this invention dynamically provides a tourism experience tailored to the emotional state of the user, thereby improving the quality and satisfaction of tourism.

[0775] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0776] Step 1:

[0777] The device is the user's everyday mobile device, and first, it obtains the user's current location using location services. In parallel, it retrieves the user's past movement history from a history database. This uses GPS signals, network data, and data from user interactions within the application. The retrieved data is temporarily stored on the device and prepared for subsequent processing.

[0778] Step 2:

[0779] The device uses its built-in camera and microphone to capture the user's current facial expressions and voice. By analyzing this sensor data, the device determines the user's emotional state. For example, an AI-based algorithm analyzes changes in voice tone and facial expressions in real time to generate emotional data such as "excited" or "relaxed." This generated emotional data, along with location and historical information, is packetized for use in later steps.

[0780] Step 3:

[0781] The device transmits the location information, history information, and sentiment data collected in steps 1 and 2 to the server via the internet connection. The data is encrypted to protect user privacy. After transmitting the data, the device waits for a response from the server.

[0782] Step 4:

[0783] The server receives the data, performs initial processing, and then analyzes this data using a generative AI model. Specifically, it identifies the current location from location information, extracts past behavioral patterns from historical information, and understands the current emotional state from sentiment data. Using this data, the server generates the optimal sightseeing route based on prompts. The generative model, based on its learned algorithms, selects the most appropriate personalized route from multiple candidate routes.

[0784] Step 5:

[0785] The server sends generated tourist routes and related information to the device. This includes dynamically adjusted tourist information and recommended entertainment content. This provides users with real-time, personalized guidance.

[0786] Step 6:

[0787] Users can view sightseeing suggestions and entertainment content received from the server via their devices and use them to help plan their activities at their destination. For example, they can enjoy sightseeing while viewing recommended routes and spots, and enrich their travel experience by using content that suits their emotional state.

[0788] Step 7:

[0789] The device continuously monitors emotional data while the user continues sightseeing, sending new data to the server whenever it becomes available. This continuous data provision and analysis ensures that the sightseeing experience is always up-to-date and that the user is continuously provided with information optimized for them.

[0790] (Application Example 2)

[0791] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0792] Current tourist information systems struggle to provide personalized experiences that take into account the emotional state of users. Furthermore, existing systems are unable to implement dynamic purchase suggestions based on emotions, resulting in limited suggestions for products and services that are best suited to the user.

[0793] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0794] In this invention, the server includes information terminal means for acquiring the user's location information, history information, and emotional state; computing device means for generating an optimal route according to the user's emotional state using a generative model based on the acquired data; and information terminal means for dynamically customizing guidance information and purchase suggestions based on the emotional state. This enables a personalized sightseeing experience and optimal product suggestions based on the user's real-time emotional state.

[0795] An "information terminal" is a portable device used to acquire a user's location information, history information, and emotional state, and to provide tourist information and purchase suggestions.

[0796] A "computational device" is a device that uses a generative model based on acquired data to dynamically generate paths and suggestions that correspond to the user's emotional state.

[0797] A "generative model" is a model that predicts users' preferences and emotional states through data analysis, and uses that information to create optimal tourism experiences and product recommendations.

[0798] "Emotional state" refers to the psychological state expressed by the user during the experience, and is analyzed from facial expressions, voice, and other factors.

[0799] "Purchase recommendations" are the process of recommending relevant products and services based on the user's emotional state.

[0800] This system consists of user-owned information terminals and a server. The information terminals are portable devices such as smartphones and smart glasses, equipped with sensors (cameras, microphones, etc.) to acquire the user's location information, history information, and emotional state. The information terminals transmit the acquired data to the server. The server has a generative AI model for analyzing the received data, which is used to generate optimal sightseeing routes and purchase suggestions that take the user's emotional state into consideration.

[0801] The server analyzes data using sentiment analysis libraries such as the Google Cloud Vision API and IBM Watson Tone Analyzer. For example, if a particular user is analyzed as being in a relaxed emotional state, the server will suggest tourist spots such as art museums or quiet cafes. At the same time, it will also present products and services suitable for that emotional state as purchase suggestions. The information terminal plays the role of guiding the user through the information fed back from the server.

[0802] For example, if a camera captures a user smiling while sightseeing, the system can determine that the user is enjoying themselves and suggest entertainment-related products and services. An example of a prompt message to the generating AI would be, "Analyze the user's smiling photo, and if it determines they are excited, recommend the latest entertainment products."

[0803] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0804] Step 1:

[0805] The user activates the information terminal and starts the tourist information application. The terminal uses its camera and microphone to capture the user's facial expressions and voice in real time. This collects information about the user's emotional state as input data.

[0806] Step 2:

[0807] The device transmits acquired emotional state information, location information, and user history information to the server. The server analyzes this input data using a generating AI model. It classifies emotional states using an emotional analysis library and narrows down potential tourist spots and purchase suggestions based on the results.

[0808] Step 3:

[0809] Based on the analysis results it generates, the server produces sightseeing routes and purchase suggestions best suited to the user's emotional state. For example, it selects active tourist spots for a user in an excited state. As output, user-optimized information is generated, and feedback is prepared.

[0810] Step 4:

[0811] The server sends optimized sightseeing routes and shopping suggestions back to the information terminal. The terminal receives this information and presents it to the user as visual or auditory information. For example, it might display a map of the sightseeing route on the screen and provide shopping suggestions via voice guidance.

[0812] Step 5:

[0813] Users can enjoy sightseeing based on the information presented. If a new emotional state is detected, the process restarts from step 1, and guidance and suggestions are updated in real time.

[0814] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0815] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0816] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0817] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0818] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0819] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0820] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0821] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0822] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0823] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0824] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0825] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0826] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0827] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0828] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0829] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0830] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0831] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0832] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0833] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0834] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0835] The following is further disclosed regarding the embodiments described above.

[0836] (Claim 1)

[0837] A terminal means for acquiring the user's location information and history information,

[0838] A computing device means that generates the optimal sightseeing route for the user using a generative model based on acquired location information and historical information,

[0839] A terminal device that provides guidance information to users based on the generated tourist route,

[0840] A system that includes means for detecting user behavior and enabling the acquisition of stamps.

[0841] (Claim 2)

[0842] A terminal that provides additional entertainment content when a user acquires a stamp, according to claim 1.

[0843] (Claim 3)

[0844] A computing device according to claim 1 that records the user's behavior history and the results of acquired stamps, and generates new guidance information for future visits.

[0845] "Example 1"

[0846] (Claim 1)

[0847] Information processing device means for acquiring the user's location information and history information,

[0848] A computing device means that generates the optimal visit route for the user using a generated AI model based on acquired location information and history information,

[0849] Information processing device means that provides guidance information to the user based on the generated visit route,

[0850] A system that includes means for detecting user behavior and providing a digital certificate obtainable at a specific location.

[0851] (Claim 2)

[0852] The information processing device system according to claim 1, which provides additional entertainment content when a user obtains a digital certificate.

[0853] (Claim 3)

[0854] A computing device according to claim 1 that records the user's behavior history and the results of acquired digital certificates, and generates new guidance information for future visits.

[0855] "Application Example 1"

[0856] (Claim 1)

[0857] A device means for acquiring user coordinate information and history information,

[0858] A computing device means that generates the optimal spatial path to the user using a generation model based on acquired coordinate information and historical information,

[0859] A device that provides information to the user based on the generated spatial path,

[0860] A device that detects user actions and enables the acquisition of a virtual signature,

[0861] A system that includes means for providing users with three-dimensional spatial information through a visual device.

[0862] (Claim 2)

[0863] A device that provides additional entertainment elements when a user obtains a virtual seal, according to claim 1.

[0864] (Claim 3)

[0865] The system according to claim 1, which records the user's activity history and the results of acquired virtual seals, and generates new guidance information for future visits.

[0866] "Example 2 of combining an emotion engine"

[0867] (Claim 1)

[0868] In addition to the user's location information and history information, a terminal means for sensing and acquiring their emotional state,

[0869] An information processing device that generates an optimal tourist route considering the user's emotional state using a generative model based on acquired location information, historical information, and emotional data,

[0870] A terminal device that provides users with dynamic guidance information based on the generated tourist route,

[0871] A system that includes means for dynamically optimizing the selection of entertainment content provided in response to the user's emotions.

[0872] (Claim 2)

[0873] A terminal that, when a user reaches a stamp point and obtains a stamp, provides customized content based on their emotional state, according to claim 1.

[0874] (Claim 3)

[0875] The information processing device according to claim 1, which records the user's location information, history information, emotional state, and acquired stamp results, and generates new guidance information for future visits.

[0876] "Application example 2 when combining with an emotional engine"

[0877] (Claim 1)

[0878] Information terminal means for acquiring the user's location information, history information, and emotional state,

[0879] A computing device means that generates an optimal route according to the user's emotional state using a generative model based on acquired location information, history information, and emotion data,

[0880] An information terminal means that provides users with guidance information corresponding to their emotional state based on the generated route,

[0881] A system that includes means for detecting the emotional state of users and dynamically customizing purchase suggestions.

[0882] (Claim 2)

[0883] The information terminal system according to claim 1, which provides personalized product suggestions when the user is in a specific emotional state.

[0884] (Claim 3)

[0885] A computing device according to claim 1 that records the user's emotional history and purchase suggestion history, and generates new guidance information for future visits. [Explanation of symbols]

[0886] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A terminal means for acquiring the user's location information and history information, A computing device means that generates the optimal sightseeing route for the user using a generative model based on acquired location information and historical information, A terminal device that provides guidance information to users based on the generated tourist route, A system that includes means for detecting user behavior and enabling the acquisition of stamps.

2. A terminal that provides additional entertainment content when a user acquires a stamp, according to claim 1.

3. A computing device according to claim 1 that records the user's behavior history and the results of acquired stamps, and generates new guidance information for future visits.

Images