System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting hot spring data and storing location information, features, and information on surrounding facilities in a database; means for inputting attribute information of a user and recommending an optimum hot spring based on the attribute information; means for generating a virtual hot spring tour based on a selected hot spring and providing the virtual hot spring tour to a terminal of the user; and means for sharing information on a virtual hot spring experience and an actual travel with other users and promoting interaction.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] While hot spring trips are a popular activity, it is difficult to visit all of the numerous hot spring resorts, and users often have trouble deciding which hot spring to visit. Furthermore, traveling to a hot spring resort takes time and money, making it inconvenient for users who want to easily experience the atmosphere of a hot spring. Furthermore, there are limited opportunities to share hot spring experiences and interact with other users. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database; a means for inputting user attribute information and recommending optimal hot springs based on the attribute information; a means for generating a virtual hot spring tour based on the selected hot springs and providing the tour to the user's device; a means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user; and a means for sharing information about virtual hot spring experiences and actual travels with other users to promote interaction. This system allows users to enjoy a variety of hot spring experiences from the comfort of their own homes, select the optimal hot spring resort, and make specific travel plans. Furthermore, users can interact with other users through the virtual hot spring experience and form new communities.

[0006] "Hot spring data" refers to information about hot springs, including location information, hot spring characteristics, facilities, prices, information about surrounding facilities, photos, videos, and reviews.

[0007] "User attribute information" refers to data regarding the type of hot spring the user prefers, travel style, budget, and companions.

[0008] "Recommendation" means proposing the best options based on the user's preferences and needs.

[0009] A "virtual hot spring tour" is content that includes 3D models, video, and audio data that are generated to allow users to simulate the experience of a real hot spring.

[0010] "Device" refers to electronic devices such as smartphones, tablets, personal computers, and VR glasses.

[0011] A "travel plan" refers to a specific schedule and arrangements created based on information such as accommodation, transportation, tourist spots, and restaurants during a trip.

[0012] The "database" is a system for organizing and storing various data related to hot springs.

[0013] "Sharing" means sharing your virtual hot spring experience and travel information with other users and interacting with them through reactions such as viewing and commenting. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention provides a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. An embodiment of this system will be described in detail below.

[0036] System Configuration

[0037] The system mainly consists of the following three main components:

[0038] 1. Server

[0039] 2. Terminal

[0040] 3. Users

[0041] Implementation flow and operation of each element

[0042] 1. Hot spring data collection and mapping

[0043] server

[0044] The server collects data about hot springs from various sources (official websites, review sites, APIs, etc.) The collected data includes location information, the type of hot spring, its characteristics, information about surrounding facilities, photos, and videos.

[0045] The collected data is organized and stored in a database that is regularly updated to reflect new and changed hot spring information.

[0046] 2. Input of user attribute information and recommendations

[0047] User

[0048] When users log in to the system for the first time, they enter attribute information such as their preferred hot spring characteristics (e.g., open-air bath, sauna, family-friendly, etc.), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[0049] server

[0050] The server reads the attribute information entered by the user and runs an algorithm to match it with hot spring data stored in a database, thereby creating a list of hot spring candidates that are most suitable for the user and recommending them.

[0051] 3. Creating a Virtual Hot Spring Tour

[0052] server

[0053] Based on the selected hot springs, the server generates a virtual hot spring tour using 3D models, footage of the actual hot springs, and audio data.

[0054] This data is then processed for virtual reality (VR) experiences and visual experiences on mobile devices.

[0055] Terminal

[0056] The generated virtual hot spring tour is provided to the user's device, and the user can start the virtual hot spring tour on the device and enjoy the experience as if they were in a real hot spring.

[0057] 4. Providing a virtual hot spring experience using smartphones and VR glasses

[0058] Terminal

[0059] The user launches the application on their smartphone or VR glasses and accesses the virtual hot spring tour provided by the server.

[0060] The device receives data from the server and plays 3D models and sound data to provide the user with a virtual experience.

[0061] 5. Travel planning generation and support

[0062] server

[0063] When a user experiences a virtual hot spring tour and selects a hot spring resort that they like, the server automatically generates a travel plan to that hot spring resort.

[0064] The travel plan includes details such as accommodation information, transportation options, information on nearby tourist attractions, and restaurants.

[0065] User

[0066] The user can review the proposed travel plan and make accommodation reservations, transportation arrangements, etc. based on the plan.

[0067] During the actual trip, the application can be used to provide navigation and tourist information.

[0068] 6. Sharing and interacting with virtual hot spring experiences

[0069] User

[0070] Users can post screenshots and videos of their virtual hot spring experience on social media and in-app communities.

[0071] Other users can comment on the posts and share the same virtual hot spring experience.

[0072] Specific examples

[0073] Below are some specific examples of how the system can be used.

[0074] Example 1:

[0075] User A likes open-air baths at hot springs and wants to travel with his family. When he logs in for the first time, he enters this information as attributes.

[0076] The server uses its database to recommend hot spring resorts with open-air baths suitable for families in areas rich in nature.

[0077] When User A selects one of the recommended hot springs and experiences a virtual hot spring tour, a 3D model of the hot spring resort and actual video and audio data are displayed on the device.

[0078] If user A likes the hot spring resort, the server generates a specific travel plan to the hot spring resort and proposes it to user A.

[0079] Example 2:

[0080] User B uses a smartphone app to select a virtual hot spring tour to enjoy the virtual reality experience of a hot spring at the summit of Mount Fuji.

[0081] The server combines the 3D model of Mount Fuji with the model data of the hot spring and provides it to User B's terminal.

[0082] User B puts on the VR glasses and enjoys the experience of soaking in a hot spring at the top of Mount Fuji.

[0083] In this way, by using this system, users can enjoy a variety of hot spring experiences from the comfort of their own home, find the perfect hot spring resort, and plan an actual trip. It is also possible to share virtual hot spring experiences with other users and create new opportunities for interaction.

[0084] The processing flow will be explained below.

[0085] Step 1:

[0086] server

[0087] Start collecting hot spring data. The server automatically collects hot spring data (location information, type of hot spring, characteristics, information on surrounding facilities, photos, videos) from various sources such as official websites, review sites, and APIs.

[0088] Step 2:

[0089] server

[0090] Organize the collected hot spring data. Convert the location information, hot spring characteristics, and information on surrounding facilities into an appropriate format and store it in a database.

[0091] Step 3:

[0092] User

[0093] When logging in to the system for the first time, users enter their personal information, including their preferred hot spring features (e.g., open-air bath, sauna, family-friendly), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[0094] Step 4:

[0095] server

[0096] The server analyzes the attribute information entered by the user, runs an algorithm to match the attribute information with hot spring data in the database, and generates a list of optimal hot spring candidates.

[0097] Step 5:

[0098] server

[0099] The generated list of hot spring candidates is sent to the terminal for recommendation to the user, allowing the user to view a list of hot springs that are best suited to them.

[0100] Step 6:

[0101] User

[0102] Check the list of recommended hot springs through your device and select the hot spring resort that interests you.

[0103] Step 7:

[0104] server

[0105] A virtual hot spring tour is generated based on the selected hot springs. The 3D model, video footage of the actual hot springs, and audio data are integrated to create the virtual hot spring tour data.

[0106] Step 8:

[0107] Terminal

[0108] The generated virtual hot spring tour data is received and provided to the user, who then begins the virtual hot spring tour using a smartphone or VR glasses.

[0109] Step 9:

[0110] User

[0111] Take a virtual hot spring tour and enjoy the atmosphere of the hot spring resort, as if you were actually there.

[0112] Step 10:

[0113] User

[0114] After experiencing the virtual hot spring tour, choose your favorite hot spring resort.

[0115] Step 11:

[0116] server

[0117] Generate a specific itinerary to the selected hot spring resort, including details on accommodation, transportation, nearby tourist attractions, restaurants, etc.

[0118] Step 12:

[0119] server

[0120] The generated travel plan is proposed to the user, who can then review the proposed plan and proceed with reservations and arrangements as necessary.

[0121] Step 13:

[0122] User

[0123] Use the application during your actual trip to receive real-time support for navigation and tourist information.

[0124] Step 14:

[0125] User

[0126] Post screenshots and videos of your virtual hot spring experience on social media and in-app communities to interact with other users.

[0127] Step 15:

[0128] User

[0129] Users can view virtual hot spring experiences posted by other users and create a new forum for interaction through comments and reactions.

[0130] As described above, this system starts by collecting hot spring data, then provides recommendations based on the user's attribute information, creates and experiences a virtual hot spring tour, proposes specific travel plans, and finally shares and interacts with the virtual hot spring experience, all in a single flow.

[0131] Example 1

[0132] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0133] Conventional hot spring information systems provide insufficient support for users to find hot spring resorts and often lack the data and algorithms required to provide virtual hot spring experiences. As a result, users have difficulty efficiently finding hot spring resorts that suit their preferences and obtaining detailed information through virtual hot spring experiences. Furthermore, the systems do not provide comprehensive information necessary for planning actual trips, which results in insufficient user convenience. Furthermore, the virtual hot spring experience does not promote interaction between users.

[0134] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0135] In this invention, the server includes: means for collecting hot spring information and storing location data, facility features, and information on surrounding services in a database; means for inputting user attribute information and recommending optimal hot springs based on the attribute information; means for organizing and frequently updating the collected hot spring information; means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal; means for generating 3D models and environmental acoustic data for the virtual hot spring tour; means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user; means for providing a virtual hot spring experience using a smartphone or VR device; and means for sharing information about the virtual hot spring experience and actual travel with other users and promoting communication. This allows users to enjoy a variety of hot spring experiences from the comfort of their own homes, find the perfect hot spring, and efficiently plan their actual travel. Furthermore, the virtual hot spring experience also promotes communication with other users, stimulating the sharing of hot spring information.

[0136] "Hot spring information" is comprehensive data including location data for hot spring facilities, the type of hot spring, its characteristics, information on surrounding services, and images and videos of the facility.

[0137] "User attribute information" is information relating to the user's preferences and needs, such as the characteristics of the hot springs the user prefers, travel style, budget, and traveling companions.

[0138] The "database" is an information management system that systematically stores and manages collected hot spring information and user attribute information, and allows searching and updating as needed.

[0139] "Recommendation" is the act of selecting the most suitable hot spring facility based on the user's attribute information and suggesting it to the user.

[0140] A "virtual hot spring tour" is a digital tour generated based on a selected hot spring, virtually recreating an actual hot spring experience using 3D models and acoustic data.

[0141] A "3D model" is digital data that recreates the selected hot spring facility and its surroundings in three dimensions, providing a three-dimensional visual experience in a virtual space.

[0142] "Environmental acoustic data" is the actual acoustic environment of selected hot spring facilities that has been recorded and saved in digital format, allowing for an auditory experience in a virtual space.

[0143] A "travel plan" is a comprehensive travel schedule that includes information on accommodations, transportation, surrounding tourist spots, and restaurants that a user needs when visiting a specific hot spring resort.

[0144] A "smartphone" is a portable telecommunications device that provides advanced computing power and a wide variety of features in addition to the functionality of a mobile phone.

[0145] A "VR device" is hardware that provides a virtual reality experience, allowing users to visually explore digital spaces freely.

[0146] "Promoting interaction" means strengthening connections between users by allowing multiple users to share information and exchange opinions through the system.

[0147] The present invention is a system that allows users to enjoy various hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. The system consists of three main components, which are detailed below:

[0148] 1. Server

[0149] 2. Terminal

[0150] 3. Users

[0151] Server Operation

[0152] Hot spring information collection and mapping

[0153] The server collects data from various sources, such as official hot spring websites, review websites, and APIs. The collected data includes hot spring location information, type, characteristics, information on surrounding facilities, and photos and videos. For example, from the official website of a hot spring facility, the server collects location information (latitude and longitude), hot spring water quality (sulfur spring, carbonated spring, etc.), and characteristics (open-air bath, family bath, with sauna), etc.

[0154] The server organizes the collected data and stores it in a database that is automatically updated to reflect new or changed hot spring information.

[0155] Input of user attribute information and recommendations

[0156] When users log in to the system for the first time, they enter information such as their preferred hot spring characteristics (e.g., open-air bath, sauna, family-friendly, etc.), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[0157] The server reads the attribute information entered by the user and matches it with hot spring information in the database. This algorithm recommends hot springs that best fit the user's preferences.

[0158] Creating a virtual hot spring tour

[0159] The server generates a virtual hot spring tour based on the selected hot spring resort using 3D models, video footage of the actual hot spring, and audio data. For example, it collects and processes 360-degree images of an open-air bath and audio data of the hot spring for VR devices.

[0160] The terminal provides the generated virtual hot spring tour data to the user, and the user can start and enjoy the virtual hot spring tour on the terminal.

[0161] Device behavior

[0162] Providing a virtual hot spring experience

[0163] The device plays a virtual hot spring tour using a smartphone or VR glasses. For example, when a user starts the application using a smartphone or VR glasses, it receives data provided by the server and plays 3D models and sound data to provide a virtual experience.

[0164] User Actions

[0165] Travel plan generation and support

[0166] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a specific travel plan to that hot spring resort, including information on accommodations, transportation, surrounding tourist spots, and restaurants.

[0167] The user can review the proposed itinerary and make accommodation reservations and transportation arrangements, for example by directly accessing a hotel booking site through the application to complete the accommodation arrangements.

[0168] Sharing and interacting with virtual hot spring experiences

[0169] Users can post screenshots and videos of their virtual hot spring experience on social media or in the app's community to share with other users, who can interact with the posts by commenting or liking them.

[0170] Specific examples

[0171] Below are some specific examples of how the system can be used.

[0172] Example 1:

[0173] User A likes outdoor baths and wants to travel with his family. He enters this information into the system when he logs in for the first time.

[0174] The server uses its database to recommend hot spring resorts with open-air baths suitable for families in areas rich in nature.

[0175] The terminal plays the virtual hot spring tour data provided by the server, and User A enjoys the 360-degree video and audio data of the open-air bath.

[0176] User A likes the experience and makes accommodation reservations and transportation arrangements based on the travel plan suggested by the system.

[0177] Example 2:

[0178] User B uses VR glasses to enjoy a virtual hot spring experience. For example, he wants to experience soaking in a hot spring at the top of Mt. Fuji.

[0179] The server generates virtual hot spring tour data using a 3D model of Mount Fuji and video and audio data of the hot springs, and provides it to the terminal.

[0180] The terminal plays back the virtual hot spring tour data to User B, who then wears the VR glasses and enjoys the hot spring experience at the top of Mount Fuji.

[0181] Prompt Sentence Examples

[0182] Below are examples of prompts for the generative AI model based on the specific features of this system.

[0183] Prompt: I'd like to travel with my family and I like places with lots of nature. I'm also looking for a hot spring resort with an open-air bath. What kind of hot spring resorts are there?

[0184] Prompt: I want to have a virtual reality experience of soaking in a hot spring at the top of Mt. Fuji. I'd like you to generate a virtual hot spring tour for that purpose.

[0185] In this way, this system allows users to enjoy a variety of hot spring experiences from the comfort of their own home, helping them not only to find the best hot spring resort but also to efficiently plan their actual trip. It also promotes interaction with other users through the virtual hot spring experience, stimulating the sharing of hot spring information.

[0186] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0187] Step 1:

[0188] Hot spring information collection and mapping

[0189] The server obtains data from various sources, including official hot spring websites, review sites, and APIs.

[0190] Input: Sources such as official website URL, review site data, API key, etc.

[0191] Processing: Access these sources and obtain hot spring location information (latitude and longitude), hot spring type (sulfur spring, carbonated spring, etc.), facility features (open-air bath, sauna, etc.), information on surrounding facilities, and photos and videos via scraping or API.

[0192] Output: The collected hot spring information is organized and stored in a database. For example, detailed information about a specific hot spring facility is saved in the database.

[0193] Step 2:

[0194] Input of user attribute information and recommendations

[0195] When users log in for the first time, they enter information into the system such as their preferred hot spring characteristics, travel style, budget, and travel companions.

[0196] Input: User preferences and travel wishes (e.g., options such as "hot spring resort with open-air baths," "family-friendly," and "luxury").

[0197] Processing: The server creates a user profile based on the input attribute information, stores it in a database, and runs an algorithm to match the user with the hot spring information in the database and find the hot spring resort that best suits them.

[0198] Output: The best hot springs are recommended to the user in a list or ranking format. For example, a list of hot springs based on the user's preferences is displayed.

[0199] Step 3:

[0200] Creating a virtual hot spring tour

[0201] The server generates a virtual hot spring tour based on the recommended hot spring resorts.

[0202] Input: Information about the recommended hot spring resort (location information, facility features, photos, videos, audio data, etc.).

[0203] Processing: The server uses this information to generate 3D models and actual video and audio data, and organizes them into virtual hot spring tour data. For example, it can stitch together 360-degree images to recreate a realistic view of a hot spring.

[0204] Output: Virtual hot spring tour data is generated and sent to the device.

[0205] Step 4:

[0206] Providing a virtual hot spring experience

[0207] The device provides a virtual hot spring experience using a smartphone or VR glasses.

[0208] Input: Virtual hot spring tour data provided by the server.

[0209] Processing: The device receives the data and presents it to the user through an application that plays back the 3D model and audio data. The user then enjoys the virtual hot spring experience through their smartphone or VR glasses. For example, they can view a 360-degree video of the hot spring through VR goggles, giving the feeling that they are actually in the hot spring.

[0210] Output: Playback of images and sounds as a virtual hot spring experience. The user experiences a virtual hot spring tour.

[0211] Step 5:

[0212] Travel plan generation and support

[0213] The server automatically creates a specific travel plan based on the user's favorite hot springs.

[0214] Input: Details of the hot spring resort selected by the user, and the user's travel preferences.

[0215] Processing: The server automatically generates a travel plan to the hot spring resort. Specifically, it creates a schedule by combining information on accommodations, transportation options (bullet trains, buses, rental cars, etc.), and information on surrounding tourist spots and restaurants. For example, it suggests the optimal route to the hot spring resort and a stay schedule.

[0216] Output: A travel itinerary is automatically generated and presented to the user, who can then make specific reservations and arrangements based on the proposed plan.

[0217] Step 6:

[0218] Sharing and interacting with virtual hot spring experiences

[0219] Users can share screenshots and videos of their virtual hot spring experience with other users on social media or within the dedicated app.

[0220] Input: Screenshots and videos taken during your virtual spa experience.

[0221] Processing: Users can post this data to social media or in-app communities, where other users can comment, like, or provide feedback on other users' posts.

[0222] Output: The posted content is visible to other users and they can comment, like, or otherwise interact with it.

[0223] (Application example 1)

[0224] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0225] In today's world, there are limited ways for users to experience various hot spring resorts from the comfort of their own homes and use the information they gain to plan their trips. This makes it difficult to fully understand the appeal of hot springs without actually visiting them, and it is also difficult to compare and consider multiple hot spring resorts at once. The present invention aims to solve these problems by providing a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own homes, select the most suitable hot spring resort, and easily plan their trips.

[0226] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0227] In this invention, the server includes means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database, means for inputting user attribute information and recommending optimal hot springs based on the attribute information, means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal, means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user, means for sharing information about the virtual hot spring experience and the actual travel with other users and promoting interaction, means for using a generative AI model to recommend optimal hot spring resorts based on the user's attribute information, and means for using prompt sentences as input when generating the virtual hot spring tour and providing an experience tailored to the user's preferences. This allows users to enjoy a variety of hot spring experiences in virtual reality from the comfort of their own home, and makes it easy to find optimal hot spring resorts and plan their travels.

[0228] "Hot spring data" is a general term for information related to hot springs, such as the location, characteristics, and information on surrounding facilities of hot springs.

[0229] "Location information" is information that indicates the geographical location of a particular hot spring resort or facility.

[0230] "Characteristics" refers to the individual characteristics of a hot spring resort, such as the type of hot spring, temperature, ingredients, and facilities for use.

[0231] "Nearby facilities" refers to information about accommodations, restaurants, tourist spots, etc. located near the hot spring resort.

[0232] The "database" is a system for storing and managing collected hot spring data.

[0233] "User attribute information" is individual information such as the user's hot spring preferences, travel style, budget, and traveling companions.

[0234] "Recommendation" refers to the act of suggesting the most suitable hot spring resort to a user based on collected data and algorithms.

[0235] "Virtual Hot Spring Tour" is content that uses virtual reality technology to provide users with a virtual hot spring experience.

[0236] "User's device" is a general term for devices used by users, such as smartphones, tablets, and VR devices.

[0237] A "travel plan" is a plan that includes information such as accommodation, transportation, and surrounding tourist facilities when the user actually visits a hot spring resort.

[0238] A "generative AI model" is an algorithm that uses artificial intelligence technology to generate a specific output result based on input data.

[0239] A "prompt sentence" is text data that is input into a generative AI model and serves as the starting point for the AI ​​to generate output based on it.

[0240] System Configuration

[0241] The system consists of the following main components:

[0242] server

[0243] Devices (smartphones, tablets, VR devices, etc.)

[0244] User

[0245] Hot Spring Data Collection and Mapping

[0246] server

[0247] The server collects various data about hot springs from official hot spring websites, review websites, APIs, etc. The collected data includes location information, characteristics, and information about surrounding facilities. This data is organized and stored in a database. The database is regularly updated to reflect new and changed hot spring information.

[0248] Input of user attribute information and recommendations

[0249] User

[0250] Users log in to the system and enter attribute information such as their hot spring preferences, travel style, budget, and travel companions.

[0251] server

[0252] The server reads the attribute information entered by the user and recommends the most suitable hot spring resort using a generative AI model, which is built using machine learning libraries such as TENSORFLOW (registered trademark).

[0253] Creating a virtual hot spring tour

[0254] server

[0255] The server generates a virtual hot spring tour based on the data of the selected hot spring resort, using 3D models and environmental acoustic data. Unity software is used to create the 3D models.

[0256] The generated virtual hot spring tour is provided to the user as input with prompt sentences, allowing the user to have an experience tailored to their preferences.

[0257] Providing a virtual hot spring experience

[0258] Terminal

[0259] The user's device (smartphone, tablet, VR device, etc.) receives the virtual hot spring tour provided by the server and provides the user with a virtual reality hot spring experience. The device renders the 3D model and plays video and audio data.

[0260] Travel plan generation and support

[0261] server

[0262] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a travel plan to that hot spring resort, including information on accommodations, transportation, and surrounding tourist attractions.

[0263] User

[0264] Based on the proposed itinerary, users can book accommodation and arrange transportation, and can also use the application to receive navigation and tourist information during the actual trip.

[0265] Sharing and interacting with virtual hot spring experiences

[0266] User

[0267] Users can share screenshots and videos of their virtual hot spring experience on social media and in-app communities, where other users can comment on the posts and share the same virtual hot spring experience.

[0268] Specific examples

[0269] User A likes open-air baths and wants to travel with his or her family. When logging in for the first time, he or she enters this information as attributes. The server uses a generative AI model to recommend hot spring resorts with open-air baths that are ideal for families. User A experiences a virtual hot spring tour using a VR device and selects a hot spring resort that he or she likes. The server generates a specific travel plan to that hot spring resort and proposes it to User A. User A makes a travel reservation based on the proposed plan and actually visits the resort.

[0270] Prompt Sentence Examples

[0271] "Data collection for hot spring tour experience:

[0272] Users like the open-air baths, relaxation, and family-friendly features.

[0273] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0274] System processing steps

[0275] Step 1: Collecting and mapping hot spring data

[0276] server

[0277] The server collects data about hot springs from the APIs of official hot spring websites and review websites. Input data includes hot spring location information, characteristics, and information about surrounding facilities. This data is scraped using Python's BeautifulSoup and Requests libraries to extract the necessary information. The extracted data is organized and stored in a database (AWS (registered trademark) RDS). The database is updated regularly to reflect new and changed hot spring information. The hot spring data stored in the database is obtained as output.

[0278] Step 2: Enter user attribute information

[0279] User

[0280] The user logs in to the system and inputs attribute information such as the characteristics of their preferred hot springs, their travel style, their budget, and the people they will be traveling with. The input data is information about the user's hot spring preferences and travel. The server receives this attribute information and stores it in a database. As an output, the user attribute information is saved in the database.

[0281] Step 3: Recommend the best hot spring

[0282] server

[0283] The server uses a generative AI model (TensorFlow) to recommend the most suitable hot spring resort based on the collected hot spring data and the user's attribute information. The input data is the user's attribute information and existing hot spring data. The generative AI model processes this information and outputs a list of the most suitable hot spring resorts for the user. The output is a list of recommended hot spring resorts.

[0284] Step 4: Generate a virtual hot spring tour

[0285] server

[0286] The server generates a virtual hot spring tour based on the data of the selected hot spring resort. The input data is detailed information about the recommended hot spring resort. Unity software is used to generate 3D models and environmental acoustic data to create the virtual hot spring tour. The output is the data of the virtual hot spring tour. Specifically, a 3D model of the hot spring resort is constructed in Unity and sound effects are incorporated.

[0287] Step 5: Providing a virtual hot spring experience

[0288] Terminal

[0289] The user's device (smartphone, tablet, VR device, etc.) receives the virtual hot spring tour provided by the server and provides the user with a virtual reality hot spring experience. The input data is the virtual hot spring tour data sent from the server. The device receives this data, renders a 3D model, and plays video and audio data. The output is the user's ability to enjoy the virtual hot spring experience in VR or on their smartphone.

[0290] Step 6: Travel plan generation and support

[0291] server

[0292] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a travel plan to that hot spring resort. The input data is information about the hot spring resort selected by the user. The server generates a travel plan including information about accommodation, transportation, and surrounding tourist spots, and proposes it to the user. The output is a specific travel plan. Specific operations include searching for various information and creating an appropriate plan.

[0293] Step 7: Share and interact with your virtual hot spring experience

[0294] User

[0295] Users share screenshots and videos of their virtual hot spring experience on social media or in-app communities. The input data are images and videos taken during the virtual hot spring experience. Other users can comment on the posted content and share the same virtual hot spring experience. The output is to encourage interaction between users. Specific operations utilize the posting and commenting functions on social media and within the app.

[0296] Prompt Sentence Examples

[0297] "Data collection for hot spring tour experience:

[0298] Users like the open-air baths, relaxation, and family-friendly features.

[0299] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0300] The present invention provides a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home, plan trips to their favorite hot spring resorts, and dynamically adjust the content of the experience by recognizing the user's emotions. An embodiment of this system is described in detail below.

[0301] System Configuration

[0302] The system mainly consists of the following four main components:

[0303] 1. Server

[0304] 2. Terminal

[0305] 3. Users

[0306] 4. Emotion Engine

[0307] Implementation flow and operation of each element

[0308] 1. Hot spring data collection and mapping

[0309] server

[0310] The server collects data about hot springs from various sources (official websites, review sites, APIs, etc.) The collected data includes location information, the type of hot spring, its characteristics, information about surrounding facilities, photos, and videos.

[0311] The collected data is organized and stored in a database that is regularly updated to reflect new and changed hot spring information.

[0312] 2. Input of user attribute information and recommendations

[0313] User

[0314] When logging in to the system for the first time, users enter their personal information, including their preferred hot spring features (e.g., open-air bath, sauna, family-friendly), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[0315] server

[0316] The server reads the attribute information entered by the user and runs an algorithm to match it with hot spring data stored in a database, thereby creating a list of hot spring candidates that are most suitable for the user and recommending them.

[0317] 3. Emotional engine for recognizing user emotions

[0318] Emotion Engine

[0319] While the user is entering their attribute information and experiencing the virtual hot spring tour, an emotion engine is activated that recognizes emotions from the user's facial expressions and voice. The emotion engine acquires emotion data and analyzes it in real time.

[0320] server

[0321] It receives emotional data sent from the emotion engine and dynamically adjusts recommendations and changes the content of the virtual experience.

[0322] 4. Creating a Virtual Hot Spring Tour

[0323] server

[0324] The server generates a virtual hot spring tour based on the selected hot springs, using 3D models, video footage of the actual hot springs, and audio data. The experience is customized according to the user's emotional state, as recognized by the emotion engine.

[0325] Terminal

[0326] The generated virtual hot spring tour is provided to the user's device, and the user can start the virtual hot spring tour on the device and enjoy the experience as if they were in a real hot spring.

[0327] 5. Travel planning generation and support

[0328] server

[0329] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a travel plan to that hot spring resort, including details on accommodation, transportation, surrounding tourist attractions, restaurants, etc.

[0330] User

[0331] The user can review the proposed travel plan and make accommodation reservations, transportation arrangements, etc. based on the plan.

[0332] During the actual trip, the application can be used to provide navigation and tourist information.

[0333] 6. Sharing and interacting with virtual hot spring experiences

[0334] User

[0335] Users can post screenshots and videos of their virtual hot spring experience on social media and in-app communities.

[0336] Other users can comment on the posts and share the same virtual hot spring experience.

[0337] Specific examples

[0338] Below are some specific examples of how the system can be used.

[0339] Example 1:

[0340] User A likes open-air baths at hot springs and wants to travel with his family. When he logs in for the first time, he enters this information as attributes.

[0341] The server uses its database to recommend hot spring resorts with open-air baths suitable for families in areas rich in nature.

[0342] User A chooses one of the recommended hot springs and experiences a virtual hot spring tour. The emotion engine recognizes that User A is relaxed from his / her facial expressions and voice, and adds further relaxing elements.

[0343] If user A likes the hot spring resort, the server generates a specific travel plan to the hot spring resort and proposes it to user A.

[0344] Example 2:

[0345] User B uses a smartphone app to select a virtual hot spring tour to enjoy the virtual reality experience of a hot spring at the summit of Mount Fuji.

[0346] The server combines the 3D model of Mount Fuji with the model data of the hot spring and provides it to User B's terminal.

[0347] When User B puts on the VR glasses and begins the experience, the emotion engine recognizes excitement and surprise and adjusts the experience to make it even more exciting.

[0348] In this way, by using this system, users can enjoy a variety of hot spring experiences from the comfort of their own home, find the perfect hot spring resort, and plan an actual trip. Furthermore, by combining it with an emotion engine, it is possible to provide an experience optimized for the user's emotional state, and share the virtual hot spring experience with other users, creating a new platform for interaction.

[0349] The processing flow will be explained below.

[0350] Step 1:

[0351] server

[0352] Start collecting hot spring data. The server automatically collects hot spring data (location information, type of hot spring, characteristics, information on surrounding facilities, photos, videos) from various sources such as official websites, review sites, and APIs.

[0353] Step 2:

[0354] server

[0355] The collected hot spring data is organized and stored in a database. Location information, hot spring characteristics, and information on surrounding facilities are converted into an appropriate format and centrally managed.

[0356] Step 3:

[0357] User

[0358] When logging in to the system for the first time, users enter their demographic information, including their preferred hot spring features (e.g., open-air bath, sauna, family-friendly, etc.), travel style (e.g., relaxation, sightseeing, activities, etc.), budget, and travel companions.

[0359] Step 4:

[0360] server

[0361] The server analyzes the attribute information entered by the user, and then executes a matching algorithm based on the hot spring data in the database and the user's attribute information to generate the most suitable hot spring candidates.

[0362] Step 5:

[0363] server

[0364] The generated list of hot spring candidates is recommended to the user, allowing the user to check a list of hot springs that suit their preferences.

[0365] Step 6:

[0366] User

[0367] Check the list of recommended hot springs through your device and select the hot spring resort that interests you.

[0368] Step 7:

[0369] Emotion Engine

[0370] When a user selects a hot spring resort or experiences a virtual hot spring tour, the system analyzes the user's facial expressions and voice in real time to generate emotional data.

[0371] Step 8:

[0372] server

[0373] It receives emotional data from the emotion engine and dynamically adjusts the hot spring recommendation. For example, if the user is excited, it will suggest an active experience, and if the user is relaxed, it will suggest a quiet hot spring spot.

[0374] Step 9:

[0375] server

[0376] It generates a virtual hot spring tour based on the selected hot spring, integrating 3D models, actual video and audio data, and customizing the experience based on emotional data.

[0377] Step 10:

[0378] Terminal

[0379] The generated virtual hot spring tour data is received and provided to the user, who then begins the virtual hot spring tour using a smartphone or VR glasses.

[0380] Step 11:

[0381] User

[0382] Experience the virtual hot spring tour and enjoy the atmosphere of the hot spring resort. The emotion engine continuously analyzes the user's emotions and adjusts the experience in real time.

[0383] Step 12:

[0384] User

[0385] After experiencing the virtual hot spring tour, users can select their favorite hot spring resort. The emotion engine analyzes the user's satisfaction and uses the results to provide feedback for future recommendations and experience adjustments.

[0386] Step 13:

[0387] server

[0388] Generate a specific itinerary to the selected hot spring resort, including details on accommodation, transportation, nearby tourist attractions, restaurants, etc.

[0389] Step 14:

[0390] server

[0391] The generated travel plan is proposed to the user, who can then confirm the proposed plan and proceed with reservations and arrangements.

[0392] Step 15:

[0393] User

[0394] Use the application during your actual trip to receive real-time support for navigation and tourist information.

[0395] Step 16:

[0396] User

[0397] Post screenshots and videos of your virtual hot spring experience on social media and in-app communities to interact with other users.

[0398] Step 17:

[0399] User

[0400] Users can view virtual hot spring experiences posted by other users and create a new forum for interaction through comments and reactions.

[0401] In this way, this system starts by collecting hot spring data, then provides recommendations based on the user's attribute information and emotional data, creates and experiences a virtual hot spring tour, proposes specific travel plans, and finally shares and interacts with the virtual hot spring experience, all in a single flow.

[0402] Example 2

[0403] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0404] Conventional hot spring information systems make it difficult for users to enjoy a variety of hot spring experiences from home. Furthermore, they lacked the ability to dynamically adjust the experience based on the user's emotional state, which prevented them from fully improving user satisfaction. Furthermore, it was time-consuming and inefficient for users to plan specific trips to their favorite hot spring resorts.

[0405] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0406] In this invention, the server includes means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database, means for inputting user attribute information and recommending optimal hot springs based on the attribute information, means equipped with an emotion engine that acquires user emotion data and analyzes it in real time and dynamically adjusts recommendations and virtual experience content based on the emotion data, means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal, means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user, and means for sharing information about the virtual hot spring experience and the actual trip with other users and promoting communication. This allows users to enjoy a variety of hot spring experiences from the comfort of their own home, dynamically adjust the experience according to their emotional state, and efficiently plan their travel by finding the optimal hot spring resort.

[0407] "Hot spring data" is information about hot spring facilities, including location information, characteristics, information about surrounding facilities, photos, videos, and the like.

[0408] The "database" is an information management system for organizing and storing collected hot spring data in a unified format.

[0409] "User" refers to an individual who uses the system to receive hot spring information recommendations and experience virtual hot spring tours.

[0410] "Attribute information" refers to personal data such as the user's preferences, travel style, budget, and traveling companions.

[0411] "Recommendation" refers to the act of recommending the most suitable hot spring facility based on the user's attribute information.

[0412] The "emotion engine" is a system that acquires emotional data from the user's facial expressions, voice, etc. and analyzes it in real time.

[0413] The "Virtual Hot Spring Tour" is virtual reality content generated using 3D models, actual hot spring footage, and audio data, allowing users to enjoy a hot spring experience on their device.

[0414] A "terminal" is a device used by a user to experience a virtual hot spring tour, and includes a VR device, a mobile terminal, etc.

[0415] A "travel plan" is a detailed plan that automatically generates a specific travel schedule to a user's favorite hot spring resort, including information on accommodation, transportation, and surrounding tourist attractions.

[0416] "Information sharing" refers to the act of sharing information about virtual hot spring experiences and real trips with other users to promote interaction.

[0417] "Dynamic adjustment" refers to changing recommendations and virtual experience content in real time based on the user's emotional state.

[0418] A "hot spring facility" is a place that provides hot bathing services and includes multiple types of baths, such as open-air baths and saunas.

[0419] The present invention is a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. This system is primarily composed of four main components: a server, a terminal, a user, and an emotion engine. An embodiment of this system is described in detail below.

[0420] Hot spring data collection

[0421] server

[0422] The server collects information about hot springs through external APIs such as Google® Places API and TriPad® Visor API. Collected data includes the name of the hot spring, its location, type and characteristics, information about surrounding facilities, photos, and videos. The collected data is organized using a Python script and stored in a MySQL® database. This process is performed periodically, and new information is automatically updated.

[0423] Input of user attribute information and recommendations

[0424] User

[0425] When users log in to the system for the first time, they enter attribute information such as their preferred hot spring characteristics, travel style, budget, and travel companion information through a dedicated form. This form is built with React.js.

[0426] server

[0427] The server parses the information entered by the user in JSON format and stores it in a database. Then, using machine learning algorithms such as Pandas and Scikit-learn, it matches this attribute information with hot spring data. The algorithm then lists and recommends the hot springs that are most suitable for the user. This recommendation is made in the form of providing a list of the top five hot spring facilities based on the filter criteria.

[0428] Recognizing user emotions with an emotion engine

[0429] Emotion Engine

[0430] While the user is entering their personal information or experiencing the virtual hot spring tour, their facial expressions and voice are captured via a webcam and microphone. Using AWS Rekognition and Google Cloud Vision, emotions are analyzed in real time. This emotional data is then sent to a server.

[0431] server

[0432] The server analyzes the data received from the emotion engine and dynamically adjusts recommendations and the content of the virtual experience according to the user's emotional state. For example, if the server detects that the user is relaxed, it will change the background music to soothing music.

[0433] Creating a virtual hot spring tour

[0434] server

[0435] Based on the selected hot spring, the server uses Unity or Unreal Engine to generate a 3D model, video footage of the hot spring, and audio data, which is then sent to the user's device.

[0436] Terminal

[0437] The device displays a 3D virtual hot spring tour on devices such as VR glasses or smartphones, allowing users to experience the hot springs as if they were actually there.

[0438] Generate specific travel plans

[0439] server

[0440] Once the user has experienced a virtual hot spring tour and selected their favorite hot spring, the server automatically generates a travel plan using the Google Maps API and Skyscanner API to collect information on accommodation, transportation, and surrounding tourist spots, and provides the user with a travel plan that is tailored to their needs.

[0441] User

[0442] Users can view the provided itinerary, book accommodation and arrange transportation based on the plan, and use the application to receive real-time navigation and tourist information during their trip.

[0443] Sharing and interacting with virtual hot spring experiences

[0444] User

[0445] Users can take screenshots and videos of their virtual hot spring experience and post them on social media or in the in-app community. Users can also deepen their interactions by commenting on and sharing content posted by other users.

[0446] Specific usage examples and prompt sentence examples

[0447] Example 1

[0448] If User A likes open-air baths and wants to go on a family trip, the server will recommend hot spring resorts with family-friendly open-air baths. User A then experiences a virtual tour of the selected hot spring resort, and if the experience is deemed relaxing, further relaxing elements are added. Finally, the server generates a specific travel plan, allowing User A to efficiently prepare for the trip.

[0449] Example 2

[0450] If User B wishes to experience a virtual hot spring at the top of Mt. Fuji using a smartphone app, the server will provide a combination of a 3D model of Mt. Fuji and hot spring data. User B puts on the VR glasses and begins the virtual experience. If excitement and surprise are recognized, the experience will be adjusted to make it even more exciting.

[0451] Prompt Sentence Examples

[0452] "If the user likes open-air baths and family trips, create a list of recommended hot spring resorts and generate a virtual hot spring tour. Then analyze the user's emotions and tailor the experience accordingly."

[0453] As described above, by using this system, users can enjoy a variety of hot spring experiences from the comfort of their own home, and receive the optimal experience according to their emotional state. It also enables users to find the best hot spring resort and efficiently plan their trip.

[0454] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0455] Step 1: Collecting hot spring data

[0456] The server sends requests to external APIs such as the Google Places API and TripAdvisor API. The input is keywords related to the specified area and hot spring facility. The output is data such as the hot spring facility's name, location, features, information on surrounding facilities, photos, and videos. This data is collected using a Python script, checked for duplicate data, and standardized data formats before being stored in a MySQL database. A job scheduler is used to perform regular updates.

[0457] Step 2: Enter user attribute information

[0458] When users log in to the system for the first time, they enter attribute information such as their preferred hot spring characteristics, travel style, budget, and travel companion information through a dedicated form built with React.js. The information entered is then sent to the server in JSON format.

[0459] Step 3: Store user attribute information

[0460] The server parses the JSON-formatted attribute information received from the user, extracts the necessary fields, and saves them in a MySQL database. The input is JSON data of attribute information, and the output is a database containing organized attribute data.

[0461] Step 4: Run the recommendation algorithm

[0462] The server uses Pandas and Scikit-learn to run an algorithm that matches the user's attribute information with hot spring data. The input is the user's attribute information and hot spring data. The output is a list of hot spring facilities that are best suited to the user. Filter conditions are applied, and the hot spring facilities that match the conditions are scored and the top five are presented to the user.

[0463] Step 5: Acquire and analyze emotion data

[0464] The emotion engine captures facial expressions and voices via a webcam and microphone while the user is entering attribute information or experiencing a virtual hot spring tour. The input is the captured facial image and voice data. This is sent to AWS Rekognition or Google Cloud Vision to obtain emotion data. The output is analyzed emotion data.

[0465] Step 6: Use emotion data

[0466] The server receives emotional data from the emotion engine and dynamically adjusts the recommendations and virtual experience content. The input is the emotional data. The output is the adjusted recommendations and virtual experience content. For example, if the emotion is determined to be relaxing, the background music is changed to soothing music.

[0467] Step 7: Generate a virtual hot spring tour

[0468] The server uses Unity or Unreal Engine to generate a 3D model based on the selected hot spring, as well as actual hot spring video and audio data. The input is the hot spring data. The output is the generated virtual hot spring tour data, which is then sent to the user's device.

[0469] Step 8: Run a virtual hot spring tour

[0470] The terminal provides users with a virtual hot spring tour using devices such as VR glasses or a smartphone. The input is virtual hot spring tour data provided by the server. Users can enjoy an interactive experience as if they were actually in the hot spring resort. The output is the user's experience data.

[0471] Step 9: Generate a travel plan

[0472] When the user selects their favorite hot spring resort, the server automatically generates a specific travel plan using the Google Maps API and Skyscanner API. The input is information about the selected hot spring resort and the user's attribute information. The output is a travel plan that includes information about accommodations, transportation, surrounding tourist attractions, and restaurants.

[0473] Step 10: Travel Plan Proposal

[0474] The user can review the proposed itinerary and make accommodation reservations and transportation arrangements. The input is the itinerary data provided by the server. The output is the user's reservation information. During the trip, the user can also use the application to receive real-time navigation and tourist information.

[0475] Step 11: Share and interact with your virtual hot spring experience

[0476] Users can post screenshots and videos of their virtual hot spring experience on social media and in-app communities. The input is the user's experience data. The output is the shared information and feedback from other users. Users can interact with other users and share their comments and experiences.

[0477] (Application example 2)

[0478] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0479] In today's busy lifestyles, users lack the means to easily plan a hot spring trip to relax and refresh. Furthermore, existing systems struggle to provide a customized experience that takes into account the user's emotions. Furthermore, they lack the ability to allow users to share their virtual experiences and interact with other users. To address these issues, a system is needed that dynamically adjusts the user's experience using emotion recognition to provide an authentic virtual hot spring experience.

[0480] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0481] In this invention, the server includes: means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database; means for inputting user attribute information and recommending optimal hot springs based on the attribute information; means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal; means for recognizing the user's emotions in real time and dynamically adjusting the content of the virtual hot spring tour based on the data; means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user; and means for sharing information about the virtual hot spring experience and the actual travel with other users and promoting interactions. This allows users to enjoy various hot spring experiences from the comfort of their own homes and find the optimal hot spring resort through experiences customized according to their emotions. Furthermore, users can obtain new information and share travel plans through interactions with other users.

[0482] "Hot spring data" is data including location information, characteristics, and information on surrounding facilities related to hot springs.

[0483] "Location information" is information that indicates the geographical location of a hot spring.

[0484] "Characteristics" is information indicating the type and characteristics of the hot spring, the details of the facilities, and so on.

[0485] "Information about surrounding facilities" refers to information about accommodations, tourist spots, restaurants, and the like located near the hot spring.

[0486] A "database" is a storage system in which collected data is systematically stored so that it can be managed and searched.

[0487] "User attribute information" is information input by the user regarding hot spring preferences, travel style, budget, and traveling companions.

[0488] "Recommendation" is a process that suggests optimal options based on user attribute information.

[0489] A "virtual hot spring tour" is a virtual hot spring experience that uses 3D models of hot spring resorts and environmental acoustic data.

[0490] "Terminal" refers to the device that a user uses to use the system, including smartphones, tablets, VR devices, etc.

[0491] "Emotion recognition" is a technology that analyzes emotions from a user's facial expressions and voice in real time.

[0492] A "travel plan" is a plan that includes a specific travel schedule to a hot spring resort, as well as information on accommodation, transportation, and surrounding tourist attractions.

[0493] "Interaction" means that users share information and communicate with each other.

[0494] MODE FOR CARRYING OUT THE INVENTION

[0495] This invention is a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. Furthermore, the system can recognize the user's emotions and dynamically adjust the content of the experience. The following describes an embodiment of the invention.

[0496] System Configuration

[0497] The system consists of four main components:

[0498] 1. Server

[0499] 2. Terminal

[0500] 3. Users

[0501] 4. Emotion Engine

[0502] The specific operation of each element will be explained below.

[0503] Hot Spring Data Collection and Mapping

[0504] The server collects data about hot springs from various sources, including their locations, characteristics, and surrounding facilities. This information is stored in a database and updated periodically.

[0505] Input of user attribute information and recommendations

[0506] When a user logs in to the system for the first time, they enter their own attribute information, including their preferred hot spring characteristics, travel style, budget, and travel companions. The server uses this information to run an algorithm that matches it with hot spring data stored in the database and recommends the most suitable hot springs to the user.

[0507] Recognizing user emotions with an emotion engine

[0508] The emotion engine recognizes emotions from the user's facial expressions and voice in real time while the user is entering attribute information or experiencing a virtual hot spring tour. The recognized emotion data is sent to the server and used to dynamically adjust the experience.

[0509] Creating and providing virtual hot spring tours

[0510] The server generates a virtual hot spring tour based on the selected hot springs, using 3D models of the hot springs and environmental acoustic data. It also has a function to customize the experience depending on the user's emotional state. The generated virtual hot spring tour is provided to the user's devices, such as smartphones, smart glasses, and head-mounted displays.

[0511] Specific examples

[0512] For example, User A likes open-air baths and is planning a trip with his or her family. When User A enters this attribute information when logging in for the first time, the server recommends the most suitable hot spring resort from the database. When User A selects the recommended hot spring resort and starts the virtual hot spring tour, the emotion engine recognizes User A's relaxation state and adjusts the experience to make it even more relaxing.

[0513] Hardware and software used

[0514] Hardware: Smartphones, smart glasses, head-mounted displays, webcams

[0515] Software: OpenCV (cv2), DeepFace, Python standard library

[0516] Prompt Sentence Examples

[0517] Below are some example prompts to input to a generative AI model:

[0518] "How can I implement real-time emotion recognition in Python to customize the user experience for a hot spring experience app? I'd like code that meets the following criteria:"

[0519] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0520] Step 1:

[0521] The server collects hot spring data and stores information on its location, characteristics, and surrounding facilities in a database. This process involves collecting data about hot springs from official websites, review sites, APIs, etc., properly formatting it, and storing it in the database. The input is data from external sources, and the output is formatted hot spring data.

[0522] Step 2:

[0523] When a user logs in for the first time, they enter attribute information such as their hot spring preferences, travel style, budget, and travel companions. The entered attribute information is saved in the database as a user profile. The input is the attribute information provided by the user, and the output is the user attribute information saved in the database.

[0524] Step 3:

[0525] The server recommends the most suitable hot spring resort from the hot spring data stored in the database based on the user's attribute information. The recommendation algorithm compares the user profile with the hot spring data and generates a list of matching hot spring resorts. The input is the user's attribute information and the hot spring data, and the output is a list of recommended hot spring resorts.

[0526] Step 4:

[0527] The server generates a virtual hot spring tour based on hot springs selected from the recommended hot spring resorts. This tour is designed using 3D models of hot springs and environmental acoustic data. The input is information about the selected hot spring resorts, and the output is the generated virtual hot spring tour data.

[0528] Step 5:

[0529] The terminal provides the generated virtual hot spring tour to the user. The user can experience the virtual hot spring tour using devices such as smartphones, smart glasses, and head-mounted displays. The input is the virtual hot spring tour data provided by the server, and the output is the virtual experience through the user's vision and hearing.

[0530] Step 6:

[0531] When a user experiences a virtual hot spring tour, the emotion engine recognizes the user's facial expressions and voice in real time and analyzes the emotional data. The analyzed emotional data is sent to the server. The input is the user's facial expressions and voice data, and the output is the analyzed emotional data.

[0532] Step 7:

[0533] The server dynamically adjusts the content of the virtual hot spring tour based on the received emotion data. For example, if the user is relaxed, it adds more relaxing content, and if the user is excited, it strengthens exciting elements. The input is the emotion data sent from the emotion engine, and the output is the adjusted virtual hot spring tour data.

[0534] Step 8:

[0535] When a user selects a hot spring resort they like, the server automatically generates a specific travel plan to that hot spring resort. This plan includes information on transportation, accommodation, surrounding tourist attractions, etc. The input is the user's selection information, and the output is the generated travel plan.

[0536] Step 9:

[0537] Users can review the proposed itinerary and make any necessary reservations. They can also share and interact with other users about their virtual hot spring experiences and travel information. The input is the proposed itinerary and the user's decision information, and the output is the shared experience information.

[0538] Step 10:

[0539] The terminal collects input from the user (attribute information, emotion data, selection information, etc.) and sends it to the server. It also displays data received from the server (recommendations, virtual tour data, travel plans, etc.). The input is based on data communication between the user and the server, and the output is the overall service experience provided to the user.

[0540] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0541] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0542] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0543] [Second embodiment]

[0544] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0545] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0546] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0547] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0548] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0549] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0550] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0551] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0552] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0553] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0554] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0555] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0556] The present invention provides a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. An embodiment of this system will be described in detail below.

[0557] System Configuration

[0558] The system mainly consists of the following three main components:

[0559] 1. Server

[0560] 2. Terminal

[0561] 3. Users

[0562] Implementation flow and operation of each element

[0563] 1. Hot spring data collection and mapping

[0564] server

[0565] The server collects data about hot springs from various sources (official websites, review sites, APIs, etc.) The collected data includes location information, the type of hot spring, its characteristics, information about surrounding facilities, photos, and videos.

[0566] The collected data is organized and stored in a database that is regularly updated to reflect new and changed hot spring information.

[0567] 2. Input of user attribute information and recommendations

[0568] User

[0569] When users log in to the system for the first time, they enter attribute information such as their preferred hot spring characteristics (e.g., open-air bath, sauna, family-friendly, etc.), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[0570] server

[0571] The server reads the attribute information entered by the user and runs an algorithm to match it with hot spring data stored in a database, thereby creating a list of hot spring candidates that are most suitable for the user and recommending them.

[0572] 3. Creating a Virtual Hot Spring Tour

[0573] server

[0574] Based on the selected hot springs, the server generates a virtual hot spring tour using 3D models, footage of the actual hot springs, and audio data.

[0575] This data is then processed for virtual reality (VR) experiences and visual experiences on mobile devices.

[0576] Terminal

[0577] The generated virtual hot spring tour is provided to the user's device, and the user can start the virtual hot spring tour on the device and enjoy the experience as if they were in a real hot spring.

[0578] 4. Providing a virtual hot spring experience using smartphones and VR glasses

[0579] Terminal

[0580] The user launches the application on their smartphone or VR glasses and accesses the virtual hot spring tour provided by the server.

[0581] The device receives data from the server and plays 3D models and sound data to provide the user with a virtual experience.

[0582] 5. Travel planning generation and support

[0583] server

[0584] When a user experiences a virtual hot spring tour and selects a hot spring resort that they like, the server automatically generates a travel plan to that hot spring resort.

[0585] The travel plan includes details such as accommodation information, transportation options, information on nearby tourist attractions, and restaurants.

[0586] User

[0587] The user can review the proposed travel plan and make accommodation reservations, transportation arrangements, etc. based on the plan.

[0588] During the actual trip, the application can be used to provide navigation and tourist information.

[0589] 6. Sharing and interacting with virtual hot spring experiences

[0590] User

[0591] Users can post screenshots and videos of their virtual hot spring experience on social media and in-app communities.

[0592] Other users can comment on the posts and share the same virtual hot spring experience.

[0593] Specific examples

[0594] Below are some specific examples of how the system can be used.

[0595] Example 1:

[0596] User A likes open-air baths at hot springs and wants to travel with his family. When he logs in for the first time, he enters this information as attributes.

[0597] The server uses its database to recommend hot spring resorts with open-air baths suitable for families in areas rich in nature.

[0598] When User A selects one of the recommended hot springs and experiences a virtual hot spring tour, a 3D model of the hot spring resort and actual video and audio data are displayed on the device.

[0599] If user A likes the hot spring resort, the server generates a specific travel plan to the hot spring resort and proposes it to user A.

[0600] Example 2:

[0601] User B uses a smartphone app to select a virtual hot spring tour to enjoy the virtual reality experience of a hot spring at the summit of Mount Fuji.

[0602] The server combines the 3D model of Mount Fuji with the model data of the hot spring and provides it to User B's terminal.

[0603] User B puts on the VR glasses and enjoys the experience of soaking in a hot spring at the top of Mount Fuji.

[0604] In this way, by using this system, users can enjoy a variety of hot spring experiences from the comfort of their own home, find the perfect hot spring resort, and plan an actual trip. It is also possible to share virtual hot spring experiences with other users and create new opportunities for interaction.

[0605] The processing flow will be explained below.

[0606] Step 1:

[0607] server

[0608] Start collecting hot spring data. The server automatically collects hot spring data (location information, type of hot spring, characteristics, information on surrounding facilities, photos, videos) from various sources such as official websites, review sites, and APIs.

[0609] Step 2:

[0610] server

[0611] Organize the collected hot spring data. Convert the location information, hot spring characteristics, and information on surrounding facilities into an appropriate format and store it in a database.

[0612] Step 3:

[0613] User

[0614] When logging in to the system for the first time, users enter their personal information, including their preferred hot spring features (e.g., open-air bath, sauna, family-friendly), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[0615] Step 4:

[0616] server

[0617] The server analyzes the attribute information entered by the user, runs an algorithm to match the attribute information with hot spring data in the database, and generates a list of optimal hot spring candidates.

[0618] Step 5:

[0619] server

[0620] The generated list of hot spring candidates is sent to the terminal for recommendation to the user, allowing the user to view a list of hot springs that are best suited to them.

[0621] Step 6:

[0622] User

[0623] Check the list of recommended hot springs through your device and select the hot spring resort that interests you.

[0624] Step 7:

[0625] server

[0626] A virtual hot spring tour is generated based on the selected hot springs. The 3D model, video footage of the actual hot springs, and audio data are integrated to create the virtual hot spring tour data.

[0627] Step 8:

[0628] Terminal

[0629] The generated virtual hot spring tour data is received and provided to the user, who then begins the virtual hot spring tour using a smartphone or VR glasses.

[0630] Step 9:

[0631] User

[0632] Take a virtual hot spring tour and enjoy the atmosphere of the hot spring resort, as if you were actually there.

[0633] Step 10:

[0634] User

[0635] After experiencing the virtual hot spring tour, choose your favorite hot spring resort.

[0636] Step 11:

[0637] server

[0638] Generate a specific itinerary to the selected hot spring resort, including details on accommodation, transportation, nearby tourist attractions, restaurants, etc.

[0639] Step 12:

[0640] server

[0641] The generated travel plan is proposed to the user, who can then review the proposed plan and proceed with reservations and arrangements as necessary.

[0642] Step 13:

[0643] User

[0644] Use the application during your actual trip to receive real-time support for navigation and tourist information.

[0645] Step 14:

[0646] User

[0647] Post screenshots and videos of your virtual hot spring experience on social media and in-app communities to interact with other users.

[0648] Step 15:

[0649] User

[0650] Users can view virtual hot spring experiences posted by other users and create a new forum for interaction through comments and reactions.

[0651] As described above, this system starts by collecting hot spring data, then provides recommendations based on the user's attribute information, creates and experiences a virtual hot spring tour, proposes specific travel plans, and finally shares and interacts with the virtual hot spring experience, all in a single flow.

[0652] Example 1

[0653] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0654] Conventional hot spring information systems provide insufficient support for users to find hot spring resorts and often lack the data and algorithms required to provide virtual hot spring experiences. As a result, users have difficulty efficiently finding hot spring resorts that suit their preferences and obtaining detailed information through virtual hot spring experiences. Furthermore, the systems do not provide comprehensive information necessary for planning actual trips, which results in insufficient user convenience. Furthermore, the virtual hot spring experience does not promote interaction between users.

[0655] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0656] In this invention, the server includes: means for collecting hot spring information and storing location data, facility features, and information on surrounding services in a database; means for inputting user attribute information and recommending optimal hot springs based on the attribute information; means for organizing and frequently updating the collected hot spring information; means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal; means for generating 3D models and environmental acoustic data for the virtual hot spring tour; means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user; means for providing a virtual hot spring experience using a smartphone or VR device; and means for sharing information about the virtual hot spring experience and actual travel with other users and promoting communication. This allows users to enjoy a variety of hot spring experiences from the comfort of their own homes, find the perfect hot spring, and efficiently plan their actual travel. Furthermore, the virtual hot spring experience also promotes communication with other users, stimulating the sharing of hot spring information.

[0657] "Hot spring information" is comprehensive data including location data for hot spring facilities, the type of hot spring, its characteristics, information on surrounding services, and images and videos of the facility.

[0658] "User attribute information" is information relating to the user's preferences and needs, such as the characteristics of the hot springs the user prefers, travel style, budget, and traveling companions.

[0659] The "database" is an information management system that systematically stores and manages collected hot spring information and user attribute information, and allows searching and updating as needed.

[0660] "Recommendation" is the act of selecting the most suitable hot spring facility based on the user's attribute information and suggesting it to the user.

[0661] A "virtual hot spring tour" is a digital tour generated based on a selected hot spring, virtually recreating an actual hot spring experience using 3D models and acoustic data.

[0662] A "3D model" is digital data that recreates the selected hot spring facility and its surroundings in three dimensions, providing a three-dimensional visual experience in a virtual space.

[0663] "Environmental acoustic data" is the actual acoustic environment of selected hot spring facilities that has been recorded and saved in digital format, allowing for an auditory experience in a virtual space.

[0664] A "travel plan" is a comprehensive travel schedule that includes information on accommodations, transportation, surrounding tourist spots, and restaurants that a user needs when visiting a specific hot spring resort.

[0665] A "smartphone" is a portable telecommunications device that provides advanced computing power and a wide variety of features in addition to the functionality of a mobile phone.

[0666] A "VR device" is hardware that provides a virtual reality experience, allowing users to visually explore digital spaces freely.

[0667] "Promoting interaction" means strengthening connections between users by allowing multiple users to share information and exchange opinions through the system.

[0668] The present invention is a system that allows users to enjoy various hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. The system consists of three main components, which are detailed below:

[0669] 1. Server

[0670] 2. Terminal

[0671] 3. Users

[0672] Server Operation

[0673] Hot spring information collection and mapping

[0674] The server collects data from various sources, such as official hot spring websites, review websites, and APIs. The collected data includes hot spring location information, type, characteristics, information on surrounding facilities, and photos and videos. For example, from the official website of a hot spring facility, the server collects location information (latitude and longitude), hot spring water quality (sulfur spring, carbonated spring, etc.), and characteristics (open-air bath, family bath, with sauna), etc.

[0675] The server organizes the collected data and stores it in a database that is automatically updated to reflect new or changed hot spring information.

[0676] Input of user attribute information and recommendations

[0677] When users log in to the system for the first time, they enter information such as their preferred hot spring characteristics (e.g., open-air bath, sauna, family-friendly, etc.), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[0678] The server reads the attribute information entered by the user and matches it with hot spring information in the database. This algorithm recommends hot springs that best fit the user's preferences.

[0679] Creating a virtual hot spring tour

[0680] The server generates a virtual hot spring tour based on the selected hot spring resort using 3D models, video footage of the actual hot spring, and audio data. For example, it collects and processes 360-degree images of an open-air bath and audio data of the hot spring for VR devices.

[0681] The terminal provides the generated virtual hot spring tour data to the user, and the user can start and enjoy the virtual hot spring tour on the terminal.

[0682] Device behavior

[0683] Providing a virtual hot spring experience

[0684] The device plays a virtual hot spring tour using a smartphone or VR glasses. For example, when a user starts the application using a smartphone or VR glasses, it receives data provided by the server and plays 3D models and sound data to provide a virtual experience.

[0685] User Actions

[0686] Travel plan generation and support

[0687] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a specific travel plan to that hot spring resort, including information on accommodations, transportation, surrounding tourist spots, and restaurants.

[0688] The user can review the proposed itinerary and make accommodation reservations and transportation arrangements, for example by directly accessing a hotel booking site through the application to complete the accommodation arrangements.

[0689] Sharing and interacting with virtual hot spring experiences

[0690] Users can post screenshots and videos of their virtual hot spring experience on social media or in the app's community to share with other users, who can interact with the posts by commenting or liking them.

[0691] Specific examples

[0692] Below are some specific examples of how the system can be used.

[0693] Example 1:

[0694] User A likes outdoor baths and wants to travel with his family. He enters this information into the system when he logs in for the first time.

[0695] The server uses its database to recommend hot spring resorts with open-air baths suitable for families in areas rich in nature.

[0696] The terminal plays the virtual hot spring tour data provided by the server, and User A enjoys the 360-degree video and audio data of the open-air bath.

[0697] User A likes the experience and makes accommodation reservations and transportation arrangements based on the travel plan suggested by the system.

[0698] Example 2:

[0699] User B uses VR glasses to enjoy a virtual hot spring experience. For example, he wants to experience soaking in a hot spring at the top of Mt. Fuji.

[0700] The server generates virtual hot spring tour data using a 3D model of Mount Fuji and video and audio data of the hot springs, and provides it to the terminal.

[0701] The terminal plays back the virtual hot spring tour data to User B, who then wears the VR glasses and enjoys the hot spring experience at the top of Mount Fuji.

[0702] Prompt Sentence Examples

[0703] Below are examples of prompts for the generative AI model based on the specific features of this system.

[0704] Prompt: I'd like to travel with my family and I like places with lots of nature. I'm also looking for a hot spring resort with an open-air bath. What kind of hot spring resorts are there?

[0705] Prompt: I want to have a virtual reality experience of soaking in a hot spring at the top of Mt. Fuji. I'd like you to generate a virtual hot spring tour for that purpose.

[0706] In this way, this system allows users to enjoy a variety of hot spring experiences from the comfort of their own home, helping them not only to find the best hot spring resort but also to efficiently plan their actual trip. It also promotes interaction with other users through the virtual hot spring experience, stimulating the sharing of hot spring information.

[0707] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0708] Step 1:

[0709] Hot spring information collection and mapping

[0710] The server obtains data from various sources, including official hot spring websites, review sites, and APIs.

[0711] Input: Sources such as official website URL, review site data, API key, etc.

[0712] Processing: Access these sources and obtain hot spring location information (latitude and longitude), hot spring type (sulfur spring, carbonated spring, etc.), facility features (open-air bath, sauna, etc.), information on surrounding facilities, and photos and videos via scraping or API.

[0713] Output: The collected hot spring information is organized and stored in a database. For example, detailed information about a specific hot spring facility is saved in the database.

[0714] Step 2:

[0715] Input of user attribute information and recommendations

[0716] When users log in for the first time, they enter information into the system such as their preferred hot spring characteristics, travel style, budget, and travel companions.

[0717] Input: User preferences and travel wishes (e.g., options such as "hot spring resort with open-air baths," "family-friendly," and "luxury").

[0718] Processing: The server creates a user profile based on the input attribute information, stores it in a database, and runs an algorithm to match the user with the hot spring information in the database and find the hot spring resort that best suits them.

[0719] Output: The best hot springs are recommended to the user in a list or ranking format. For example, a list of hot springs based on the user's preferences is displayed.

[0720] Step 3:

[0721] Creating a virtual hot spring tour

[0722] The server generates a virtual hot spring tour based on the recommended hot spring resorts.

[0723] Input: Information about the recommended hot spring resort (location information, facility features, photos, videos, audio data, etc.).

[0724] Processing: The server uses this information to generate 3D models and actual video and audio data, and organizes them into virtual hot spring tour data. For example, it can stitch together 360-degree images to recreate a realistic view of a hot spring.

[0725] Output: Virtual hot spring tour data is generated and sent to the device.

[0726] Step 4:

[0727] Providing a virtual hot spring experience

[0728] The device provides a virtual hot spring experience using a smartphone or VR glasses.

[0729] Input: Virtual hot spring tour data provided by the server.

[0730] Processing: The device receives the data and presents it to the user through an application that plays back the 3D model and audio data. The user then enjoys the virtual hot spring experience through their smartphone or VR glasses. For example, they can view a 360-degree video of the hot spring through VR goggles, giving the feeling that they are actually in the hot spring.

[0731] Output: Playback of images and sounds as a virtual hot spring experience. The user experiences a virtual hot spring tour.

[0732] Step 5:

[0733] Travel plan generation and support

[0734] The server automatically creates a specific travel plan based on the user's favorite hot springs.

[0735] Input: Details of the hot spring resort selected by the user, and the user's travel preferences.

[0736] Processing: The server automatically generates a travel plan to the hot spring resort. Specifically, it creates a schedule by combining information on accommodations, transportation options (bullet trains, buses, rental cars, etc.), and information on surrounding tourist spots and restaurants. For example, it suggests the optimal route to the hot spring resort and a stay schedule.

[0737] Output: A travel itinerary is automatically generated and presented to the user, who can then make specific reservations and arrangements based on the proposed plan.

[0738] Step 6:

[0739] Sharing and interacting with virtual hot spring experiences

[0740] Users can share screenshots and videos of their virtual hot spring experience with other users on social media or within the dedicated app.

[0741] Input: Screenshots and videos taken during your virtual spa experience.

[0742] Processing: Users can post this data to social media or in-app communities, where other users can comment, like, or provide feedback on other users' posts.

[0743] Output: The posted content is visible to other users and they can comment, like, or otherwise interact with it.

[0744] (Application example 1)

[0745] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0746] In today's world, there are limited ways for users to experience various hot spring resorts from the comfort of their own homes and use the information they gain to plan their trips. This makes it difficult to fully understand the appeal of hot springs without actually visiting them, and it is also difficult to compare and consider multiple hot spring resorts at once. The present invention aims to solve these problems by providing a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own homes, select the most suitable hot spring resort, and easily plan their trips.

[0747] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0748] In this invention, the server includes means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database, means for inputting user attribute information and recommending optimal hot springs based on the attribute information, means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal, means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user, means for sharing information about the virtual hot spring experience and the actual travel with other users and promoting interaction, means for using a generative AI model to recommend optimal hot spring resorts based on the user's attribute information, and means for using prompt sentences as input when generating the virtual hot spring tour and providing an experience tailored to the user's preferences. This allows users to enjoy a variety of hot spring experiences in virtual reality from the comfort of their own home, and makes it easy to find optimal hot spring resorts and plan their travels.

[0749] "Hot spring data" is a general term for information related to hot springs, such as the location, characteristics, and information on surrounding facilities of hot springs.

[0750] "Location information" is information that indicates the geographical location of a particular hot spring resort or facility.

[0751] "Characteristics" refers to the individual characteristics of a hot spring resort, such as the type of hot spring, temperature, ingredients, and facilities for use.

[0752] "Nearby facilities" refers to information about accommodations, restaurants, tourist spots, etc. located near the hot spring resort.

[0753] The "database" is a system for storing and managing collected hot spring data.

[0754] "User attribute information" is individual information such as the user's hot spring preferences, travel style, budget, and traveling companions.

[0755] "Recommendation" refers to the act of suggesting the most suitable hot spring resort to a user based on collected data and algorithms.

[0756] "Virtual Hot Spring Tour" is content that uses virtual reality technology to provide users with a virtual hot spring experience.

[0757] "User's device" is a general term for devices used by users, such as smartphones, tablets, and VR devices.

[0758] A "travel plan" is a plan that includes information such as accommodation, transportation, and surrounding tourist facilities when the user actually visits a hot spring resort.

[0759] A "generative AI model" is an algorithm that uses artificial intelligence technology to generate a specific output result based on input data.

[0760] A "prompt sentence" is text data that is input into a generative AI model and serves as the starting point for the AI ​​to generate output based on it.

[0761] System Configuration

[0762] The system consists of the following main components:

[0763] server

[0764] Devices (smartphones, tablets, VR devices, etc.)

[0765] User

[0766] Hot Spring Data Collection and Mapping

[0767] server

[0768] The server collects various data about hot springs from official hot spring websites, review websites, APIs, etc. The collected data includes location information, characteristics, and information about surrounding facilities. This data is organized and stored in a database. The database is regularly updated to reflect new and changed hot spring information.

[0769] Input of user attribute information and recommendations

[0770] User

[0771] Users log in to the system and enter attribute information such as their hot spring preferences, travel style, budget, and travel companions.

[0772] server

[0773] The server reads the attribute information entered by the user and recommends the best hot spring resort using a generative AI model, which is built using machine learning libraries such as TensorFlow.

[0774] Creating a virtual hot spring tour

[0775] server

[0776] The server generates a virtual hot spring tour based on the data of the selected hot spring resort, using 3D models and environmental acoustic data. Unity software is used to create the 3D models.

[0777] The generated virtual hot spring tour is provided to the user as input with prompt sentences, allowing the user to have an experience tailored to their preferences.

[0778] Providing a virtual hot spring experience

[0779] Terminal

[0780] The user's device (smartphone, tablet, VR device, etc.) receives the virtual hot spring tour provided by the server and provides the user with a virtual reality hot spring experience. The device renders the 3D model and plays video and audio data.

[0781] Travel plan generation and support

[0782] server

[0783] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a travel plan to that hot spring resort, including information on accommodations, transportation, and surrounding tourist attractions.

[0784] User

[0785] Based on the proposed itinerary, users can book accommodation and arrange transportation, and can also use the application to receive navigation and tourist information during the actual trip.

[0786] Sharing and interacting with virtual hot spring experiences

[0787] User

[0788] Users can share screenshots and videos of their virtual hot spring experience on social media and in-app communities, where other users can comment on the posts and share the same virtual hot spring experience.

[0789] Specific examples

[0790] User A likes open-air baths and wants to travel with his or her family. When logging in for the first time, he or she enters this information as attributes. The server uses a generative AI model to recommend hot spring resorts with open-air baths that are ideal for families. User A experiences a virtual hot spring tour using a VR device and selects a hot spring resort that he or she likes. The server generates a specific travel plan to that hot spring resort and proposes it to User A. User A makes a travel reservation based on the proposed plan and actually visits the resort.

[0791] Prompt Sentence Examples

[0792] "Data collection for hot spring tour experience:

[0793] Users like the open-air baths, relaxation, and family-friendly features.

[0794] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0795] System processing steps

[0796] Step 1: Collecting and mapping hot spring data

[0797] server

[0798] The server collects data about hot springs from the APIs of official hot spring websites and review websites. Input data includes hot spring location information, characteristics, and information about surrounding facilities. This data is scraped using Python's BeautifulSoup and Requests libraries to extract the necessary information. The extracted data is organized and stored in a database (AWS RDS). The database is updated regularly to reflect new and changed hot spring information. The hot spring data stored in the database is obtained as output.

[0799] Step 2: Enter user attribute information

[0800] User

[0801] The user logs in to the system and inputs attribute information such as the characteristics of their preferred hot springs, their travel style, their budget, and the people they will be traveling with. The input data is information about the user's hot spring preferences and travel. The server receives this attribute information and stores it in a database. As an output, the user attribute information is saved in the database.

[0802] Step 3: Recommend the best hot spring

[0803] server

[0804] The server uses a generative AI model (TensorFlow) to recommend the most suitable hot spring resort based on the collected hot spring data and the user's attribute information. The input data is the user's attribute information and existing hot spring data. The generative AI model processes this information and outputs a list of the most suitable hot spring resorts for the user. The output is a list of recommended hot spring resorts.

[0805] Step 4: Generate a virtual hot spring tour

[0806] server

[0807] The server generates a virtual hot spring tour based on the data of the selected hot spring resort. The input data is detailed information about the recommended hot spring resort. Unity software is used to generate 3D models and environmental acoustic data to create the virtual hot spring tour. The output is the data of the virtual hot spring tour. Specifically, a 3D model of the hot spring resort is constructed in Unity and sound effects are incorporated.

[0808] Step 5: Providing a virtual hot spring experience

[0809] Terminal

[0810] The user's device (smartphone, tablet, VR device, etc.) receives the virtual hot spring tour provided by the server and provides the user with a virtual reality hot spring experience. The input data is the virtual hot spring tour data sent from the server. The device receives this data, renders a 3D model, and plays video and audio data. The output is the user's ability to enjoy the virtual hot spring experience in VR or on their smartphone.

[0811] Step 6: Travel plan generation and support

[0812] server

[0813] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a travel plan to that hot spring resort. The input data is information about the hot spring resort selected by the user. The server generates a travel plan including information about accommodation, transportation, and surrounding tourist spots, and proposes it to the user. The output is a specific travel plan. Specific operations include searching for various information and creating an appropriate plan.

[0814] Step 7: Share and interact with your virtual hot spring experience

[0815] User

[0816] Users share screenshots and videos of their virtual hot spring experience on social media or in-app communities. The input data are images and videos taken during the virtual hot spring experience. Other users can comment on the posted content and share the same virtual hot spring experience. The output is to encourage interaction between users. Specific operations utilize the posting and commenting functions on social media and within the app.

[0817] Prompt Sentence Examples

[0818] "Data collection for hot spring tour experience:

[0819] Users like the open-air baths, relaxation, and family-friendly features.

[0820] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0821] The present invention provides a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home, plan trips to their favorite hot spring resorts, and dynamically adjust the content of the experience by recognizing the user's emotions. An embodiment of this system is described in detail below.

[0822] System Configuration

[0823] The system mainly consists of the following four main components:

[0824] 1. Server

[0825] 2. Terminal

[0826] 3. Users

[0827] 4. Emotion Engine

[0828] Implementation flow and operation of each element

[0829] 1. Hot spring data collection and mapping

[0830] server

[0831] The server collects data about hot springs from various sources (official websites, review sites, APIs, etc.) The collected data includes location information, the type of hot spring, its characteristics, information about surrounding facilities, photos, and videos.

[0832] The collected data is organized and stored in a database that is regularly updated to reflect new and changed hot spring information.

[0833] 2. Input of user attribute information and recommendations

[0834] User

[0835] When logging in to the system for the first time, users enter their personal information, including their preferred hot spring features (e.g., open-air bath, sauna, family-friendly), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[0836] server

[0837] The server reads the attribute information entered by the user and runs an algorithm to match it with hot spring data stored in a database, thereby creating a list of hot spring candidates that are most suitable for the user and recommending them.

[0838] 3. Emotional engine for recognizing user emotions

[0839] Emotion Engine

[0840] While the user is entering their attribute information and experiencing the virtual hot spring tour, an emotion engine is activated that recognizes emotions from the user's facial expressions and voice. The emotion engine acquires emotion data and analyzes it in real time.

[0841] server

[0842] It receives emotional data sent from the emotion engine and dynamically adjusts recommendations and changes the content of the virtual experience.

[0843] 4. Creating a Virtual Hot Spring Tour

[0844] server

[0845] The server generates a virtual hot spring tour based on the selected hot springs, using 3D models, video footage of the actual hot springs, and audio data. The experience is customized according to the user's emotional state, as recognized by the emotion engine.

[0846] Terminal

[0847] The generated virtual hot spring tour is provided to the user's device, and the user can start the virtual hot spring tour on the device and enjoy the experience as if they were in a real hot spring.

[0848] 5. Travel planning generation and support

[0849] server

[0850] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a travel plan to that hot spring resort, including details on accommodation, transportation, surrounding tourist attractions, restaurants, etc.

[0851] User

[0852] The user can review the proposed travel plan and make accommodation reservations, transportation arrangements, etc. based on the plan.

[0853] During the actual trip, the application can be used to provide navigation and tourist information.

[0854] 6. Sharing and interacting with virtual hot spring experiences

[0855] User

[0856] Users can post screenshots and videos of their virtual hot spring experience on social media and in-app communities.

[0857] Other users can comment on the posts and share the same virtual hot spring experience.

[0858] Specific examples

[0859] Below are some specific examples of how the system can be used.

[0860] Example 1:

[0861] User A likes open-air baths at hot springs and wants to travel with his family. When he logs in for the first time, he enters this information as attributes.

[0862] The server uses its database to recommend hot spring resorts with open-air baths suitable for families in areas rich in nature.

[0863] User A chooses one of the recommended hot springs and experiences a virtual hot spring tour. The emotion engine recognizes that User A is relaxed from his / her facial expressions and voice, and adds further relaxing elements.

[0864] If user A likes the hot spring resort, the server generates a specific travel plan to the hot spring resort and proposes it to user A.

[0865] Example 2:

[0866] User B uses a smartphone app to select a virtual hot spring tour to enjoy the virtual reality experience of a hot spring at the summit of Mount Fuji.

[0867] The server combines the 3D model of Mount Fuji with the model data of the hot spring and provides it to User B's terminal.

[0868] When User B puts on the VR glasses and begins the experience, the emotion engine recognizes excitement and surprise and adjusts the experience to make it even more exciting.

[0869] In this way, by using this system, users can enjoy a variety of hot spring experiences from the comfort of their own home, find the perfect hot spring resort, and plan an actual trip. Furthermore, by combining it with an emotion engine, it is possible to provide an experience optimized for the user's emotional state, and share the virtual hot spring experience with other users, creating a new platform for interaction.

[0870] The processing flow will be explained below.

[0871] Step 1:

[0872] server

[0873] Start collecting hot spring data. The server automatically collects hot spring data (location information, type of hot spring, characteristics, information on surrounding facilities, photos, videos) from various sources such as official websites, review sites, and APIs.

[0874] Step 2:

[0875] server

[0876] The collected hot spring data is organized and stored in a database. Location information, hot spring characteristics, and information on surrounding facilities are converted into an appropriate format and centrally managed.

[0877] Step 3:

[0878] User

[0879] When logging in to the system for the first time, users enter their demographic information, including their preferred hot spring features (e.g., open-air bath, sauna, family-friendly, etc.), travel style (e.g., relaxation, sightseeing, activities, etc.), budget, and travel companions.

[0880] Step 4:

[0881] server

[0882] The server analyzes the attribute information entered by the user, and then executes a matching algorithm based on the hot spring data in the database and the user's attribute information to generate the most suitable hot spring candidates.

[0883] Step 5:

[0884] server

[0885] The generated list of hot spring candidates is recommended to the user, allowing the user to check a list of hot springs that suit their preferences.

[0886] Step 6:

[0887] User

[0888] Check the list of recommended hot springs through your device and select the hot spring resort that interests you.

[0889] Step 7:

[0890] Emotion Engine

[0891] When a user selects a hot spring resort or experiences a virtual hot spring tour, the system analyzes the user's facial expressions and voice in real time to generate emotional data.

[0892] Step 8:

[0893] server

[0894] It receives emotional data from the emotion engine and dynamically adjusts the hot spring recommendation. For example, if the user is excited, it will suggest an active experience, and if the user is relaxed, it will suggest a quiet hot spring spot.

[0895] Step 9:

[0896] server

[0897] It generates a virtual hot spring tour based on the selected hot spring, integrating 3D models, actual video and audio data, and customizing the experience based on emotional data.

[0898] Step 10:

[0899] Terminal

[0900] The generated virtual hot spring tour data is received and provided to the user, who then begins the virtual hot spring tour using a smartphone or VR glasses.

[0901] Step 11:

[0902] User

[0903] Experience the virtual hot spring tour and enjoy the atmosphere of the hot spring resort. The emotion engine continuously analyzes the user's emotions and adjusts the experience in real time.

[0904] Step 12:

[0905] User

[0906] After experiencing the virtual hot spring tour, users can select their favorite hot spring resort. The emotion engine analyzes the user's satisfaction and uses the results to provide feedback for future recommendations and experience adjustments.

[0907] Step 13:

[0908] server

[0909] Generate a specific itinerary to the selected hot spring resort, including details on accommodation, transportation, nearby tourist attractions, restaurants, etc.

[0910] Step 14:

[0911] server

[0912] The generated travel plan is proposed to the user, who can then confirm the proposed plan and proceed with reservations and arrangements.

[0913] Step 15:

[0914] User

[0915] Use the application during your actual trip to receive real-time support for navigation and tourist information.

[0916] Step 16:

[0917] User

[0918] Post screenshots and videos of your virtual hot spring experience on social media and in-app communities to interact with other users.

[0919] Step 17:

[0920] User

[0921] Users can view virtual hot spring experiences posted by other users and create a new forum for interaction through comments and reactions.

[0922] In this way, this system starts by collecting hot spring data, then provides recommendations based on the user's attribute information and emotional data, creates and experiences a virtual hot spring tour, proposes specific travel plans, and finally shares and interacts with the virtual hot spring experience, all in a single flow.

[0923] Example 2

[0924] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0925] Conventional hot spring information systems make it difficult for users to enjoy a variety of hot spring experiences from home. Furthermore, they lacked the ability to dynamically adjust the experience based on the user's emotional state, which prevented them from fully improving user satisfaction. Furthermore, it was time-consuming and inefficient for users to plan specific trips to their favorite hot spring resorts.

[0926] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0927] In this invention, the server includes means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database, means for inputting user attribute information and recommending optimal hot springs based on the attribute information, means equipped with an emotion engine that acquires user emotion data and analyzes it in real time and dynamically adjusts recommendations and virtual experience content based on the emotion data, means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal, means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user, and means for sharing information about the virtual hot spring experience and the actual trip with other users and promoting communication. This allows users to enjoy a variety of hot spring experiences from the comfort of their own home, dynamically adjust the experience according to their emotional state, and efficiently plan their travel by finding the optimal hot spring resort.

[0928] "Hot spring data" is information about hot spring facilities, including location information, characteristics, information about surrounding facilities, photos, videos, and the like.

[0929] The "database" is an information management system for organizing and storing collected hot spring data in a unified format.

[0930] "User" refers to an individual who uses the system to receive hot spring information recommendations and experience virtual hot spring tours.

[0931] "Attribute information" refers to personal data such as the user's preferences, travel style, budget, and traveling companions.

[0932] "Recommendation" refers to the act of recommending the most suitable hot spring facility based on the user's attribute information.

[0933] The "emotion engine" is a system that acquires emotional data from the user's facial expressions, voice, etc. and analyzes it in real time.

[0934] The "Virtual Hot Spring Tour" is virtual reality content generated using 3D models, actual hot spring footage, and audio data, allowing users to enjoy a hot spring experience on their device.

[0935] A "terminal" is a device used by a user to experience a virtual hot spring tour, and includes a VR device, a mobile terminal, etc.

[0936] A "travel plan" is a detailed plan that automatically generates a specific travel schedule to a user's favorite hot spring resort, including information on accommodation, transportation, and surrounding tourist attractions.

[0937] "Information sharing" refers to the act of sharing information about virtual hot spring experiences and real trips with other users to promote interaction.

[0938] "Dynamic adjustment" refers to changing recommendations and virtual experience content in real time based on the user's emotional state.

[0939] A "hot spring facility" is a place that provides hot bathing services and includes multiple types of baths, such as open-air baths and saunas.

[0940] The present invention is a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. This system is primarily composed of four main components: a server, a terminal, a user, and an emotion engine. An embodiment of this system is described in detail below.

[0941] Hot spring data collection

[0942] server

[0943] The server collects information about hot springs through external APIs such as the Google Places API and TripAdvisor API. The collected data includes the hot spring's name, location, type and characteristics, information about surrounding facilities, photos, and videos. The collected data is organized using a Python script and stored in a MySQL database. This process is performed periodically, and new information is automatically updated.

[0944] Input of user attribute information and recommendations

[0945] User

[0946] When users log in to the system for the first time, they enter attribute information such as their preferred hot spring characteristics, travel style, budget, and travel companion information through a dedicated form. This form is built with React.js.

[0947] server

[0948] The server parses the information entered by the user in JSON format and stores it in a database. Then, using machine learning algorithms such as Pandas and Scikit-learn, it matches this attribute information with hot spring data. The algorithm then lists and recommends the hot springs that are most suitable for the user. This recommendation is made in the form of providing a list of the top five hot spring facilities based on the filter criteria.

[0949] Recognizing user emotions with an emotion engine

[0950] Emotion Engine

[0951] While the user is entering their personal information or experiencing the virtual hot spring tour, their facial expressions and voice are captured via a webcam and microphone. Using AWS Rekognition and Google Cloud Vision, emotions are analyzed in real time. This emotional data is then sent to a server.

[0952] server

[0953] The server analyzes the data received from the emotion engine and dynamically adjusts recommendations and the content of the virtual experience according to the user's emotional state. For example, if the server detects that the user is relaxed, it will change the background music to soothing music.

[0954] Creating a virtual hot spring tour

[0955] server

[0956] Based on the selected hot spring, the server uses Unity or Unreal Engine to generate a 3D model, video footage of the hot spring, and audio data, which is then sent to the user's device.

[0957] Terminal

[0958] The device displays a 3D virtual hot spring tour on devices such as VR glasses or smartphones, allowing users to experience the hot springs as if they were actually there.

[0959] Generate specific travel plans

[0960] server

[0961] Once the user has experienced a virtual hot spring tour and selected their favorite hot spring, the server automatically generates a travel plan using the Google Maps API and Skyscanner API to collect information on accommodation, transportation, and surrounding tourist spots, and provides the user with a travel plan that is tailored to their needs.

[0962] User

[0963] Users can view the provided itinerary, book accommodation and arrange transportation based on the plan, and use the application to receive real-time navigation and tourist information during their trip.

[0964] Sharing and interacting with virtual hot spring experiences

[0965] User

[0966] Users can take screenshots and videos of their virtual hot spring experience and post them on social media or in the in-app community. Users can also deepen their interactions by commenting on and sharing content posted by other users.

[0967] Specific usage examples and prompt sentence examples

[0968] Example 1

[0969] If User A likes open-air baths and wants to go on a family trip, the server will recommend hot spring resorts with family-friendly open-air baths. User A then experiences a virtual tour of the selected hot spring resort, and if the experience is deemed relaxing, further relaxing elements are added. Finally, the server generates a specific travel plan, allowing User A to efficiently prepare for the trip.

[0970] Example 2

[0971] If User B wishes to experience a virtual hot spring at the top of Mt. Fuji using a smartphone app, the server will provide a combination of a 3D model of Mt. Fuji and hot spring data. User B puts on the VR glasses and begins the virtual experience. If excitement and surprise are recognized, the experience will be adjusted to make it even more exciting.

[0972] Prompt Sentence Examples

[0973] "If the user likes open-air baths and family trips, create a list of recommended hot spring resorts and generate a virtual hot spring tour. Then analyze the user's emotions and tailor the experience accordingly."

[0974] As described above, by using this system, users can enjoy a variety of hot spring experiences from the comfort of their own home, and receive the optimal experience according to their emotional state. It also enables users to find the best hot spring resort and efficiently plan their trip.

[0975] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0976] Step 1: Collecting hot spring data

[0977] The server sends requests to external APIs such as the Google Places API and TripAdvisor API. The input is keywords related to the specified area and hot spring facility. The output is data such as the hot spring facility's name, location, features, information on surrounding facilities, photos, and videos. This data is collected using a Python script, checked for duplicate data, and standardized data formats before being stored in a MySQL database. A job scheduler is used to perform regular updates.

[0978] Step 2: Enter user attribute information

[0979] When users log in to the system for the first time, they enter attribute information such as their preferred hot spring characteristics, travel style, budget, and travel companion information through a dedicated form built with React.js. The information entered is then sent to the server in JSON format.

[0980] Step 3: Store user attribute information

[0981] The server parses the JSON-formatted attribute information received from the user, extracts the necessary fields, and saves them in a MySQL database. The input is JSON data of attribute information, and the output is a database containing organized attribute data.

[0982] Step 4: Run the recommendation algorithm

[0983] The server uses Pandas and Scikit-learn to run an algorithm that matches the user's attribute information with hot spring data. The input is the user's attribute information and hot spring data. The output is a list of hot spring facilities that are best suited to the user. Filter conditions are applied, and the hot spring facilities that match the conditions are scored and the top five are presented to the user.

[0984] Step 5: Acquire and analyze emotion data

[0985] The emotion engine captures facial expressions and voices via a webcam and microphone while the user is entering attribute information or experiencing a virtual hot spring tour. The input is the captured facial image and voice data. This is sent to AWS Rekognition or Google Cloud Vision to obtain emotion data. The output is analyzed emotion data.

[0986] Step 6: Use emotion data

[0987] The server receives emotional data from the emotion engine and dynamically adjusts the recommendations and virtual experience content. The input is the emotional data. The output is the adjusted recommendations and virtual experience content. For example, if the emotion is determined to be relaxing, the background music is changed to soothing music.

[0988] Step 7: Generate a virtual hot spring tour

[0989] The server uses Unity or Unreal Engine to generate a 3D model based on the selected hot spring, as well as actual hot spring video and audio data. The input is the hot spring data. The output is the generated virtual hot spring tour data, which is then sent to the user's device.

[0990] Step 8: Run a virtual hot spring tour

[0991] The terminal provides users with a virtual hot spring tour using devices such as VR glasses or a smartphone. The input is virtual hot spring tour data provided by the server. Users can enjoy an interactive experience as if they were actually in the hot spring resort. The output is the user's experience data.

[0992] Step 9: Generate a travel plan

[0993] When the user selects their favorite hot spring resort, the server automatically generates a specific travel plan using the Google Maps API and Skyscanner API. The input is information about the selected hot spring resort and the user's attribute information. The output is a travel plan that includes information about accommodations, transportation, surrounding tourist attractions, and restaurants.

[0994] Step 10: Travel Plan Proposal

[0995] The user can review the proposed itinerary and make accommodation reservations and transportation arrangements. The input is the itinerary data provided by the server. The output is the user's reservation information. During the trip, the user can also use the application to receive real-time navigation and tourist information.

[0996] Step 11: Share and interact with your virtual hot spring experience

[0997] Users can post screenshots and videos of their virtual hot spring experience on social media and in-app communities. The input is the user's experience data. The output is the shared information and feedback from other users. Users can interact with other users and share their comments and experiences.

[0998] (Application example 2)

[0999] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1000] In today's busy lifestyles, users lack the means to easily plan a hot spring trip to relax and refresh. Furthermore, existing systems struggle to provide a customized experience that takes into account the user's emotions. Furthermore, they lack the ability to allow users to share their virtual experiences and interact with other users. To address these issues, a system is needed that dynamically adjusts the user's experience using emotion recognition to provide an authentic virtual hot spring experience.

[1001] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1002] In this invention, the server includes: means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database; means for inputting user attribute information and recommending optimal hot springs based on the attribute information; means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal; means for recognizing the user's emotions in real time and dynamically adjusting the content of the virtual hot spring tour based on the data; means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user; and means for sharing information about the virtual hot spring experience and the actual travel with other users and promoting interactions. This allows users to enjoy various hot spring experiences from the comfort of their own homes and find the optimal hot spring resort through experiences customized according to their emotions. Furthermore, users can obtain new information and share travel plans through interactions with other users.

[1003] "Hot spring data" is data including location information, characteristics, and information on surrounding facilities related to hot springs.

[1004] "Location information" is information that indicates the geographical location of a hot spring.

[1005] "Characteristics" is information indicating the type and characteristics of the hot spring, the details of the facilities, and so on.

[1006] "Information about surrounding facilities" refers to information about accommodations, tourist spots, restaurants, and the like located near the hot spring.

[1007] A "database" is a storage system in which collected data is systematically stored so that it can be managed and searched.

[1008] "User attribute information" is information input by the user regarding hot spring preferences, travel style, budget, and traveling companions.

[1009] "Recommendation" is a process that suggests optimal options based on user attribute information.

[1010] A "virtual hot spring tour" is a virtual hot spring experience that uses 3D models of hot spring resorts and environmental acoustic data.

[1011] "Terminal" refers to the device that a user uses to use the system, including smartphones, tablets, VR devices, etc.

[1012] "Emotion recognition" is a technology that analyzes emotions from a user's facial expressions and voice in real time.

[1013] A "travel plan" is a plan that includes a specific travel schedule to a hot spring resort, as well as information on accommodation, transportation, and surrounding tourist attractions.

[1014] "Interaction" means that users share information and communicate with each other.

[1015] MODE FOR CARRYING OUT THE INVENTION

[1016] This invention is a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. Furthermore, the system can recognize the user's emotions and dynamically adjust the content of the experience. The following describes an embodiment of the invention.

[1017] System Configuration

[1018] The system consists of four main components:

[1019] 1. Server

[1020] 2. Terminal

[1021] 3. Users

[1022] 4. Emotion Engine

[1023] The specific operation of each element will be explained below.

[1024] Hot Spring Data Collection and Mapping

[1025] The server collects data about hot springs from various sources, including their locations, characteristics, and surrounding facilities. This information is stored in a database and updated periodically.

[1026] Input of user attribute information and recommendations

[1027] When a user logs in to the system for the first time, they enter their own attribute information, including their preferred hot spring characteristics, travel style, budget, and travel companions. The server uses this information to run an algorithm that matches it with hot spring data stored in the database and recommends the most suitable hot springs to the user.

[1028] Recognizing user emotions with an emotion engine

[1029] The emotion engine recognizes emotions from the user's facial expressions and voice in real time while the user is entering attribute information or experiencing a virtual hot spring tour. The recognized emotion data is sent to the server and used to dynamically adjust the experience.

[1030] Creating and providing virtual hot spring tours

[1031] The server generates a virtual hot spring tour based on the selected hot springs, using 3D models of the hot springs and environmental acoustic data. It also has a function to customize the experience depending on the user's emotional state. The generated virtual hot spring tour is provided to the user's devices, such as smartphones, smart glasses, and head-mounted displays.

[1032] Specific examples

[1033] For example, User A likes open-air baths and is planning a trip with his or her family. When User A enters this attribute information when logging in for the first time, the server recommends the most suitable hot spring resort from the database. When User A selects the recommended hot spring resort and starts the virtual hot spring tour, the emotion engine recognizes User A's relaxation state and adjusts the experience to make it even more relaxing.

[1034] Hardware and software used

[1035] Hardware: Smartphones, smart glasses, head-mounted displays, webcams

[1036] Software: OpenCV (cv2), DeepFace, Python standard library

[1037] Prompt Sentence Examples

[1038] Below are some example prompts to input to a generative AI model:

[1039] "How can I implement real-time emotion recognition in Python to customize the user experience for a hot spring experience app? I'd like code that meets the following criteria:"

[1040] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1041] Step 1:

[1042] The server collects hot spring data and stores information on its location, characteristics, and surrounding facilities in a database. This process involves collecting data about hot springs from official websites, review sites, APIs, etc., properly formatting it, and storing it in the database. The input is data from external sources, and the output is formatted hot spring data.

[1043] Step 2:

[1044] When a user logs in for the first time, they enter attribute information such as their hot spring preferences, travel style, budget, and travel companions. The entered attribute information is saved in the database as a user profile. The input is the attribute information provided by the user, and the output is the user attribute information saved in the database.

[1045] Step 3:

[1046] The server recommends the most suitable hot spring resort from the hot spring data stored in the database based on the user's attribute information. The recommendation algorithm compares the user profile with the hot spring data and generates a list of matching hot spring resorts. The input is the user's attribute information and the hot spring data, and the output is a list of recommended hot spring resorts.

[1047] Step 4:

[1048] The server generates a virtual hot spring tour based on hot springs selected from the recommended hot spring resorts. This tour is designed using 3D models of hot springs and environmental acoustic data. The input is information about the selected hot spring resorts, and the output is the generated virtual hot spring tour data.

[1049] Step 5:

[1050] The terminal provides the generated virtual hot spring tour to the user. The user can experience the virtual hot spring tour using devices such as smartphones, smart glasses, and head-mounted displays. The input is the virtual hot spring tour data provided by the server, and the output is the virtual experience through the user's vision and hearing.

[1051] Step 6:

[1052] When a user experiences a virtual hot spring tour, the emotion engine recognizes the user's facial expressions and voice in real time and analyzes the emotional data. The analyzed emotional data is sent to the server. The input is the user's facial expressions and voice data, and the output is the analyzed emotional data.

[1053] Step 7:

[1054] The server dynamically adjusts the content of the virtual hot spring tour based on the received emotion data. For example, if the user is relaxed, it adds more relaxing content, and if the user is excited, it strengthens exciting elements. The input is the emotion data sent from the emotion engine, and the output is the adjusted virtual hot spring tour data.

[1055] Step 8:

[1056] When a user selects a hot spring resort they like, the server automatically generates a specific travel plan to that hot spring resort. This plan includes information on transportation, accommodation, surrounding tourist attractions, etc. The input is the user's selection information, and the output is the generated travel plan.

[1057] Step 9:

[1058] Users can review the proposed itinerary and make any necessary reservations. They can also share and interact with other users about their virtual hot spring experiences and travel information. The input is the proposed itinerary and the user's decision information, and the output is the shared experience information.

[1059] Step 10:

[1060] The terminal collects input from the user (attribute information, emotion data, selection information, etc.) and sends it to the server. It also displays data received from the server (recommendations, virtual tour data, travel plans, etc.). The input is based on data communication between the user and the server, and the output is the overall service experience provided to the user.

[1061] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1062] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1063] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1064] [Third embodiment]

[1065] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1066] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1067] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1068] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1069] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1070] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1071] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1072] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1073] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1074] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1075] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1076] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1077] The present invention provides a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. An embodiment of this system will be described in detail below.

[1078] System Configuration

[1079] The system mainly consists of the following three main components:

[1080] 1. Server

[1081] 2. Terminal

[1082] 3. Users

[1083] Implementation flow and operation of each element

[1084] 1. Hot spring data collection and mapping

[1085] server

[1086] The server collects data about hot springs from various sources (official websites, review sites, APIs, etc.) The collected data includes location information, the type of hot spring, its characteristics, information about surrounding facilities, photos, and videos.

[1087] The collected data is organized and stored in a database that is regularly updated to reflect new and changed hot spring information.

[1088] 2. Input of user attribute information and recommendations

[1089] User

[1090] When users log in to the system for the first time, they enter attribute information such as their preferred hot spring characteristics (e.g., open-air bath, sauna, family-friendly, etc.), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[1091] server

[1092] The server reads the attribute information entered by the user and runs an algorithm to match it with hot spring data stored in a database, thereby creating a list of hot spring candidates that are most suitable for the user and recommending them.

[1093] 3. Creating a Virtual Hot Spring Tour

[1094] server

[1095] Based on the selected hot springs, the server generates a virtual hot spring tour using 3D models, footage of the actual hot springs, and audio data.

[1096] This data is then processed for virtual reality (VR) experiences and visual experiences on mobile devices.

[1097] Terminal

[1098] The generated virtual hot spring tour is provided to the user's device, and the user can start the virtual hot spring tour on the device and enjoy the experience as if they were in a real hot spring.

[1099] 4. Providing a virtual hot spring experience using smartphones and VR glasses

[1100] Terminal

[1101] The user launches the application on their smartphone or VR glasses and accesses the virtual hot spring tour provided by the server.

[1102] The device receives data from the server and plays 3D models and sound data to provide the user with a virtual experience.

[1103] 5. Travel planning generation and support

[1104] server

[1105] When a user experiences a virtual hot spring tour and selects a hot spring resort that they like, the server automatically generates a travel plan to that hot spring resort.

[1106] The travel plan includes details such as accommodation information, transportation options, information on nearby tourist attractions, and restaurants.

[1107] User

[1108] The user can review the proposed travel plan and make accommodation reservations, transportation arrangements, etc. based on the plan.

[1109] During the actual trip, the application can be used to provide navigation and tourist information.

[1110] 6. Sharing and interacting with virtual hot spring experiences

[1111] User

[1112] Users can post screenshots and videos of their virtual hot spring experience on social media and in-app communities.

[1113] Other users can comment on the posts and share the same virtual hot spring experience.

[1114] Specific examples

[1115] Below are some specific examples of how the system can be used.

[1116] Example 1:

[1117] User A likes open-air baths at hot springs and wants to travel with his family. When he logs in for the first time, he enters this information as attributes.

[1118] The server uses its database to recommend hot spring resorts with open-air baths suitable for families in areas rich in nature.

[1119] When User A selects one of the recommended hot springs and experiences a virtual hot spring tour, a 3D model of the hot spring resort and actual video and audio data are displayed on the device.

[1120] If user A likes the hot spring resort, the server generates a specific travel plan to the hot spring resort and proposes it to user A.

[1121] Example 2:

[1122] User B uses a smartphone app to select a virtual hot spring tour to enjoy the virtual reality experience of a hot spring at the summit of Mount Fuji.

[1123] The server combines the 3D model of Mount Fuji with the model data of the hot spring and provides it to User B's terminal.

[1124] User B puts on the VR glasses and enjoys the experience of soaking in a hot spring at the top of Mount Fuji.

[1125] In this way, by using this system, users can enjoy a variety of hot spring experiences from the comfort of their own home, find the perfect hot spring resort, and plan an actual trip. It is also possible to share virtual hot spring experiences with other users and create new opportunities for interaction.

[1126] The processing flow will be explained below.

[1127] Step 1:

[1128] server

[1129] Start collecting hot spring data. The server automatically collects hot spring data (location information, type of hot spring, characteristics, information on surrounding facilities, photos, videos) from various sources such as official websites, review sites, and APIs.

[1130] Step 2:

[1131] server

[1132] Organize the collected hot spring data. Convert the location information, hot spring characteristics, and information on surrounding facilities into an appropriate format and store it in a database.

[1133] Step 3:

[1134] User

[1135] When logging in to the system for the first time, users enter their personal information, including their preferred hot spring features (e.g., open-air bath, sauna, family-friendly), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[1136] Step 4:

[1137] server

[1138] The server analyzes the attribute information entered by the user, runs an algorithm to match the attribute information with hot spring data in the database, and generates a list of optimal hot spring candidates.

[1139] Step 5:

[1140] server

[1141] The generated list of hot spring candidates is sent to the terminal for recommendation to the user, allowing the user to view a list of hot springs that are best suited to them.

[1142] Step 6:

[1143] User

[1144] Check the list of recommended hot springs through your device and select the hot spring resort that interests you.

[1145] Step 7:

[1146] server

[1147] A virtual hot spring tour is generated based on the selected hot springs. The 3D model, video footage of the actual hot springs, and audio data are integrated to create the virtual hot spring tour data.

[1148] Step 8:

[1149] Terminal

[1150] The generated virtual hot spring tour data is received and provided to the user, who then begins the virtual hot spring tour using a smartphone or VR glasses.

[1151] Step 9:

[1152] User

[1153] Take a virtual hot spring tour and enjoy the atmosphere of the hot spring resort, as if you were actually there.

[1154] Step 10:

[1155] User

[1156] After experiencing the virtual hot spring tour, choose your favorite hot spring resort.

[1157] Step 11:

[1158] server

[1159] Generate a specific itinerary to the selected hot spring resort, including details on accommodation, transportation, nearby tourist attractions, restaurants, etc.

[1160] Step 12:

[1161] server

[1162] The generated travel plan is proposed to the user, who can then review the proposed plan and proceed with reservations and arrangements as necessary.

[1163] Step 13:

[1164] User

[1165] Use the application during your actual trip to receive real-time support for navigation and tourist information.

[1166] Step 14:

[1167] User

[1168] Post screenshots and videos of your virtual hot spring experience on social media and in-app communities to interact with other users.

[1169] Step 15:

[1170] User

[1171] Users can view virtual hot spring experiences posted by other users and create a new forum for interaction through comments and reactions.

[1172] As described above, this system starts by collecting hot spring data, then provides recommendations based on the user's attribute information, creates and experiences a virtual hot spring tour, proposes specific travel plans, and finally shares and interacts with the virtual hot spring experience, all in a single flow.

[1173] Example 1

[1174] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1175] Conventional hot spring information systems provide insufficient support for users to find hot spring resorts and often lack the data and algorithms required to provide virtual hot spring experiences. As a result, users have difficulty efficiently finding hot spring resorts that suit their preferences and obtaining detailed information through virtual hot spring experiences. Furthermore, the systems do not provide comprehensive information necessary for planning actual trips, which results in insufficient user convenience. Furthermore, the virtual hot spring experience does not promote interaction between users.

[1176] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1177] In this invention, the server includes: means for collecting hot spring information and storing location data, facility features, and information on surrounding services in a database; means for inputting user attribute information and recommending optimal hot springs based on the attribute information; means for organizing and frequently updating the collected hot spring information; means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal; means for generating 3D models and environmental acoustic data for the virtual hot spring tour; means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user; means for providing a virtual hot spring experience using a smartphone or VR device; and means for sharing information about the virtual hot spring experience and actual travel with other users and promoting communication. This allows users to enjoy a variety of hot spring experiences from the comfort of their own homes, find the perfect hot spring, and efficiently plan their actual travel. Furthermore, the virtual hot spring experience also promotes communication with other users, stimulating the sharing of hot spring information.

[1178] "Hot spring information" is comprehensive data including location data for hot spring facilities, the type of hot spring, its characteristics, information on surrounding services, and images and videos of the facility.

[1179] "User attribute information" is information relating to the user's preferences and needs, such as the characteristics of the hot springs the user prefers, travel style, budget, and traveling companions.

[1180] The "database" is an information management system that systematically stores and manages collected hot spring information and user attribute information, and allows searching and updating as needed.

[1181] "Recommendation" is the act of selecting the most suitable hot spring facility based on the user's attribute information and suggesting it to the user.

[1182] A "virtual hot spring tour" is a digital tour generated based on a selected hot spring, virtually recreating an actual hot spring experience using 3D models and acoustic data.

[1183] A "3D model" is digital data that recreates the selected hot spring facility and its surroundings in three dimensions, providing a three-dimensional visual experience in a virtual space.

[1184] "Environmental acoustic data" is the actual acoustic environment of selected hot spring facilities that has been recorded and saved in digital format, allowing for an auditory experience in a virtual space.

[1185] A "travel plan" is a comprehensive travel schedule that includes information on accommodations, transportation, surrounding tourist spots, and restaurants that a user needs when visiting a specific hot spring resort.

[1186] A "smartphone" is a portable telecommunications device that provides advanced computing power and a wide variety of features in addition to the functionality of a mobile phone.

[1187] A "VR device" is hardware that provides a virtual reality experience, allowing users to visually explore digital spaces freely.

[1188] "Promoting interaction" means strengthening connections between users by allowing multiple users to share information and exchange opinions through the system.

[1189] The present invention is a system that allows users to enjoy various hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. The system consists of three main components, which are detailed below:

[1190] 1. Server

[1191] 2. Terminal

[1192] 3. Users

[1193] Server Operation

[1194] Hot spring information collection and mapping

[1195] The server collects data from various sources, such as official hot spring websites, review websites, and APIs. The collected data includes hot spring location information, type, characteristics, information on surrounding facilities, and photos and videos. For example, from the official website of a hot spring facility, the server collects location information (latitude and longitude), hot spring water quality (sulfur spring, carbonated spring, etc.), and characteristics (open-air bath, family bath, with sauna), etc.

[1196] The server organizes the collected data and stores it in a database that is automatically updated to reflect new or changed hot spring information.

[1197] Input of user attribute information and recommendations

[1198] When users log in to the system for the first time, they enter information such as their preferred hot spring characteristics (e.g., open-air bath, sauna, family-friendly, etc.), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[1199] The server reads the attribute information entered by the user and matches it with hot spring information in the database. This algorithm recommends hot springs that best fit the user's preferences.

[1200] Creating a virtual hot spring tour

[1201] The server generates a virtual hot spring tour based on the selected hot spring resort using 3D models, video footage of the actual hot spring, and audio data. For example, it collects and processes 360-degree images of an open-air bath and audio data of the hot spring for VR devices.

[1202] The terminal provides the generated virtual hot spring tour data to the user, and the user can start and enjoy the virtual hot spring tour on the terminal.

[1203] Device behavior

[1204] Providing a virtual hot spring experience

[1205] The device plays a virtual hot spring tour using a smartphone or VR glasses. For example, when a user starts the application using a smartphone or VR glasses, it receives data provided by the server and plays 3D models and sound data to provide a virtual experience.

[1206] User Actions

[1207] Travel plan generation and support

[1208] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a specific travel plan to that hot spring resort, including information on accommodations, transportation, surrounding tourist spots, and restaurants.

[1209] The user can review the proposed itinerary and make accommodation reservations and transportation arrangements, for example by directly accessing a hotel booking site through the application to complete the accommodation arrangements.

[1210] Sharing and interacting with virtual hot spring experiences

[1211] Users can post screenshots and videos of their virtual hot spring experience on social media or in the app's community to share with other users, who can interact with the posts by commenting or liking them.

[1212] Specific examples

[1213] Below are some specific examples of how the system can be used.

[1214] Example 1:

[1215] User A likes outdoor baths and wants to travel with his family. He enters this information into the system when he logs in for the first time.

[1216] The server uses its database to recommend hot spring resorts with open-air baths suitable for families in areas rich in nature.

[1217] The terminal plays the virtual hot spring tour data provided by the server, and User A enjoys the 360-degree video and audio data of the open-air bath.

[1218] User A likes the experience and makes accommodation reservations and transportation arrangements based on the travel plan suggested by the system.

[1219] Example 2:

[1220] User B uses VR glasses to enjoy a virtual hot spring experience. For example, he wants to experience soaking in a hot spring at the top of Mt. Fuji.

[1221] The server generates virtual hot spring tour data using a 3D model of Mount Fuji and video and audio data of the hot springs, and provides it to the terminal.

[1222] The terminal plays back the virtual hot spring tour data to User B, who then wears the VR glasses and enjoys the hot spring experience at the top of Mount Fuji.

[1223] Prompt Sentence Examples

[1224] Below are examples of prompts for the generative AI model based on the specific features of this system.

[1225] Prompt: I'd like to travel with my family and I like places with lots of nature. I'm also looking for a hot spring resort with an open-air bath. What kind of hot spring resorts are there?

[1226] Prompt: I want to have a virtual reality experience of soaking in a hot spring at the top of Mt. Fuji. I'd like you to generate a virtual hot spring tour for that purpose.

[1227] In this way, this system allows users to enjoy a variety of hot spring experiences from the comfort of their own home, helping them not only to find the best hot spring resort but also to efficiently plan their actual trip. It also promotes interaction with other users through the virtual hot spring experience, stimulating the sharing of hot spring information.

[1228] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1229] Step 1:

[1230] Hot spring information collection and mapping

[1231] The server obtains data from various sources, including official hot spring websites, review sites, and APIs.

[1232] Input: Sources such as official website URL, review site data, API key, etc.

[1233] Processing: Access these sources and obtain hot spring location information (latitude and longitude), hot spring type (sulfur spring, carbonated spring, etc.), facility features (open-air bath, sauna, etc.), information on surrounding facilities, and photos and videos via scraping or API.

[1234] Output: The collected hot spring information is organized and stored in a database. For example, detailed information about a specific hot spring facility is saved in the database.

[1235] Step 2:

[1236] Input of user attribute information and recommendations

[1237] When users log in for the first time, they enter information into the system such as their preferred hot spring characteristics, travel style, budget, and travel companions.

[1238] Input: User preferences and travel wishes (e.g., options such as "hot spring resort with open-air baths," "family-friendly," and "luxury").

[1239] Processing: The server creates a user profile based on the input attribute information, stores it in a database, and runs an algorithm to match the user with the hot spring information in the database and find the hot spring resort that best suits them.

[1240] Output: The best hot springs are recommended to the user in a list or ranking format. For example, a list of hot springs based on the user's preferences is displayed.

[1241] Step 3:

[1242] Creating a virtual hot spring tour

[1243] The server generates a virtual hot spring tour based on the recommended hot spring resorts.

[1244] Input: Information about the recommended hot spring resort (location information, facility features, photos, videos, audio data, etc.).

[1245] Processing: The server uses this information to generate 3D models and actual video and audio data, and organizes them into virtual hot spring tour data. For example, it can stitch together 360-degree images to recreate a realistic view of a hot spring.

[1246] Output: Virtual hot spring tour data is generated and sent to the device.

[1247] Step 4:

[1248] Providing a virtual hot spring experience

[1249] The device provides a virtual hot spring experience using a smartphone or VR glasses.

[1250] Input: Virtual hot spring tour data provided by the server.

[1251] Processing: The device receives the data and presents it to the user through an application that plays back the 3D model and audio data. The user then enjoys the virtual hot spring experience through their smartphone or VR glasses. For example, they can view a 360-degree video of the hot spring through VR goggles, giving the feeling that they are actually in the hot spring.

[1252] Output: Playback of images and sounds as a virtual hot spring experience. The user experiences a virtual hot spring tour.

[1253] Step 5:

[1254] Travel plan generation and support

[1255] The server automatically creates a specific travel plan based on the user's favorite hot springs.

[1256] Input: Details of the hot spring resort selected by the user, and the user's travel preferences.

[1257] Processing: The server automatically generates a travel plan to the hot spring resort. Specifically, it creates a schedule by combining information on accommodations, transportation options (bullet trains, buses, rental cars, etc.), and information on surrounding tourist spots and restaurants. For example, it suggests the optimal route to the hot spring resort and a stay schedule.

[1258] Output: A travel itinerary is automatically generated and presented to the user, who can then make specific reservations and arrangements based on the proposed plan.

[1259] Step 6:

[1260] Sharing and interacting with virtual hot spring experiences

[1261] Users can share screenshots and videos of their virtual hot spring experience with other users on social media or within the dedicated app.

[1262] Input: Screenshots and videos taken during your virtual spa experience.

[1263] Processing: Users can post this data to social media or in-app communities, where other users can comment, like, or provide feedback on other users' posts.

[1264] Output: The posted content is visible to other users and they can comment, like, or otherwise interact with it.

[1265] (Application example 1)

[1266] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1267] In today's world, there are limited ways for users to experience various hot spring resorts from the comfort of their own homes and use the information they gain to plan their trips. This makes it difficult to fully understand the appeal of hot springs without actually visiting them, and it is also difficult to compare and consider multiple hot spring resorts at once. The present invention aims to solve these problems by providing a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own homes, select the most suitable hot spring resort, and easily plan their trips.

[1268] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1269] In this invention, the server includes means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database, means for inputting user attribute information and recommending optimal hot springs based on the attribute information, means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal, means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user, means for sharing information about the virtual hot spring experience and the actual travel with other users and promoting interaction, means for using a generative AI model to recommend optimal hot spring resorts based on the user's attribute information, and means for using prompt sentences as input when generating the virtual hot spring tour and providing an experience tailored to the user's preferences. This allows users to enjoy a variety of hot spring experiences in virtual reality from the comfort of their own home, and makes it easy to find optimal hot spring resorts and plan their travels.

[1270] "Hot spring data" is a general term for information related to hot springs, such as the location, characteristics, and information on surrounding facilities of hot springs.

[1271] "Location information" is information that indicates the geographical location of a particular hot spring resort or facility.

[1272] "Characteristics" refers to the individual characteristics of a hot spring resort, such as the type of hot spring, temperature, ingredients, and facilities for use.

[1273] "Nearby facilities" refers to information about accommodations, restaurants, tourist spots, etc. located near the hot spring resort.

[1274] The "database" is a system for storing and managing collected hot spring data.

[1275] "User attribute information" is individual information such as the user's hot spring preferences, travel style, budget, and traveling companions.

[1276] "Recommendation" refers to the act of suggesting the most suitable hot spring resort to a user based on collected data and algorithms.

[1277] "Virtual Hot Spring Tour" is content that uses virtual reality technology to provide users with a virtual hot spring experience.

[1278] "User's device" is a general term for devices used by users, such as smartphones, tablets, and VR devices.

[1279] A "travel plan" is a plan that includes information such as accommodation, transportation, and surrounding tourist facilities when the user actually visits a hot spring resort.

[1280] A "generative AI model" is an algorithm that uses artificial intelligence technology to generate a specific output result based on input data.

[1281] A "prompt sentence" is text data that is input into a generative AI model and serves as the starting point for the AI ​​to generate output based on it.

[1282] System Configuration

[1283] The system consists of the following main components:

[1284] server

[1285] Devices (smartphones, tablets, VR devices, etc.)

[1286] User

[1287] Hot Spring Data Collection and Mapping

[1288] server

[1289] The server collects various data about hot springs from official hot spring websites, review websites, APIs, etc. The collected data includes location information, characteristics, and information about surrounding facilities. This data is organized and stored in a database. The database is regularly updated to reflect new and changed hot spring information.

[1290] Input of user attribute information and recommendations

[1291] User

[1292] Users log in to the system and enter attribute information such as their hot spring preferences, travel style, budget, and travel companions.

[1293] server

[1294] The server reads the attribute information entered by the user and recommends the best hot spring resort using a generative AI model, which is built using machine learning libraries such as TensorFlow.

[1295] Creating a virtual hot spring tour

[1296] server

[1297] The server generates a virtual hot spring tour based on the data of the selected hot spring resort, using 3D models and environmental acoustic data. Unity software is used to create the 3D models.

[1298] The generated virtual hot spring tour is provided to the user as input with prompt sentences, allowing the user to have an experience tailored to their preferences.

[1299] Providing a virtual hot spring experience

[1300] Terminal

[1301] The user's device (smartphone, tablet, VR device, etc.) receives the virtual hot spring tour provided by the server and provides the user with a virtual reality hot spring experience. The device renders the 3D model and plays video and audio data.

[1302] Travel plan generation and support

[1303] server

[1304] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a travel plan to that hot spring resort, including information on accommodations, transportation, and surrounding tourist attractions.

[1305] User

[1306] Based on the proposed itinerary, users can book accommodation and arrange transportation, and can also use the application to receive navigation and tourist information during the actual trip.

[1307] Sharing and interacting with virtual hot spring experiences

[1308] User

[1309] Users can share screenshots and videos of their virtual hot spring experience on social media and in-app communities, where other users can comment on the posts and share the same virtual hot spring experience.

[1310] Specific examples

[1311] User A likes open-air baths and wants to travel with his or her family. When logging in for the first time, he or she enters this information as attributes. The server uses a generative AI model to recommend hot spring resorts with open-air baths that are ideal for families. User A experiences a virtual hot spring tour using a VR device and selects a hot spring resort that he or she likes. The server generates a specific travel plan to that hot spring resort and proposes it to User A. User A makes a travel reservation based on the proposed plan and actually visits the resort.

[1312] Prompt Sentence Examples

[1313] "Data collection for hot spring tour experience:

[1314] Users like the open-air baths, relaxation, and family-friendly features.

[1315] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1316] System processing steps

[1317] Step 1: Collecting and mapping hot spring data

[1318] server

[1319] The server collects data about hot springs from the APIs of official hot spring websites and review websites. Input data includes hot spring location information, characteristics, and information about surrounding facilities. This data is scraped using Python's BeautifulSoup and Requests libraries to extract the necessary information. The extracted data is organized and stored in a database (AWS RDS). The database is updated regularly to reflect new and changed hot spring information. The hot spring data stored in the database is obtained as output.

[1320] Step 2: Enter user attribute information

[1321] User

[1322] The user logs in to the system and inputs attribute information such as the characteristics of their preferred hot springs, their travel style, their budget, and the people they will be traveling with. The input data is information about the user's hot spring preferences and travel. The server receives this attribute information and stores it in a database. As an output, the user attribute information is saved in the database.

[1323] Step 3: Recommend the best hot spring

[1324] server

[1325] The server uses a generative AI model (TensorFlow) to recommend the most suitable hot spring resort based on the collected hot spring data and the user's attribute information. The input data is the user's attribute information and existing hot spring data. The generative AI model processes this information and outputs a list of the most suitable hot spring resorts for the user. The output is a list of recommended hot spring resorts.

[1326] Step 4: Generate a virtual hot spring tour

[1327] server

[1328] The server generates a virtual hot spring tour based on the data of the selected hot spring resort. The input data is detailed information about the recommended hot spring resort. Unity software is used to generate 3D models and environmental acoustic data to create the virtual hot spring tour. The output is the data of the virtual hot spring tour. Specifically, a 3D model of the hot spring resort is constructed in Unity and sound effects are incorporated.

[1329] Step 5: Providing a virtual hot spring experience

[1330] Terminal

[1331] The user's device (smartphone, tablet, VR device, etc.) receives the virtual hot spring tour provided by the server and provides the user with a virtual reality hot spring experience. The input data is the virtual hot spring tour data sent from the server. The device receives this data, renders a 3D model, and plays video and audio data. The output is the user's ability to enjoy the virtual hot spring experience in VR or on their smartphone.

[1332] Step 6: Travel plan generation and support

[1333] server

[1334] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a travel plan to that hot spring resort. The input data is information about the hot spring resort selected by the user. The server generates a travel plan including information about accommodation, transportation, and surrounding tourist spots, and proposes it to the user. The output is a specific travel plan. Specific operations include searching for various information and creating an appropriate plan.

[1335] Step 7: Share and interact with your virtual hot spring experience

[1336] User

[1337] Users share screenshots and videos of their virtual hot spring experience on social media or in-app communities. The input data are images and videos taken during the virtual hot spring experience. Other users can comment on the posted content and share the same virtual hot spring experience. The output is to encourage interaction between users. Specific operations utilize the posting and commenting functions on social media and within the app.

[1338] Prompt Sentence Examples

[1339] "Data collection for hot spring tour experience:

[1340] Users like the open-air baths, relaxation, and family-friendly features.

[1341] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1342] The present invention provides a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home, plan trips to their favorite hot spring resorts, and dynamically adjust the content of the experience by recognizing the user's emotions. An embodiment of this system is described in detail below.

[1343] System Configuration

[1344] The system mainly consists of the following four main components:

[1345] 1. Server

[1346] 2. Terminal

[1347] 3. Users

[1348] 4. Emotion Engine

[1349] Implementation flow and operation of each element

[1350] 1. Hot spring data collection and mapping

[1351] server

[1352] The server collects data about hot springs from various sources (official websites, review sites, APIs, etc.) The collected data includes location information, the type of hot spring, its characteristics, information about surrounding facilities, photos, and videos.

[1353] The collected data is organized and stored in a database that is regularly updated to reflect new and changed hot spring information.

[1354] 2. Input of user attribute information and recommendations

[1355] User

[1356] When logging in to the system for the first time, users enter their personal information, including their preferred hot spring features (e.g., open-air bath, sauna, family-friendly), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[1357] server

[1358] The server reads the attribute information entered by the user and runs an algorithm to match it with hot spring data stored in a database, thereby creating a list of hot spring candidates that are most suitable for the user and recommending them.

[1359] 3. Emotional engine for recognizing user emotions

[1360] Emotion Engine

[1361] While the user is entering their attribute information and experiencing the virtual hot spring tour, an emotion engine is activated that recognizes emotions from the user's facial expressions and voice. The emotion engine acquires emotion data and analyzes it in real time.

[1362] server

[1363] It receives emotional data sent from the emotion engine and dynamically adjusts recommendations and changes the content of the virtual experience.

[1364] 4. Creating a Virtual Hot Spring Tour

[1365] server

[1366] The server generates a virtual hot spring tour based on the selected hot springs, using 3D models, video footage of the actual hot springs, and audio data. The experience is customized according to the user's emotional state, as recognized by the emotion engine.

[1367] Terminal

[1368] The generated virtual hot spring tour is provided to the user's device, and the user can start the virtual hot spring tour on the device and enjoy the experience as if they were in a real hot spring.

[1369] 5. Travel planning generation and support

[1370] server

[1371] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a travel plan to that hot spring resort, including details on accommodation, transportation, surrounding tourist attractions, restaurants, etc.

[1372] User

[1373] The user can review the proposed travel plan and make accommodation reservations, transportation arrangements, etc. based on the plan.

[1374] During the actual trip, the application can be used to provide navigation and tourist information.

[1375] 6. Sharing and interacting with virtual hot spring experiences

[1376] User

[1377] Users can post screenshots and videos of their virtual hot spring experience on social media and in-app communities.

[1378] Other users can comment on the posts and share the same virtual hot spring experience.

[1379] Specific examples

[1380] Below are some specific examples of how the system can be used.

[1381] Example 1:

[1382] User A likes open-air baths at hot springs and wants to travel with his family. When he logs in for the first time, he enters this information as attributes.

[1383] The server uses its database to recommend hot spring resorts with open-air baths suitable for families in areas rich in nature.

[1384] User A chooses one of the recommended hot springs and experiences a virtual hot spring tour. The emotion engine recognizes that User A is relaxed from his / her facial expressions and voice, and adds further relaxing elements.

[1385] If user A likes the hot spring resort, the server generates a specific travel plan to the hot spring resort and proposes it to user A.

[1386] Example 2:

[1387] User B uses a smartphone app to select a virtual hot spring tour to enjoy the virtual reality experience of a hot spring at the summit of Mount Fuji.

[1388] The server combines the 3D model of Mount Fuji with the model data of the hot spring and provides it to User B's terminal.

[1389] When User B puts on the VR glasses and begins the experience, the emotion engine recognizes excitement and surprise and adjusts the experience to make it even more exciting.

[1390] In this way, by using this system, users can enjoy a variety of hot spring experiences from the comfort of their own home, find the perfect hot spring resort, and plan an actual trip. Furthermore, by combining it with an emotion engine, it is possible to provide an experience optimized for the user's emotional state, and share the virtual hot spring experience with other users, creating a new platform for interaction.

[1391] The processing flow will be explained below.

[1392] Step 1:

[1393] server

[1394] Start collecting hot spring data. The server automatically collects hot spring data (location information, type of hot spring, characteristics, information on surrounding facilities, photos, videos) from various sources such as official websites, review sites, and APIs.

[1395] Step 2:

[1396] server

[1397] The collected hot spring data is organized and stored in a database. Location information, hot spring characteristics, and information on surrounding facilities are converted into an appropriate format and centrally managed.

[1398] Step 3:

[1399] User

[1400] When logging in to the system for the first time, users enter their demographic information, including their preferred hot spring features (e.g., open-air bath, sauna, family-friendly, etc.), travel style (e.g., relaxation, sightseeing, activities, etc.), budget, and travel companions.

[1401] Step 4:

[1402] server

[1403] The server analyzes the attribute information entered by the user, and then executes a matching algorithm based on the hot spring data in the database and the user's attribute information to generate the most suitable hot spring candidates.

[1404] Step 5:

[1405] server

[1406] The generated list of hot spring candidates is recommended to the user, allowing the user to check a list of hot springs that suit their preferences.

[1407] Step 6:

[1408] User

[1409] Check the list of recommended hot springs through your device and select the hot spring resort that interests you.

[1410] Step 7:

[1411] Emotion Engine

[1412] When a user selects a hot spring resort or experiences a virtual hot spring tour, the system analyzes the user's facial expressions and voice in real time to generate emotional data.

[1413] Step 8:

[1414] server

[1415] It receives emotional data from the emotion engine and dynamically adjusts the hot spring recommendation. For example, if the user is excited, it will suggest an active experience, and if the user is relaxed, it will suggest a quiet hot spring spot.

[1416] Step 9:

[1417] server

[1418] It generates a virtual hot spring tour based on the selected hot spring, integrating 3D models, actual video and audio data, and customizing the experience based on emotional data.

[1419] Step 10:

[1420] Terminal

[1421] The generated virtual hot spring tour data is received and provided to the user, who then begins the virtual hot spring tour using a smartphone or VR glasses.

[1422] Step 11:

[1423] User

[1424] Experience the virtual hot spring tour and enjoy the atmosphere of the hot spring resort. The emotion engine continuously analyzes the user's emotions and adjusts the experience in real time.

[1425] Step 12:

[1426] User

[1427] After experiencing the virtual hot spring tour, users can select their favorite hot spring resort. The emotion engine analyzes the user's satisfaction and uses the results to provide feedback for future recommendations and experience adjustments.

[1428] Step 13:

[1429] server

[1430] Generate a specific itinerary to the selected hot spring resort, including details on accommodation, transportation, nearby tourist attractions, restaurants, etc.

[1431] Step 14:

[1432] server

[1433] The generated travel plan is proposed to the user, who can then confirm the proposed plan and proceed with reservations and arrangements.

[1434] Step 15:

[1435] User

[1436] Use the application during your actual trip to receive real-time support for navigation and tourist information.

[1437] Step 16:

[1438] User

[1439] Post screenshots and videos of your virtual hot spring experience on social media and in-app communities to interact with other users.

[1440] Step 17:

[1441] User

[1442] Users can view virtual hot spring experiences posted by other users and create a new forum for interaction through comments and reactions.

[1443] In this way, this system starts by collecting hot spring data, then provides recommendations based on the user's attribute information and emotional data, creates and experiences a virtual hot spring tour, proposes specific travel plans, and finally shares and interacts with the virtual hot spring experience, all in a single flow.

[1444] Example 2

[1445] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1446] Conventional hot spring information systems make it difficult for users to enjoy a variety of hot spring experiences from home. Furthermore, they lacked the ability to dynamically adjust the experience based on the user's emotional state, which prevented them from fully improving user satisfaction. Furthermore, it was time-consuming and inefficient for users to plan specific trips to their favorite hot spring resorts.

[1447] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1448] In this invention, the server includes means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database, means for inputting user attribute information and recommending optimal hot springs based on the attribute information, means equipped with an emotion engine that acquires user emotion data and analyzes it in real time and dynamically adjusts recommendations and virtual experience content based on the emotion data, means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal, means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user, and means for sharing information about the virtual hot spring experience and the actual trip with other users and promoting communication. This allows users to enjoy a variety of hot spring experiences from the comfort of their own home, dynamically adjust the experience according to their emotional state, and efficiently plan their travel by finding the optimal hot spring resort.

[1449] "Hot spring data" is information about hot spring facilities, including location information, characteristics, information about surrounding facilities, photos, videos, and the like.

[1450] The "database" is an information management system for organizing and storing collected hot spring data in a unified format.

[1451] "User" refers to an individual who uses the system to receive hot spring information recommendations and experience virtual hot spring tours.

[1452] "Attribute information" refers to personal data such as the user's preferences, travel style, budget, and traveling companions.

[1453] "Recommendation" refers to the act of recommending the most suitable hot spring facility based on the user's attribute information.

[1454] The "emotion engine" is a system that acquires emotional data from the user's facial expressions, voice, etc. and analyzes it in real time.

[1455] The "Virtual Hot Spring Tour" is virtual reality content generated using 3D models, actual hot spring footage, and audio data, allowing users to enjoy a hot spring experience on their device.

[1456] A "terminal" is a device used by a user to experience a virtual hot spring tour, and includes a VR device, a mobile terminal, etc.

[1457] A "travel plan" is a detailed plan that automatically generates a specific travel schedule to a user's favorite hot spring resort, including information on accommodation, transportation, and surrounding tourist attractions.

[1458] "Information sharing" refers to the act of sharing information about virtual hot spring experiences and real trips with other users to promote interaction.

[1459] "Dynamic adjustment" refers to changing recommendations and virtual experience content in real time based on the user's emotional state.

[1460] A "hot spring facility" is a place that provides hot bathing services and includes multiple types of baths, such as open-air baths and saunas.

[1461] The present invention is a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. This system is primarily composed of four main components: a server, a terminal, a user, and an emotion engine. An embodiment of this system is described in detail below.

[1462] Hot spring data collection

[1463] server

[1464] The server collects information about hot springs through external APIs such as the Google Places API and TripAdvisor API. The collected data includes the hot spring's name, location, type and characteristics, information about surrounding facilities, photos, and videos. The collected data is organized using a Python script and stored in a MySQL database. This process is performed periodically, and new information is automatically updated.

[1465] Input of user attribute information and recommendations

[1466] User

[1467] When users log in to the system for the first time, they enter attribute information such as their preferred hot spring characteristics, travel style, budget, and travel companion information through a dedicated form. This form is built with React.js.

[1468] server

[1469] The server parses the information entered by the user in JSON format and stores it in a database. Then, using machine learning algorithms such as Pandas and Scikit-learn, it matches this attribute information with hot spring data. The algorithm then lists and recommends the hot springs that are most suitable for the user. This recommendation is made in the form of providing a list of the top five hot spring facilities based on the filter criteria.

[1470] Recognizing user emotions with an emotion engine

[1471] Emotion Engine

[1472] While the user is entering their personal information or experiencing the virtual hot spring tour, their facial expressions and voice are captured via a webcam and microphone. Using AWS Rekognition and Google Cloud Vision, emotions are analyzed in real time. This emotional data is then sent to a server.

[1473] server

[1474] The server analyzes the data received from the emotion engine and dynamically adjusts recommendations and the content of the virtual experience according to the user's emotional state. For example, if the server detects that the user is relaxed, it will change the background music to soothing music.

[1475] Creating a virtual hot spring tour

[1476] server

[1477] Based on the selected hot spring, the server uses Unity or Unreal Engine to generate a 3D model, video footage of the hot spring, and audio data, which is then sent to the user's device.

[1478] Terminal

[1479] The device displays a 3D virtual hot spring tour on devices such as VR glasses or smartphones, allowing users to experience the hot springs as if they were actually there.

[1480] Generate specific travel plans

[1481] server

[1482] Once the user has experienced a virtual hot spring tour and selected their favorite hot spring, the server automatically generates a travel plan using the Google Maps API and Skyscanner API to collect information on accommodation, transportation, and surrounding tourist spots, and provides the user with a travel plan that is tailored to their needs.

[1483] User

[1484] Users can view the provided itinerary, book accommodation and arrange transportation based on the plan, and use the application to receive real-time navigation and tourist information during their trip.

[1485] Sharing and interacting with virtual hot spring experiences

[1486] User

[1487] Users can take screenshots and videos of their virtual hot spring experience and post them on social media or in the in-app community. Users can also deepen their interactions by commenting on and sharing content posted by other users.

[1488] Specific usage examples and prompt sentence examples

[1489] Example 1

[1490] If User A likes open-air baths and wants to go on a family trip, the server will recommend hot spring resorts with family-friendly open-air baths. User A then experiences a virtual tour of the selected hot spring resort, and if the experience is deemed relaxing, further relaxing elements are added. Finally, the server generates a specific travel plan, allowing User A to efficiently prepare for the trip.

[1491] Example 2

[1492] If User B wishes to experience a virtual hot spring at the top of Mt. Fuji using a smartphone app, the server will provide a combination of a 3D model of Mt. Fuji and hot spring data. User B puts on the VR glasses and begins the virtual experience. If excitement and surprise are recognized, the experience will be adjusted to make it even more exciting.

[1493] Prompt Sentence Examples

[1494] "If the user likes open-air baths and family trips, create a list of recommended hot spring resorts and generate a virtual hot spring tour. Then analyze the user's emotions and tailor the experience accordingly."

[1495] As described above, by using this system, users can enjoy a variety of hot spring experiences from the comfort of their own home, and receive the optimal experience according to their emotional state. It also enables users to find the best hot spring resort and efficiently plan their trip.

[1496] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1497] Step 1: Collecting hot spring data

[1498] The server sends requests to external APIs such as the Google Places API and TripAdvisor API. The input is keywords related to the specified area and hot spring facility. The output is data such as the hot spring facility's name, location, features, information on surrounding facilities, photos, and videos. This data is collected using a Python script, checked for duplicate data, and standardized data formats before being stored in a MySQL database. A job scheduler is used to perform regular updates.

[1499] Step 2: Enter user attribute information

[1500] When users log in to the system for the first time, they enter attribute information such as their preferred hot spring characteristics, travel style, budget, and travel companion information through a dedicated form built with React.js. The information entered is then sent to the server in JSON format.

[1501] Step 3: Store user attribute information

[1502] The server parses the JSON-formatted attribute information received from the user, extracts the necessary fields, and saves them in a MySQL database. The input is JSON data of attribute information, and the output is a database containing organized attribute data.

[1503] Step 4: Run the recommendation algorithm

[1504] The server uses Pandas and Scikit-learn to run an algorithm that matches the user's attribute information with hot spring data. The input is the user's attribute information and hot spring data. The output is a list of hot spring facilities that are best suited to the user. Filter conditions are applied, and the hot spring facilities that match the conditions are scored and the top five are presented to the user.

[1505] Step 5: Acquire and analyze emotion data

[1506] The emotion engine captures facial expressions and voices via a webcam and microphone while the user is entering attribute information or experiencing a virtual hot spring tour. The input is the captured facial image and voice data. This is sent to AWS Rekognition or Google Cloud Vision to obtain emotion data. The output is analyzed emotion data.

[1507] Step 6: Use emotion data

[1508] The server receives emotional data from the emotion engine and dynamically adjusts the recommendations and virtual experience content. The input is the emotional data. The output is the adjusted recommendations and virtual experience content. For example, if the emotion is determined to be relaxing, the background music is changed to soothing music.

[1509] Step 7: Generate a virtual hot spring tour

[1510] The server uses Unity or Unreal Engine to generate a 3D model based on the selected hot spring, as well as actual hot spring video and audio data. The input is the hot spring data. The output is the generated virtual hot spring tour data, which is then sent to the user's device.

[1511] Step 8: Run a virtual hot spring tour

[1512] The terminal provides users with a virtual hot spring tour using devices such as VR glasses or a smartphone. The input is virtual hot spring tour data provided by the server. Users can enjoy an interactive experience as if they were actually in the hot spring resort. The output is the user's experience data.

[1513] Step 9: Generate a travel plan

[1514] When the user selects their favorite hot spring resort, the server automatically generates a specific travel plan using the Google Maps API and Skyscanner API. The input is information about the selected hot spring resort and the user's attribute information. The output is a travel plan that includes information about accommodations, transportation, surrounding tourist attractions, and restaurants.

[1515] Step 10: Travel Plan Proposal

[1516] The user can review the proposed itinerary and make accommodation reservations and transportation arrangements. The input is the itinerary data provided by the server. The output is the user's reservation information. During the trip, the user can also use the application to receive real-time navigation and tourist information.

[1517] Step 11: Share and interact with your virtual hot spring experience

[1518] Users can post screenshots and videos of their virtual hot spring experience on social media and in-app communities. The input is the user's experience data. The output is the shared information and feedback from other users. Users can interact with other users and share their comments and experiences.

[1519] (Application example 2)

[1520] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1521] In today's busy lifestyles, users lack the means to easily plan a hot spring trip to relax and refresh. Furthermore, existing systems struggle to provide a customized experience that takes into account the user's emotions. Furthermore, they lack the ability to allow users to share their virtual experiences and interact with other users. To address these issues, a system is needed that dynamically adjusts the user's experience using emotion recognition to provide an authentic virtual hot spring experience.

[1522] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1523] In this invention, the server includes: means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database; means for inputting user attribute information and recommending optimal hot springs based on the attribute information; means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal; means for recognizing the user's emotions in real time and dynamically adjusting the content of the virtual hot spring tour based on the data; means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user; and means for sharing information about the virtual hot spring experience and the actual travel with other users and promoting interactions. This allows users to enjoy various hot spring experiences from the comfort of their own homes and find the optimal hot spring resort through experiences customized according to their emotions. Furthermore, users can obtain new information and share travel plans through interactions with other users.

[1524] "Hot spring data" is data including location information, characteristics, and information on surrounding facilities related to hot springs.

[1525] "Location information" is information that indicates the geographical location of a hot spring.

[1526] "Characteristics" is information indicating the type and characteristics of the hot spring, the details of the facilities, and so on.

[1527] "Information about surrounding facilities" refers to information about accommodations, tourist spots, restaurants, and the like located near the hot spring.

[1528] A "database" is a storage system in which collected data is systematically stored so that it can be managed and searched.

[1529] "User attribute information" is information input by the user regarding hot spring preferences, travel style, budget, and traveling companions.

[1530] "Recommendation" is a process that suggests optimal options based on user attribute information.

[1531] A "virtual hot spring tour" is a virtual hot spring experience that uses 3D models of hot spring resorts and environmental acoustic data.

[1532] "Terminal" refers to the device that a user uses to use the system, including smartphones, tablets, VR devices, etc.

[1533] "Emotion recognition" is a technology that analyzes emotions from a user's facial expressions and voice in real time.

[1534] A "travel plan" is a plan that includes a specific travel schedule to a hot spring resort, as well as information on accommodation, transportation, and surrounding tourist attractions.

[1535] "Interaction" means that users share information and communicate with each other.

[1536] MODE FOR CARRYING OUT THE INVENTION

[1537] This invention is a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. Furthermore, the system can recognize the user's emotions and dynamically adjust the content of the experience. The following describes an embodiment of the invention.

[1538] System Configuration

[1539] The system consists of four main components:

[1540] 1. Server

[1541] 2. Terminal

[1542] 3. Users

[1543] 4. Emotion Engine

[1544] The specific operation of each element will be explained below.

[1545] Hot Spring Data Collection and Mapping

[1546] The server collects data about hot springs from various sources, including their locations, characteristics, and surrounding facilities. This information is stored in a database and updated periodically.

[1547] Input of user attribute information and recommendations

[1548] When a user logs in to the system for the first time, they enter their own attribute information, including their preferred hot spring characteristics, travel style, budget, and travel companions. The server uses this information to run an algorithm that matches it with hot spring data stored in the database and recommends the most suitable hot springs to the user.

[1549] Recognizing user emotions with an emotion engine

[1550] The emotion engine recognizes emotions from the user's facial expressions and voice in real time while the user is entering attribute information or experiencing a virtual hot spring tour. The recognized emotion data is sent to the server and used to dynamically adjust the experience.

[1551] Creating and providing virtual hot spring tours

[1552] The server generates a virtual hot spring tour based on the selected hot springs, using 3D models of the hot springs and environmental acoustic data. It also has a function to customize the experience depending on the user's emotional state. The generated virtual hot spring tour is provided to the user's devices, such as smartphones, smart glasses, and head-mounted displays.

[1553] Specific examples

[1554] For example, User A likes open-air baths and is planning a trip with his or her family. When User A enters this attribute information when logging in for the first time, the server recommends the most suitable hot spring resort from the database. When User A selects the recommended hot spring resort and starts the virtual hot spring tour, the emotion engine recognizes User A's relaxation state and adjusts the experience to make it even more relaxing.

[1555] Hardware and software used

[1556] Hardware: Smartphones, smart glasses, head-mounted displays, webcams

[1557] Software: OpenCV (cv2), DeepFace, Python standard library

[1558] Prompt Sentence Examples

[1559] Below are some example prompts to input to a generative AI model:

[1560] "How can I implement real-time emotion recognition in Python to customize the user experience for a hot spring experience app? I'd like code that meets the following criteria:"

[1561] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1562] Step 1:

[1563] The server collects hot spring data and stores information on its location, characteristics, and surrounding facilities in a database. This process involves collecting data about hot springs from official websites, review sites, APIs, etc., properly formatting it, and storing it in the database. The input is data from external sources, and the output is formatted hot spring data.

[1564] Step 2:

[1565] When a user logs in for the first time, they enter attribute information such as their hot spring preferences, travel style, budget, and travel companions. The entered attribute information is saved in the database as a user profile. The input is the attribute information provided by the user, and the output is the user attribute information saved in the database.

[1566] Step 3:

[1567] The server recommends the most suitable hot spring resort from the hot spring data stored in the database based on the user's attribute information. The recommendation algorithm compares the user profile with the hot spring data and generates a list of matching hot spring resorts. The input is the user's attribute information and the hot spring data, and the output is a list of recommended hot spring resorts.

[1568] Step 4:

[1569] The server generates a virtual hot spring tour based on hot springs selected from the recommended hot spring resorts. This tour is designed using 3D models of hot springs and environmental acoustic data. The input is information about the selected hot spring resorts, and the output is the generated virtual hot spring tour data.

[1570] Step 5:

[1571] The terminal provides the generated virtual hot spring tour to the user. The user can experience the virtual hot spring tour using devices such as smartphones, smart glasses, and head-mounted displays. The input is the virtual hot spring tour data provided by the server, and the output is the virtual experience through the user's vision and hearing.

[1572] Step 6:

[1573] When a user experiences a virtual hot spring tour, the emotion engine recognizes the user's facial expressions and voice in real time and analyzes the emotional data. The analyzed emotional data is sent to the server. The input is the user's facial expressions and voice data, and the output is the analyzed emotional data.

[1574] Step 7:

[1575] The server dynamically adjusts the content of the virtual hot spring tour based on the received emotion data. For example, if the user is relaxed, it adds more relaxing content, and if the user is excited, it strengthens exciting elements. The input is the emotion data sent from the emotion engine, and the output is the adjusted virtual hot spring tour data.

[1576] Step 8:

[1577] When a user selects a hot spring resort they like, the server automatically generates a specific travel plan to that hot spring resort. This plan includes information on transportation, accommodation, surrounding tourist attractions, etc. The input is the user's selection information, and the output is the generated travel plan.

[1578] Step 9:

[1579] Users can review the proposed itinerary and make any necessary reservations. They can also share and interact with other users about their virtual hot spring experiences and travel information. The input is the proposed itinerary and the user's decision information, and the output is the shared experience information.

[1580] Step 10:

[1581] The terminal collects input from the user (attribute information, emotion data, selection information, etc.) and sends it to the server. It also displays data received from the server (recommendations, virtual tour data, travel plans, etc.). The input is based on data communication between the user and the server, and the output is the overall service experience provided to the user.

[1582] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1583] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1584] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1585] [Fourth embodiment]

[1586] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1587] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1588] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1589] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1590] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1591] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1592] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1593] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1594] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1595] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1596] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1597] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1598] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1599] The present invention provides a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. An embodiment of this system will be described in detail below.

[1600] System Configuration

[1601] The system mainly consists of the following three main components:

[1602] 1. Server

[1603] 2. Terminal

[1604] 3. Users

[1605] Implementation flow and operation of each element

[1606] 1. Hot spring data collection and mapping

[1607] server

[1608] The server collects data about hot springs from various sources (official websites, review sites, APIs, etc.) The collected data includes location information, the type of hot spring, its characteristics, information about surrounding facilities, photos, and videos.

[1609] The collected data is organized and stored in a database that is regularly updated to reflect new and changed hot spring information.

[1610] 2. Input of user attribute information and recommendations

[1611] User

[1612] When users log in to the system for the first time, they enter attribute information such as their preferred hot spring characteristics (e.g., open-air bath, sauna, family-friendly, etc.), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[1613] server

[1614] The server reads the attribute information entered by the user and runs an algorithm to match it with hot spring data stored in a database, thereby creating a list of hot spring candidates that are most suitable for the user and recommending them.

[1615] 3. Creating a Virtual Hot Spring Tour

[1616] server

[1617] Based on the selected hot springs, the server generates a virtual hot spring tour using 3D models, footage of the actual hot springs, and audio data.

[1618] This data is then processed for virtual reality (VR) experiences and visual experiences on mobile devices.

[1619] Terminal

[1620] The generated virtual hot spring tour is provided to the user's device, and the user can start the virtual hot spring tour on the device and enjoy the experience as if they were in a real hot spring.

[1621] 4. Providing a virtual hot spring experience using smartphones and VR glasses

[1622] Terminal

[1623] The user launches the application on their smartphone or VR glasses and accesses the virtual hot spring tour provided by the server.

[1624] The device receives data from the server and plays 3D models and sound data to provide the user with a virtual experience.

[1625] 5. Travel planning generation and support

[1626] server

[1627] When a user experiences a virtual hot spring tour and selects a hot spring resort that they like, the server automatically generates a travel plan to that hot spring resort.

[1628] The travel plan includes details such as accommodation information, transportation options, information on nearby tourist attractions, and restaurants.

[1629] User

[1630] The user can review the proposed travel plan and make accommodation reservations, transportation arrangements, etc. based on the plan.

[1631] During the actual trip, the application can be used to provide navigation and tourist information.

[1632] 6. Sharing and interacting with virtual hot spring experiences

[1633] User

[1634] Users can post screenshots and videos of their virtual hot spring experience on social media and in-app communities.

[1635] Other users can comment on the posts and share the same virtual hot spring experience.

[1636] Specific examples

[1637] Below are some specific examples of how the system can be used.

[1638] Example 1:

[1639] User A likes open-air baths at hot springs and wants to travel with his family. When he logs in for the first time, he enters this information as attributes.

[1640] The server uses its database to recommend hot spring resorts with open-air baths suitable for families in areas rich in nature.

[1641] When User A selects one of the recommended hot springs and experiences a virtual hot spring tour, a 3D model of the hot spring resort and actual video and audio data are displayed on the device.

[1642] If user A likes the hot spring resort, the server generates a specific travel plan to the hot spring resort and proposes it to user A.

[1643] Example 2:

[1644] User B uses a smartphone app to select a virtual hot spring tour to enjoy the virtual reality experience of a hot spring at the summit of Mount Fuji.

[1645] The server combines the 3D model of Mount Fuji with the model data of the hot spring and provides it to User B's terminal.

[1646] User B puts on the VR glasses and enjoys the experience of soaking in a hot spring at the top of Mount Fuji.

[1647] In this way, by using this system, users can enjoy a variety of hot spring experiences from the comfort of their own home, find the perfect hot spring resort, and plan an actual trip. It is also possible to share virtual hot spring experiences with other users and create new opportunities for interaction.

[1648] The processing flow will be explained below.

[1649] Step 1:

[1650] server

[1651] Start collecting hot spring data. The server automatically collects hot spring data (location information, type of hot spring, characteristics, information on surrounding facilities, photos, videos) from various sources such as official websites, review sites, and APIs.

[1652] Step 2:

[1653] server

[1654] Organize the collected hot spring data. Convert the location information, hot spring characteristics, and information on surrounding facilities into an appropriate format and store it in a database.

[1655] Step 3:

[1656] User

[1657] When logging in to the system for the first time, users enter their personal information, including their preferred hot spring features (e.g., open-air bath, sauna, family-friendly), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[1658] Step 4:

[1659] server

[1660] The server analyzes the attribute information entered by the user, runs an algorithm to match the attribute information with hot spring data in the database, and generates a list of optimal hot spring candidates.

[1661] Step 5:

[1662] server

[1663] The generated list of hot spring candidates is sent to the terminal for recommendation to the user, allowing the user to view a list of hot springs that are best suited to them.

[1664] Step 6:

[1665] User

[1666] Check the list of recommended hot springs through your device and select the hot spring resort that interests you.

[1667] Step 7:

[1668] server

[1669] A virtual hot spring tour is generated based on the selected hot springs. The 3D model, video footage of the actual hot springs, and audio data are integrated to create the virtual hot spring tour data.

[1670] Step 8:

[1671] Terminal

[1672] The generated virtual hot spring tour data is received and provided to the user, who then begins the virtual hot spring tour using a smartphone or VR glasses.

[1673] Step 9:

[1674] User

[1675] Take a virtual hot spring tour and enjoy the atmosphere of the hot spring resort, as if you were actually there.

[1676] Step 10:

[1677] User

[1678] After experiencing the virtual hot spring tour, choose your favorite hot spring resort.

[1679] Step 11:

[1680] server

[1681] Generate a specific itinerary to the selected hot spring resort, including details on accommodation, transportation, nearby tourist attractions, restaurants, etc.

[1682] Step 12:

[1683] server

[1684] The generated travel plan is proposed to the user, who can then review the proposed plan and proceed with reservations and arrangements as necessary.

[1685] Step 13:

[1686] User

[1687] Use the application during your actual trip to receive real-time support for navigation and tourist information.

[1688] Step 14:

[1689] User

[1690] Post screenshots and videos of your virtual hot spring experience on social media and in-app communities to interact with other users.

[1691] Step 15:

[1692] User

[1693] Users can view virtual hot spring experiences posted by other users and create a new forum for interaction through comments and reactions.

[1694] As described above, this system starts by collecting hot spring data, then provides recommendations based on the user's attribute information, creates and experiences a virtual hot spring tour, proposes specific travel plans, and finally shares and interacts with the virtual hot spring experience, all in a single flow.

[1695] Example 1

[1696] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1697] Conventional hot spring information systems provide insufficient support for users to find hot spring resorts and often lack the data and algorithms required to provide virtual hot spring experiences. As a result, users have difficulty efficiently finding hot spring resorts that suit their preferences and obtaining detailed information through virtual hot spring experiences. Furthermore, the systems do not provide comprehensive information necessary for planning actual trips, which results in insufficient user convenience. Furthermore, the virtual hot spring experience does not promote interaction between users.

[1698] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1699] In this invention, the server includes: means for collecting hot spring information and storing location data, facility features, and information on surrounding services in a database; means for inputting user attribute information and recommending optimal hot springs based on the attribute information; means for organizing and frequently updating the collected hot spring information; means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal; means for generating 3D models and environmental acoustic data for the virtual hot spring tour; means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user; means for providing a virtual hot spring experience using a smartphone or VR device; and means for sharing information about the virtual hot spring experience and actual travel with other users and promoting communication. This allows users to enjoy a variety of hot spring experiences from the comfort of their own homes, find the perfect hot spring, and efficiently plan their actual travel. Furthermore, the virtual hot spring experience also promotes communication with other users, stimulating the sharing of hot spring information.

[1700] "Hot spring information" is comprehensive data including location data for hot spring facilities, the type of hot spring, its characteristics, information on surrounding services, and images and videos of the facility.

[1701] "User attribute information" is information relating to the user's preferences and needs, such as the characteristics of the hot springs the user prefers, travel style, budget, and traveling companions.

[1702] The "database" is an information management system that systematically stores and manages collected hot spring information and user attribute information, and allows searching and updating as needed.

[1703] "Recommendation" is the act of selecting the most suitable hot spring facility based on the user's attribute information and suggesting it to the user.

[1704] A "virtual hot spring tour" is a digital tour generated based on a selected hot spring, virtually recreating an actual hot spring experience using 3D models and acoustic data.

[1705] A "3D model" is digital data that recreates the selected hot spring facility and its surroundings in three dimensions, providing a three-dimensional visual experience in a virtual space.

[1706] "Environmental acoustic data" is the actual acoustic environment of selected hot spring facilities that has been recorded and saved in digital format, allowing for an auditory experience in a virtual space.

[1707] A "travel plan" is a comprehensive travel schedule that includes information on accommodations, transportation, surrounding tourist spots, and restaurants that a user needs when visiting a specific hot spring resort.

[1708] A "smartphone" is a portable telecommunications device that provides advanced computing power and a wide variety of features in addition to the functionality of a mobile phone.

[1709] A "VR device" is hardware that provides a virtual reality experience, allowing users to visually explore digital spaces freely.

[1710] "Promoting interaction" means strengthening connections between users by allowing multiple users to share information and exchange opinions through the system.

[1711] The present invention is a system that allows users to enjoy various hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. The system consists of three main components, which are detailed below:

[1712] 1. Server

[1713] 2. Terminal

[1714] 3. Users

[1715] Server Operation

[1716] Hot spring information collection and mapping

[1717] The server collects data from various sources, such as official hot spring websites, review websites, and APIs. The collected data includes hot spring location information, type, characteristics, information on surrounding facilities, and photos and videos. For example, from the official website of a hot spring facility, the server collects location information (latitude and longitude), hot spring water quality (sulfur spring, carbonated spring, etc.), and characteristics (open-air bath, family bath, with sauna), etc.

[1718] The server organizes the collected data and stores it in a database that is automatically updated to reflect new or changed hot spring information.

[1719] Input of user attribute information and recommendations

[1720] When users log in to the system for the first time, they enter information such as their preferred hot spring characteristics (e.g., open-air bath, sauna, family-friendly, etc.), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[1721] The server reads the attribute information entered by the user and matches it with hot spring information in the database. This algorithm recommends hot springs that best fit the user's preferences.

[1722] Creating a virtual hot spring tour

[1723] The server generates a virtual hot spring tour based on the selected hot spring resort using 3D models, video footage of the actual hot spring, and audio data. For example, it collects and processes 360-degree images of an open-air bath and audio data of the hot spring for VR devices.

[1724] The terminal provides the generated virtual hot spring tour data to the user, and the user can start and enjoy the virtual hot spring tour on the terminal.

[1725] Device behavior

[1726] Providing a virtual hot spring experience

[1727] The device plays a virtual hot spring tour using a smartphone or VR glasses. For example, when a user starts the application using a smartphone or VR glasses, it receives data provided by the server and plays 3D models and sound data to provide a virtual experience.

[1728] User Actions

[1729] Travel plan generation and support

[1730] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a specific travel plan to that hot spring resort, including information on accommodations, transportation, surrounding tourist spots, and restaurants.

[1731] The user can review the proposed itinerary and make accommodation reservations and transportation arrangements, for example by directly accessing a hotel booking site through the application to complete the accommodation arrangements.

[1732] Sharing and interacting with virtual hot spring experiences

[1733] Users can post screenshots and videos of their virtual hot spring experience on social media or in the app's community to share with other users, who can interact with the posts by commenting or liking them.

[1734] Specific examples

[1735] Below are some specific examples of how the system can be used.

[1736] Example 1:

[1737] User A likes outdoor baths and wants to travel with his family. He enters this information into the system when he logs in for the first time.

[1738] The server uses its database to recommend hot spring resorts with open-air baths suitable for families in areas rich in nature.

[1739] The terminal plays the virtual hot spring tour data provided by the server, and User A enjoys the 360-degree video and audio data of the open-air bath.

[1740] User A likes the experience and makes accommodation reservations and transportation arrangements based on the travel plan suggested by the system.

[1741] Example 2:

[1742] User B uses VR glasses to enjoy a virtual hot spring experience. For example, he wants to experience soaking in a hot spring at the top of Mt. Fuji.

[1743] The server generates virtual hot spring tour data using a 3D model of Mount Fuji and video and audio data of the hot springs, and provides it to the terminal.

[1744] The terminal plays back the virtual hot spring tour data to User B, who then wears the VR glasses and enjoys the hot spring experience at the top of Mount Fuji.

[1745] Prompt Sentence Examples

[1746] Below are examples of prompts for the generative AI model based on the specific features of this system.

[1747] Prompt: I'd like to travel with my family and I like places with lots of nature. I'm also looking for a hot spring resort with an open-air bath. What kind of hot spring resorts are there?

[1748] Prompt: I want to have a virtual reality experience of soaking in a hot spring at the top of Mt. Fuji. I'd like you to generate a virtual hot spring tour for that purpose.

[1749] In this way, this system allows users to enjoy a variety of hot spring experiences from the comfort of their own home, helping them not only to find the best hot spring resort but also to efficiently plan their actual trip. It also promotes interaction with other users through the virtual hot spring experience, stimulating the sharing of hot spring information.

[1750] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1751] Step 1:

[1752] Hot spring information collection and mapping

[1753] The server obtains data from various sources, including official hot spring websites, review sites, and APIs.

[1754] Input: Sources such as official website URL, review site data, API key, etc.

[1755] Processing: Access these sources and obtain hot spring location information (latitude and longitude), hot spring type (sulfur spring, carbonated spring, etc.), facility features (open-air bath, sauna, etc.), information on surrounding facilities, and photos and videos via scraping or API.

[1756] Output: The collected hot spring information is organized and stored in a database. For example, detailed information about a specific hot spring facility is saved in the database.

[1757] Step 2:

[1758] Input of user attribute information and recommendations

[1759] When users log in for the first time, they enter information into the system such as their preferred hot spring characteristics, travel style, budget, and travel companions.

[1760] Input: User preferences and travel wishes (e.g., options such as "hot spring resort with open-air baths," "family-friendly," and "luxury").

[1761] Processing: The server creates a user profile based on the input attribute information, stores it in a database, and runs an algorithm to match the user with the hot spring information in the database and find the hot spring resort that best suits them.

[1762] Output: The best hot springs are recommended to the user in a list or ranking format. For example, a list of hot springs based on the user's preferences is displayed.

[1763] Step 3:

[1764] Creating a virtual hot spring tour

[1765] The server generates a virtual hot spring tour based on the recommended hot spring resorts.

[1766] Input: Information about the recommended hot spring resort (location information, facility features, photos, videos, audio data, etc.).

[1767] Processing: The server uses this information to generate 3D models and actual video and audio data, and organizes them into virtual hot spring tour data. For example, it can stitch together 360-degree images to recreate a realistic view of a hot spring.

[1768] Output: Virtual hot spring tour data is generated and sent to the device.

[1769] Step 4:

[1770] Providing a virtual hot spring experience

[1771] The device provides a virtual hot spring experience using a smartphone or VR glasses.

[1772] Input: Virtual hot spring tour data provided by the server.

[1773] Processing: The device receives the data and presents it to the user through an application that plays back the 3D model and audio data. The user then enjoys the virtual hot spring experience through their smartphone or VR glasses. For example, they can view a 360-degree video of the hot spring through VR goggles, giving the feeling that they are actually in the hot spring.

[1774] Output: Playback of images and sounds as a virtual hot spring experience. The user experiences a virtual hot spring tour.

[1775] Step 5:

[1776] Travel plan generation and support

[1777] The server automatically creates a specific travel plan based on the user's favorite hot springs.

[1778] Input: Details of the hot spring resort selected by the user, and the user's travel preferences.

[1779] Processing: The server automatically generates a travel plan to the hot spring resort. Specifically, it creates a schedule by combining information on accommodations, transportation options (bullet trains, buses, rental cars, etc.), and information on surrounding tourist spots and restaurants. For example, it suggests the optimal route to the hot spring resort and a stay schedule.

[1780] Output: A travel itinerary is automatically generated and presented to the user, who can then make specific reservations and arrangements based on the proposed plan.

[1781] Step 6:

[1782] Sharing and interacting with virtual hot spring experiences

[1783] Users can share screenshots and videos of their virtual hot spring experience with other users on social media or within the dedicated app.

[1784] Input: Screenshots and videos taken during your virtual spa experience.

[1785] Processing: Users can post this data to social media or in-app communities, where other users can comment, like, or provide feedback on other users' posts.

[1786] Output: The posted content is visible to other users and they can comment, like, or otherwise interact with it.

[1787] (Application example 1)

[1788] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1789] In today's world, there are limited ways for users to experience various hot spring resorts from the comfort of their own homes and use the information they gain to plan their trips. This makes it difficult to fully understand the appeal of hot springs without actually visiting them, and it is also difficult to compare and consider multiple hot spring resorts at once. The present invention aims to solve these problems by providing a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own homes, select the most suitable hot spring resort, and easily plan their trips.

[1790] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1791] In this invention, the server includes means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database, means for inputting user attribute information and recommending optimal hot springs based on the attribute information, means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal, means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user, means for sharing information about the virtual hot spring experience and the actual travel with other users and promoting interaction, means for using a generative AI model to recommend optimal hot spring resorts based on the user's attribute information, and means for using prompt sentences as input when generating the virtual hot spring tour and providing an experience tailored to the user's preferences. This allows users to enjoy a variety of hot spring experiences in virtual reality from the comfort of their own home, and makes it easy to find optimal hot spring resorts and plan their travels.

[1792] "Hot spring data" is a general term for information related to hot springs, such as the location, characteristics, and information on surrounding facilities of hot springs.

[1793] "Location information" is information that indicates the geographical location of a particular hot spring resort or facility.

[1794] "Characteristics" refers to the individual characteristics of a hot spring resort, such as the type of hot spring, temperature, ingredients, and facilities for use.

[1795] "Nearby facilities" refers to information about accommodations, restaurants, tourist spots, etc. located near the hot spring resort.

[1796] The "database" is a system for storing and managing collected hot spring data.

[1797] "User attribute information" is individual information such as the user's hot spring preferences, travel style, budget, and traveling companions.

[1798] "Recommendation" refers to the act of suggesting the most suitable hot spring resort to a user based on collected data and algorithms.

[1799] "Virtual Hot Spring Tour" is content that uses virtual reality technology to provide users with a virtual hot spring experience.

[1800] "User's device" is a general term for devices used by users, such as smartphones, tablets, and VR devices.

[1801] A "travel plan" is a plan that includes information such as accommodation, transportation, and surrounding tourist facilities when the user actually visits a hot spring resort.

[1802] A "generative AI model" is an algorithm that uses artificial intelligence technology to generate a specific output result based on input data.

[1803] A "prompt sentence" is text data that is input into a generative AI model and serves as the starting point for the AI ​​to generate output based on it.

[1804] System Configuration

[1805] The system consists of the following main components:

[1806] server

[1807] Devices (smartphones, tablets, VR devices, etc.)

[1808] User

[1809] Hot Spring Data Collection and Mapping

[1810] server

[1811] The server collects various data about hot springs from official hot spring websites, review websites, APIs, etc. The collected data includes location information, characteristics, and information about surrounding facilities. This data is organized and stored in a database. The database is regularly updated to reflect new and changed hot spring information.

[1812] Input of user attribute information and recommendations

[1813] User

[1814] Users log in to the system and enter attribute information such as their hot spring preferences, travel style, budget, and travel companions.

[1815] server

[1816] The server reads the attribute information entered by the user and recommends the best hot spring resort using a generative AI model, which is built using machine learning libraries such as TensorFlow.

[1817] Creating a virtual hot spring tour

[1818] server

[1819] The server generates a virtual hot spring tour based on the data of the selected hot spring resort, using 3D models and environmental acoustic data. Unity software is used to create the 3D models.

[1820] The generated virtual hot spring tour is provided to the user as input with prompt sentences, allowing the user to have an experience tailored to their preferences.

[1821] Providing a virtual hot spring experience

[1822] Terminal

[1823] The user's device (smartphone, tablet, VR device, etc.) receives the virtual hot spring tour provided by the server and provides the user with a virtual reality hot spring experience. The device renders the 3D model and plays video and audio data.

[1824] Travel plan generation and support

[1825] server

[1826] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a travel plan to that hot spring resort, including information on accommodations, transportation, and surrounding tourist attractions.

[1827] User

[1828] Based on the proposed itinerary, users can book accommodation and arrange transportation, and can also use the application to receive navigation and tourist information during the actual trip.

[1829] Sharing and interacting with virtual hot spring experiences

[1830] User

[1831] Users can share screenshots and videos of their virtual hot spring experience on social media and in-app communities, where other users can comment on the posts and share the same virtual hot spring experience.

[1832] Specific examples

[1833] User A likes open-air baths and wants to travel with his or her family. When logging in for the first time, he or she enters this information as attributes. The server uses a generative AI model to recommend hot spring resorts with open-air baths that are ideal for families. User A experiences a virtual hot spring tour using a VR device and selects a hot spring resort that he or she likes. The server generates a specific travel plan to that hot spring resort and proposes it to User A. User A makes a travel reservation based on the proposed plan and actually visits the resort.

[1834] Prompt Sentence Examples

[1835] "Data collection for hot spring tour experience:

[1836] Users like the open-air baths, relaxation, and family-friendly features.

[1837] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1838] System processing steps

[1839] Step 1: Collecting and mapping hot spring data

[1840] server

[1841] The server collects data about hot springs from the APIs of official hot spring websites and review websites. Input data includes hot spring location information, characteristics, and information about surrounding facilities. This data is scraped using Python's BeautifulSoup and Requests libraries to extract the necessary information. The extracted data is organized and stored in a database (AWS RDS). The database is updated regularly to reflect new and changed hot spring information. The hot spring data stored in the database is obtained as output.

[1842] Step 2: Enter user attribute information

[1843] User

[1844] The user logs in to the system and inputs attribute information such as the characteristics of their preferred hot springs, their travel style, their budget, and the people they will be traveling with. The input data is information about the user's hot spring preferences and travel. The server receives this attribute information and stores it in a database. As an output, the user attribute information is saved in the database.

[1845] Step 3: Recommend the best hot spring

[1846] server

[1847] The server uses a generative AI model (TensorFlow) to recommend the most suitable hot spring resort based on the collected hot spring data and the user's attribute information. The input data is the user's attribute information and existing hot spring data. The generative AI model processes this information and outputs a list of the most suitable hot spring resorts for the user. The output is a list of recommended hot spring resorts.

[1848] Step 4: Generate a virtual hot spring tour

[1849] server

[1850] The server generates a virtual hot spring tour based on the data of the selected hot spring resort. The input data is detailed information about the recommended hot spring resort. Unity software is used to generate 3D models and environmental acoustic data to create the virtual hot spring tour. The output is the data of the virtual hot spring tour. Specifically, a 3D model of the hot spring resort is constructed in Unity and sound effects are incorporated.

[1851] Step 5: Providing a virtual hot spring experience

[1852] Terminal

[1853] The user's device (smartphone, tablet, VR device, etc.) receives the virtual hot spring tour provided by the server and provides the user with a virtual reality hot spring experience. The input data is the virtual hot spring tour data sent from the server. The device receives this data, renders a 3D model, and plays video and audio data. The output is the user's ability to enjoy the virtual hot spring experience in VR or on their smartphone.

[1854] Step 6: Travel plan generation and support

[1855] server

[1856] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a travel plan to that hot spring resort. The input data is information about the hot spring resort selected by the user. The server generates a travel plan including information about accommodation, transportation, and surrounding tourist spots, and proposes it to the user. The output is a specific travel plan. Specific operations include searching for various information and creating an appropriate plan.

[1857] Step 7: Share and interact with your virtual hot spring experience

[1858] User

[1859] Users share screenshots and videos of their virtual hot spring experience on social media or in-app communities. The input data are images and videos taken during the virtual hot spring experience. Other users can comment on the posted content and share the same virtual hot spring experience. The output is to encourage interaction between users. Specific operations utilize the posting and commenting functions on social media and within the app.

[1860] Prompt Sentence Examples

[1861] "Data collection for hot spring tour experience:

[1862] Users like the open-air baths, relaxation, and family-friendly features.

[1863] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1864] The present invention provides a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home, plan trips to their favorite hot spring resorts, and dynamically adjust the content of the experience by recognizing the user's emotions. An embodiment of this system is described in detail below.

[1865] System Configuration

[1866] The system mainly consists of the following four main components:

[1867] 1. Server

[1868] 2. Terminal

[1869] 3. Users

[1870] 4. Emotion Engine

[1871] Implementation flow and operation of each element

[1872] 1. Hot spring data collection and mapping

[1873] server

[1874] The server collects data about hot springs from various sources (official websites, review sites, APIs, etc.) The collected data includes location information, the type of hot spring, its characteristics, information about surrounding facilities, photos, and videos.

[1875] The collected data is organized and stored in a database that is regularly updated to reflect new and changed hot spring information.

[1876] 2. Input of user attribute information and recommendations

[1877] User

[1878] When logging in to the system for the first time, users enter their personal information, including their preferred hot spring features (e.g., open-air bath, sauna, family-friendly), travel style (e.g., relaxation, sightseeing, activities), budget, and travel companions.

[1879] server

[1880] The server reads the attribute information entered by the user and runs an algorithm to match it with hot spring data stored in a database, thereby creating a list of hot spring candidates that are most suitable for the user and recommending them.

[1881] 3. Emotional engine for recognizing user emotions

[1882] Emotion Engine

[1883] While the user is entering their attribute information and experiencing the virtual hot spring tour, an emotion engine is activated that recognizes emotions from the user's facial expressions and voice. The emotion engine acquires emotion data and analyzes it in real time.

[1884] server

[1885] It receives emotional data sent from the emotion engine and dynamically adjusts recommendations and changes the content of the virtual experience.

[1886] 4. Creating a Virtual Hot Spring Tour

[1887] server

[1888] The server generates a virtual hot spring tour based on the selected hot springs, using 3D models, video footage of the actual hot springs, and audio data. The experience is customized according to the user's emotional state, as recognized by the emotion engine.

[1889] Terminal

[1890] The generated virtual hot spring tour is provided to the user's device, and the user can start the virtual hot spring tour on the device and enjoy the experience as if they were in a real hot spring.

[1891] 5. Travel planning generation and support

[1892] server

[1893] When a user experiences a virtual hot spring tour and selects a hot spring resort they like, the server automatically generates a travel plan to that hot spring resort, including details on accommodation, transportation, surrounding tourist attractions, restaurants, etc.

[1894] User

[1895] The user can review the proposed travel plan and make accommodation reservations, transportation arrangements, etc. based on the plan.

[1896] During the actual trip, the application can be used to provide navigation and tourist information.

[1897] 6. Sharing and interacting with virtual hot spring experiences

[1898] User

[1899] Users can post screenshots and videos of their virtual hot spring experience on social media and in-app communities.

[1900] Other users can comment on the posts and share the same virtual hot spring experience.

[1901] Specific examples

[1902] Below are some specific examples of how the system can be used.

[1903] Example 1:

[1904] User A likes open-air baths at hot springs and wants to travel with his family. When he logs in for the first time, he enters this information as attributes.

[1905] The server uses its database to recommend hot spring resorts with open-air baths suitable for families in areas rich in nature.

[1906] User A chooses one of the recommended hot springs and experiences a virtual hot spring tour. The emotion engine recognizes that User A is relaxed from his / her facial expressions and voice, and adds further relaxing elements.

[1907] If user A likes the hot spring resort, the server generates a specific travel plan to the hot spring resort and proposes it to user A.

[1908] Example 2:

[1909] User B uses a smartphone app to select a virtual hot spring tour to enjoy the virtual reality experience of a hot spring at the summit of Mount Fuji.

[1910] The server combines the 3D model of Mount Fuji with the model data of the hot spring and provides it to User B's terminal.

[1911] When User B puts on the VR glasses and begins the experience, the emotion engine recognizes excitement and surprise and adjusts the experience to make it even more exciting.

[1912] In this way, by using this system, users can enjoy a variety of hot spring experiences from the comfort of their own home, find the perfect hot spring resort, and plan an actual trip. Furthermore, by combining it with an emotion engine, it is possible to provide an experience optimized for the user's emotional state, and share the virtual hot spring experience with other users, creating a new platform for interaction.

[1913] The processing flow will be explained below.

[1914] Step 1:

[1915] server

[1916] Start collecting hot spring data. The server automatically collects hot spring data (location information, type of hot spring, characteristics, information on surrounding facilities, photos, videos) from various sources such as official websites, review sites, and APIs.

[1917] Step 2:

[1918] server

[1919] The collected hot spring data is organized and stored in a database. Location information, hot spring characteristics, and information on surrounding facilities are converted into an appropriate format and centrally managed.

[1920] Step 3:

[1921] User

[1922] When logging in to the system for the first time, users enter their demographic information, including their preferred hot spring features (e.g., open-air bath, sauna, family-friendly, etc.), travel style (e.g., relaxation, sightseeing, activities, etc.), budget, and travel companions.

[1923] Step 4:

[1924] server

[1925] The server analyzes the attribute information entered by the user, and then executes a matching algorithm based on the hot spring data in the database and the user's attribute information to generate the most suitable hot spring candidates.

[1926] Step 5:

[1927] server

[1928] The generated list of hot spring candidates is recommended to the user, allowing the user to check a list of hot springs that suit their preferences.

[1929] Step 6:

[1930] User

[1931] Check the list of recommended hot springs through your device and select the hot spring resort that interests you.

[1932] Step 7:

[1933] Emotion Engine

[1934] When a user selects a hot spring resort or experiences a virtual hot spring tour, the system analyzes the user's facial expressions and voice in real time to generate emotional data.

[1935] Step 8:

[1936] server

[1937] It receives emotional data from the emotion engine and dynamically adjusts the hot spring recommendation. For example, if the user is excited, it will suggest an active experience, and if the user is relaxed, it will suggest a quiet hot spring spot.

[1938] Step 9:

[1939] server

[1940] It generates a virtual hot spring tour based on the selected hot spring, integrating 3D models, actual video and audio data, and customizing the experience based on emotional data.

[1941] Step 10:

[1942] Terminal

[1943] The generated virtual hot spring tour data is received and provided to the user, who then begins the virtual hot spring tour using a smartphone or VR glasses.

[1944] Step 11:

[1945] User

[1946] Experience the virtual hot spring tour and enjoy the atmosphere of the hot spring resort. The emotion engine continuously analyzes the user's emotions and adjusts the experience in real time.

[1947] Step 12:

[1948] User

[1949] After experiencing the virtual hot spring tour, users can select their favorite hot spring resort. The emotion engine analyzes the user's satisfaction and uses the results to provide feedback for future recommendations and experience adjustments.

[1950] Step 13:

[1951] server

[1952] Generate a specific itinerary to the selected hot spring resort, including details on accommodation, transportation, nearby tourist attractions, restaurants, etc.

[1953] Step 14:

[1954] server

[1955] The generated travel plan is proposed to the user, who can then confirm the proposed plan and proceed with reservations and arrangements.

[1956] Step 15:

[1957] User

[1958] Use the application during your actual trip to receive real-time support for navigation and tourist information.

[1959] Step 16:

[1960] User

[1961] Post screenshots and videos of your virtual hot spring experience on social media and in-app communities to interact with other users.

[1962] Step 17:

[1963] User

[1964] Users can view virtual hot spring experiences posted by other users and create a new forum for interaction through comments and reactions.

[1965] In this way, this system starts by collecting hot spring data, then provides recommendations based on the user's attribute information and emotional data, creates and experiences a virtual hot spring tour, proposes specific travel plans, and finally shares and interacts with the virtual hot spring experience, all in a single flow.

[1966] Example 2

[1967] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1968] Conventional hot spring information systems make it difficult for users to enjoy a variety of hot spring experiences from home. Furthermore, they lacked the ability to dynamically adjust the experience based on the user's emotional state, which prevented them from fully improving user satisfaction. Furthermore, it was time-consuming and inefficient for users to plan specific trips to their favorite hot spring resorts.

[1969] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1970] In this invention, the server includes means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database, means for inputting user attribute information and recommending optimal hot springs based on the attribute information, means equipped with an emotion engine that acquires user emotion data and analyzes it in real time and dynamically adjusts recommendations and virtual experience content based on the emotion data, means for generating a virtual hot spring tour based on the selected hot springs and providing it to the user's terminal, means for automatically generating a specific travel plan to the user's favorite hot springs and proposing the travel plan to the user, and means for sharing information about the virtual hot spring experience and the actual trip with other users and promoting communication. This allows users to enjoy a variety of hot spring experiences from the comfort of their own home, dynamically adjust the experience according to their emotional state, and efficiently plan their travel by finding the optimal hot spring resort.

[1971] "Hot spring data" is information about hot spring facilities, including location information, characteristics, information about surrounding facilities, photos, videos, and the like.

[1972] The "database" is an information management system for organizing and storing collected hot spring data in a unified format.

[1973] "User" refers to an individual who uses the system to receive hot spring information recommendations and experience virtual hot spring tours.

[1974] "Attribute information" refers to personal data such as the user's preferences, travel style, budget, and traveling companions.

[1975] "Recommendation" refers to the act of recommending the most suitable hot spring facility based on the user's attribute information.

[1976] The "emotion engine" is a system that acquires emotional data from the user's facial expressions, voice, etc. and analyzes it in real time.

[1977] The "Virtual Hot Spring Tour" is virtual reality content generated using 3D models, actual hot spring footage, and audio data, allowing users to enjoy a hot spring experience on their device.

[1978] A "terminal" is a device used by a user to experience a virtual hot spring tour, and includes a VR device, a mobile terminal, etc.

[1979] A "travel plan" is a detailed plan that automatically generates a specific travel schedule to a user's favorite hot spring resort, including information on accommodation, transportation, and surrounding tourist attractions.

[1980] "Information sharing" refers to the act of sharing information about virtual hot spring experiences and real trips with other users to promote interaction.

[1981] "Dynamic adjustment" refers to changing recommendations and virtual experience content in real time based on the user's emotional state.

[1982] A "hot spring facility" is a place that provides hot bathing services and includes multiple types of baths, such as open-air baths and saunas.

[1983] The present invention is a system that allows users to enjoy a variety of hot spring experiences from the comfort of their own home and plan trips to their favorite hot spring resorts. This system is primarily composed of four main components: a server, a terminal, a user, and an emotion engine. An embodiment of this system is described in detail below.

[1984] Hot spring data collection

[1985] server

[1986] The server collects information about hot springs through external APIs such as the Google Places API and TripAdvisor API. The collected data includes the hot spring's name, location, type and characteristics, information about surrounding facilities, photos, and videos. The collected data is organized using a Python script and stored in a MySQL database. This process is performed periodically, and new information is automatically updated.

[1987] Input of user attribute information and recommendations

[1988] User

[1989] When users log in to the system for the first time, they enter attribute information such as their preferred hot spring characteristics, travel style, budget, and travel companion information through a dedicated form. This form is built with React.js.

[1990] server

[1991] The server parses the information entered by the user in JSON format and stores it in a database. Then, using machine learning algorithms such as Pandas and Scikit-learn, it matches this attribute information with hot spring data. The algorithm then lists and recommends the hot springs that are most suitable for the user. This recommendation is made in the form of providing a list of the top five hot spring facilities based on the filter criteria.

[1992] Recognizing user emotions with an emotion engine

[1993] Emotion Engine

[1994] While the user is entering their personal information or experiencing the virtual hot spring tour, their facial expressions and voice are captured via a webcam and microphone. Using AWS Rekognition and Google Cloud Vision, emotions are analyzed in real time. This emotional data is then sent to a server.

[1995] server

[1996] The server analyzes t...

Claims

1. A means for collecting hot spring data and storing location information, characteristics, and information on surrounding facilities in a database; A means for inputting attribute information of a user and recommending the most suitable hot spring based on the attribute information; A means for generating a virtual hot spring tour based on the selected hot springs and providing the tour to a user's terminal; means for automatically generating a specific travel plan to a hot spring that the user likes and proposing the travel plan to the user; A means to share virtual hot spring experiences and real travel information with other users and promote interaction, A system including:

2. The system according to claim 1, further comprising means for generating a virtual hot spring tour using the 3D model and environmental acoustic data and providing the tour to a user's VR device or mobile terminal.

3. The system according to claim 1, further comprising means for having an algorithm for learning attribute information such as a user's hot spring preferences, travel style, budget, and companions, and for recommending optimal hot spring candidates.

Images