system

Abstract

A system is provided.SOLUTION: A system including means for allowing a user to scan a standing picture panel installed in a tourist spot with a device such as a smartphone, means for allowing the device to transmit position information and QR Code information of the user to a server, means for allowing the server to analyze the user's interest, preference, and past visit history based on these pieces of information and generate a sightseeing route and a mission individually suitable for the user using a generative AI model, means for allowing the server to transmit the generated sightseeing route and mission to the device, and means for allowing the device to display a personalized guide character and message using AR technology.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional sightseeing tours and stamp rallies have the problem that their appeal fades once experienced, making it difficult to encourage tourists to return. This has resulted in insufficient efforts to revitalize tourist destinations or improve tourist satisfaction. Furthermore, it has been difficult to provide personalized experiences tailored to the preferences and interests of individuals and groups, and large amounts of tourist information and guides have generally been provided. Furthermore, the lack of real-time feedback and dynamic tourist route generation has resulted in a low quality user experience. [Means for solving the problem]

[0005] To address the above-mentioned challenges, the present invention provides a sightseeing tour provision system that utilizes a generative AI model and AR technology. Specifically, the system includes a means for a user to scan a character panel installed at a tourist attraction with a device such as a smartphone, a means for the device to transmit the user's location information and QR code (registered trademark) information to a server, and a means for the server to analyze the user's interests, preferences, and past visit history based on this information and generate individually tailored sightseeing routes and missions using a generative AI model. The system also includes a means for the server to transmit the generated sightseeing routes and missions to the device, and a means for the device to display personalized guide characters and messages using AR technology. This system continuously provides users with new entertainment and encourages them to return to tourist attractions. Furthermore, the device transmits the user's current location to the server in real time, and the server analyzes the user's progress in real time to dynamically generate the next tourist spot and mission, improving the quality of the user experience. The generative AI model generates sightseeing routes and missions taking past user data and current trends into account, and the system uses push notifications to guide users to next steps and recommended tourist spots, thereby providing a personalized sightseeing experience tailored to the user's needs.

[0006] "User" refers to an individual or group visiting a tourist destination who uses this system to experience sightseeing tours and missions.

[0007] A "terminal" is a device used by a user, such as a smartphone or tablet, that connects to this system to send and receive information and display AR content.

[0008] The "server" is the central computer system of this system, which analyzes user information, operates the generative AI model, generates tourist routes, and provides missions.

[0009] A "generative AI model" is a type of artificial intelligence that refers to an algorithm or program that generates optimal tourist routes and missions based on a user's interests and preferences, past visit history, and current trends.

[0010] "AR technology" is an abbreviation for augmented reality technology, and refers to technology that allows users to overlay virtual information and characters onto real-world scenery.

[0011] A "tourist route" refers to a plan that specifies routes and visiting spots for users to efficiently travel around tourist spots.

[0012] "Mission" refers to the tasks and activities provided to users to help them enjoy their experience at tourist spots more.

[0013] A "personalized guide character" is a virtual character that is displayed based on the user's profile and interests, and provides sightseeing information and guides on missions.

[0014] A "standing picture panel" is a fixed sign or poster installed at a tourist spot, which includes a marker that users can scan with a terminal to obtain information.

[0015] "Push notification" refers to a communication method in which a server sends a notification to a device to inform the user of new information or next steps. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that utilizes generative AI models and AR technology to provide users with individually tailored sightseeing tours. The system generates personalized sightseeing routes and missions based on the user's preferences, interests, and past visit history, and delivers them in real time.

[0038] System configuration

[0039] The system mainly consists of the following components:

[0040] 1. User's device (smartphone, etc.)

[0041] 2. Server

[0042] 3. Generative AI Models

[0043] 4. Character portrait panels as AR markers

[0044] Explanation of program processing

[0045] 1. Collecting User Information

[0046] The user finds a portrait panel installed at a tourist spot.

[0047] The user starts up the device (smartphone) and opens the dedicated app.

[0048] The device's camera will activate and scan the QR code or AR marker on the character portrait panel.

[0049] The device uses location information services to obtain the current location.

[0050] The device sends the acquired information (QR code information, current location) to the server.

[0051] 2. Analyzing data and applying generative AI models

[0052] The server compares the user's information with a database to obtain the user's past visit history and preference data.

[0053] The server uses a generative AI model to analyze this data and generate sightseeing routes and missions based on the user's interests.

[0054] Generative AI models take into account historical data and current trends to create optimal plans.

[0055] The server sends the generated sightseeing routes and missions to the device.

[0056] 3. Delivering personalized AR content

[0057] The device analyzes the tourist routes and missions received and displays notifications to the user.

[0058] The user travels along a route selected by the user and arrives at the designated tourist spot.

[0059] The device uses AR technology to display personalized guide characters and messages from a portrait panel.

[0060] Example: When you arrive at a specific spot, a guide character will say, "This is your next destination."

[0061] 4. Real-time feedback

[0062] The device periodically acquires its current location and transmits it to the server.

[0063] The server analyzes the user's progress in real time and dynamically generates next steps and recommended tourist spots.

[0064] The server sends the latest instructions to the terminal and notifies the user.

[0065] Example: Send a notification that says, "If you stop by a specific cafe on your way to the next location, a hidden mission will occur."

[0066] Specific examples

[0067] Example 1: Family sightseeing

[0068] A user (family) visits a tourist spot and scans a portrait panel.

[0069] The device sends information to a server, which then analyzes the family's past history and preferences.

[0070] The generative AI model suggests "experiential activities for parents and children to enjoy together" and "nature walks and picnic spots."

[0071] The device will notify the family of the mission it has received, and a guide character will provide guidance using AR technology.

[0072] Example 2: Solo tourism

[0073] The user (traveling alone) visits a tourist spot and scans the portrait panel.

[0074] The device sends the information to the server, which takes into account past visits and interests.

[0075] The generative AI model suggests "tours of historical sites" and "hidden cafes."

[0076] The terminal notifies the received mission and moves along the route selected by the user.

[0077] When you arrive at a specific spot, a guide character will guide you to your next destination.

[0078] The above is a specific description of the embodiment of the present invention. This system allows users to have new sightseeing experiences and encourages them to revisit tourist spots.

[0079] The processing flow will be explained below.

[0080] Step 1:

[0081] The user discovers a standing picture panel installed at a tourist spot.

[0082] The user launches the dedicated app on their device and points the camera at the character portrait panel.

[0083] Step 2:

[0084] The device uses the camera to scan the QR code or AR marker on the character portrait panel.

[0085] The device acquires the location information and sends it to the server along with the QR code information.

[0086] Step 3:

[0087] Based on the user information received by the server, the user's past visit history and preference data are obtained from the database.

[0088] The server analyzes this information and inputs it into the generative AI model.

[0089] Step 4:

[0090] The generative AI model takes into account the analysis results and current trends to generate optimal sightseeing routes and missions for users.

[0091] The server formats the generated tourist routes and missions and sends them to the terminal.

[0092] Step 5:

[0093] The device displays the sightseeing route and missions received from the server,

[0094] Notify the user.

[0095] The user confirms the notification and starts the tour.

[0096] Step 6:

[0097] When the device reaches a designated tourist spot, AR technology is used to display a guide character or message from a portrait panel.

[0098] A guide character will guide you to your next steps and tourist spots.

[0099] Step 7:

[0100] The terminal periodically acquires the user's current location and transmits it to the server.

[0101] The server analyzes your progress in real time and dynamically generates new attractions and missions.

[0102] Step 8:

[0103] The server sends the latest instructions and information to the terminal and notifies the user.

[0104] Example: New missions and recommended spots will be announced via push notifications.

[0105] Step 9:

[0106] The user checks the notification from the device and moves to the next spot.

[0107] The device continuously captures the user's location and progress and sends feedback to the server.

[0108] Step 10:

[0109] The server dynamically updates next steps and new tourist spots based on feedback.

[0110] The above is the specific program processing flow of the system. Through these processing steps, users can have a sightseeing experience that is tailored to their individual needs.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] Conventional tourism systems provide uniform tourist information, making it difficult to provide a personalized experience tailored to each user's individual preferences and interests. Furthermore, they lack the functionality to track a user's progress in real time and dynamically suggest the next tourist spot or mission, forcing users to follow a predetermined route. Furthermore, they lacked a mechanism to provide new experiences when a user arrives at a specific tourist spot. These issues need to be resolved.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes: a means for a user to read a display panel installed at a tourist attraction with an information terminal; a means for the terminal to transmit the user's location information and identification information to the server; a means for the server to analyze the user's preferences and past history based on the received information and generate personalized tourist routes and missions using a generative AI model; a means for the server to transmit the generated tourist routes and missions to the terminal; a means for the terminal to display personalized guide characters and messages using augmented reality technology; a means for the terminal to continuously transmit the user's current location to the server, and the server to dynamically analyze the user's progress and generate the next tourist spot and mission; and a means for the generative AI model to generate tourist routes and missions taking past user data and current trends into consideration and to guide the user to the next step and recommended tourist spots via push notifications. This makes it possible to provide a personalized tourist experience for each user, and to dynamically suggest the next step by analyzing the user's progress in real time.

[0116] "User" refers to an individual who uses this system to obtain information at a tourist spot and enjoy sightseeing.

[0117] "Display panel" refers to a panel installed at a tourist spot that contains a QR code or AR marker that users can use to obtain information.

[0118] "Information terminal" refers to an electronic device such as a smartphone or tablet that a user owns.

[0119] "Location Information" refers to information about a user's current location obtained using GPS or other means.

[0120] "Identification information" refers to unique information obtained by scanning the QR code or AR marker printed on the display panel.

[0121] "Server" refers to a computer system for receiving and analyzing data sent by users.

[0122] A "generative AI model" refers to an artificial intelligence system that analyzes user preferences, past history, current trends, etc. to generate tourist routes and missions.

[0123] "Tourist route" refers to a specific route suggested by the server to enable the user to efficiently tour tourist spots.

[0124] A "mission" refers to a specific task or activity that a user must accomplish at a tourist destination.

[0125] "Augmented reality technology" refers to a technology that overlays digital information on a device that displays the real world.

[0126] "Guide character" refers to a virtual character that provides sightseeing guidance to users using augmented reality technology.

[0127] "Message" refers to notifications and guidance information displayed to the user by the guide character or the system.

[0128] "Push notification" refers to a message sent in real time from a server to a user's information terminal.

[0129] "Trends" refers to information about current trends and popular spots.

[0130] "Continuous" means that the information is updated at regular intervals.

[0131] "Dynamic" refers to the way something changes in real time depending on the situation.

[0132] The present invention is a system that provides users with individually tailored tourist experiences at tourist destinations. This system uses generative AI models and augmented reality technology to generate personalized tourist routes and missions based on the user's preferences and past visit history, and provides them in real time. Specific embodiments of this system are described below.

[0133] 1. Collecting User Information

[0134] Users first find a display panel installed at a tourist spot and begin gathering information by scanning it with an information terminal (e.g., a smartphone). The user then launches a dedicated app on their smartphone and uses the camera to scan the QR code or AR marker on the display panel. The device then uses GPS to obtain the user's current location and sends it to the server along with the QR code information. The location information service uses the Google (registered trademark) Maps API.

[0135] 2. Analyzing data and applying generative AI models

[0136] The server compares the user information sent from the device with a database (e.g., MySQL®, MongoDB) to obtain the user's past visit history and preference data. The server then uses a generative AI model (e.g., GPT-4®) to analyze this data and generate sightseeing routes and missions based on the user's interests. Examples of prompts are as follows:

[0137] "Spots visited in the past: Mount Takao, Interests: Historical landmarks. Based on this, please suggest the next tourist spot and mission."

[0138] The generative AI model takes into account past user data and current trends to create an optimal plan, and the generated sightseeing route and missions are sent from the server to the device.

[0139] 3. Delivering personalized AR content

[0140] The device analyzes the tourist routes and missions received from the server and displays them to the user as push notifications. The user then moves toward the designated tourist spots based on the notifications. Upon arriving at each spot, the device uses augmented reality technology (e.g., ARKit, ARCore) to display a personalized guide character and message on the display panel. For example, when arriving at a specific spot, a guide character will appear saying, "This is your next destination."

[0141] 4. Real-time feedback

[0142] As the user travels along a sightseeing route, the device periodically acquires its current location and sends it to the server. The server analyzes the user's progress in real time and dynamically generates next steps and recommended sightseeing spots. These results are then sent back to the device and displayed as notifications to the user. For example, a notification might say, "If you stop by a specific cafe on your way to the next spot, a hidden mission will occur."

[0143] Specific examples

[0144] For example, suppose a family visits a tourist spot and scans the display panel. The device sends the information to the server, which analyzes the family's past history and preferences. The generative AI model suggests "hands-on activities for parents and children" and "nature walks and picnic spots." The device notifies the family of the mission received, and a guide character uses augmented reality technology to provide guidance.

[0145] For solo travelers, when they visit a tourist spot, they scan the display panel, and the device sends the information to the server. The server takes into account their past visit history and interests, and the generative AI model suggests "tours of historical landmarks" and "hidden cafes." The device notifies the user of the received mission, and as they travel along the route selected by the user, a guide character will guide them to their next destination when they arrive at a specific spot.

[0146] This system allows users to have new sightseeing experiences and encourages them to revisit tourist destinations.

[0147] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0148] The flow of this system's program processing

[0149] Step 1: The user finds the display panel

[0150] Input: Display panels installed in various locations in tourist spots

[0151] Action: The user finds the display panel.

[0152] Output: Your smartphone is ready to get the information from the display panel.

[0153] Specific actions: The user walks around a tourist spot, searches for a display panel, and when they find one, takes out their smartphone.

[0154] Step 2: The user starts the device and opens the dedicated app.

[0155] Input: A smartphone owned by the user

[0156] Process: The user takes out their smartphone and launches the dedicated app.

[0157] Output: The dedicated app will launch and you will be ready to scan.

[0158] Specific operation: The user touches the smartphone screen, selects and opens the dedicated app.

[0159] Step 3: Scan the QR code or AR marker on the character panel with your device

[0160] Input: QR code or AR marker placed on the display panel

[0161] Processing: Activate the smartphone camera and scan the QR code or AR marker.

[0162] Output: QR code and AR marker information is obtained

[0163] Specific operation: The user points the smartphone camera at the display panel and scans the QR code or AR marker.

[0164] Step 4: The device acquires location information and sends it to the server

[0165] Input: Current location information of the device using GPS, as well as information on QR codes and AR markers

[0166] Processing: Send location information and scanned information to the server

[0167] Output: Location information and QR code information are sent to the server

[0168] Specific operation: The device uses location services (e.g., Google Maps API) to obtain the current location and sends the obtained information to the server via HTTPS protocol.

[0169] Step 5: The server checks the user information against the database

[0170] Input: Location information and QR code information sent from the device

[0171] Processing: The server checks the database to obtain the user's past visit history and preference data.

[0172] Output: User's past visit history and preference data

[0173] Specific operation: The server accesses a database (e.g., MySQL, MongoDB) and retrieves the information to be matched.

[0174] Step 6: The server generates tourist routes and missions using the generative AI model

[0175] Input: User's past visit history, preference data, current location information

[0176] Processing: Input prompts into the generative AI model to generate personalized sightseeing routes and missions.

[0177] Output: personalized tourist routes and missions

[0178] Specific operation: An example of a prompt sentence: "Spots visited in the past: Mt. Takao, Interests: Historical buildings. Based on this, please suggest the next tourist spot and mission." is input into a generative AI model (e.g., GPT-4) and the generated results are obtained.

[0179] Step 7: The server sends the generated results to the device.

[0180] Input: Tourist routes and missions generated by generative AI models

[0181] Process: Send the generated results to the terminal

[0182] Output: Tourist route and mission data sent to the device

[0183] Specific operation: The server sends the generated results to the terminal using the HTTPS protocol.

[0184] Step 8: The device analyzes the sightseeing route and mission and displays a notification

[0185] Input: Tourist route and mission data sent from the server

[0186] Processing: The device analyzes the data and displays a notification to the user

[0187] Output: The notification displayed to the user

[0188] Specific operation: The device analyzes the sightseeing route and mission data and displays a notification saying, "Your next destination is XX Park."

[0189] Step 9: User follows notification

[0190] Input: Device notifications

[0191] Action: The user follows the notification and moves to the designated tourist spot.

[0192] Output: The user arrives at the specified location.

[0193] Specific behavior: The user checks the smartphone screen and moves according to the displayed route.

[0194] Step 10: The device uses AR technology to display personalized content.

[0195] Input: User location information, AR content data

[0196] Processing: The device displays a guide character or message using augmented reality technology.

[0197] Output: Personalized AR content displayed to the user

[0198] Specific operation: When the user arrives at the designated spot, a guide character will be displayed using the device's camera, saying, "This is your next destination."

[0199] Step 11: The device periodically acquires location information and sends it to the server.

[0200] Input: current location of the user

[0201] Processing: The device periodically acquires location information using GPS and sends it to the server.

[0202] Output: Location information sent to the server

[0203] Specific operation: The device acquires its current location at regular intervals and sends that information to the server.

[0204] Step 12: The server analyzes the progress and generates the next step

[0205] Input: current location of the user

[0206] Processing: The server analyzes the progress and generates the next attractions and missions.

[0207] Output: Data on the next tourist spot or mission

[0208] Specific operation: The server uses the generative AI model to generate the next route and mission based on the user's current location.

[0209] Step 13: The server sends the latest instructions to the terminal and notifies the user.

[0210] Input: Data for the next tourist spot or mission

[0211] Processing: The server sends the latest instructions to the terminal and notifies the user.

[0212] Output: Notification displayed on the user's device

[0213] Specific operation: The server generates the next step and sends it to the device, which then displays a notification saying, "If you stop by a specific cafe on your way to the next spot, a hidden mission will occur."

[0214] (Application example 1)

[0215] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0216] There is a demand for tourist destinations and brick-and-mortar stores to provide users with personalized information based on their preferences, interests, and past visit history, thereby providing a more fulfilling experience. However, it is difficult for users to efficiently and intuitively obtain information and find optimal routes and recommended products without receiving a large amount of information all at once. Current systems do not adequately provide individually tailored information, making it difficult for users to obtain information that perfectly matches their interests. Therefore, the objective of this invention is to provide a system that can provide users with individually tailored sightseeing routes and shopping routes in real time.

[0217] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0218] In this invention, the server includes means for analyzing the user's interests, preferences, and past visit history based on the user's location information and QR code information, means for generating individually suited sightseeing routes and missions using a generative AI model, and means for displaying AR content based on AR markers detected by the device's camera, thereby enabling the provision of personalized sightseeing routes, shopping routes, and recommended products to the user.

[0219] "User" refers to an individual who uses this system to receive information at tourist spots and brick-and-mortar stores.

[0220] "Terminal" refers to a device carried by a user, such as a smartphone, smart glasses, or head-mounted display, that acquires QR code information and location information and displays AR content.

[0221] "Server" refers to the central processing unit that processes user information and generates sightseeing routes, missions, and shopping routes using generative AI models.

[0222] "Generative AI model" refers to an artificial intelligence model that analyzes a user's interests and preferences, past visit history, and current trends to generate individually appropriate tourist routes and missions.

[0223] "AR technology" refers to augmented reality technology, which displays digital content overlaid on real-world information.

[0224] A "tourist route" refers to the optimal order or route for a user to visit tourist spots.

[0225] A "mission" refers to a specific action or task presented to a user.

[0226] "QR code" refers to a two-dimensional barcode consisting of a rectangular pattern of dots, and is a format for encoding information.

[0227] "AR marker" refers to a printed or digital image that serves as a reference for displaying content using augmented reality technology.

[0228] "Location Information" means a user's current geographic location, which may be obtained using GPS or other location-determining technology.

[0229] "Progress" refers to information indicating how far a user has progressed along a given sightseeing or shopping route.

[0230] A "shopping route" refers to the optimal order or route for a user to efficiently shop in a physical store.

[0231] "Benefit information" refers to information such as discounts, campaigns, and bonuses offered to users.

[0232] This invention is a system for providing users with personalized sightseeing and shopping routes at tourist spots and brick-and-mortar stores. The system utilizes generative AI models and AR technology to provide individually tailored information based on the user's preferences, interests, and past visit history.

[0233] System configuration

[0234] The system mainly consists of the following components:

[0235] 1. User's device (smartphone, smart glasses, head-mounted display, etc.)

[0236] 2. Server

[0237] 3. Generative AI Models

[0238] 4. AR markers (including QR codes)

[0239] User's device

[0240] The user's device may be a smartphone, smart glasses, or a head-mounted display. These devices have a camera function and can scan AR markers and QR codes installed at tourist spots and in brick-and-mortar stores. When the device scans these markers, the acquired data along with location information is sent to a server. Furthermore, the device uses AR technology to display personalized guide characters and messages.

[0241] server

[0242] The server has the ability to analyze the user's interests, preferences, and past visit history based on the user's location information and QR code information. A generative AI model running on the server takes this data and current trends into account to generate optimal sightseeing and shopping routes. The generated routes and missions are then sent to the user's device in real time.

[0243] Generative AI Models

[0244] A generative AI model is an artificial intelligence model that analyzes a user's interests, preferences, and past visit history to personalize sightseeing or shopping routes. This model uses user data and current trends to generate an optimal plan and sends it to the user's device via a server. An example of a specific prompt is, "User 'User123' has the following past purchase history: chairs, tables, and shelves. Taking into account the latest trends, please recommend new furniture that may be of interest to them. Also, please generate the optimal shopping route within the store."

[0245] AR technology

[0246] AR technology is a technology that overlays digital content onto the real world through the user's device. When a user scans an AR marker in a tourist spot or physical store, personalized guide characters, product information, and messages are displayed on the device screen. This plays an important role in improving the user experience.

[0247] Specific examples

[0248] Example 1: Family sightseeing

[0249] A user (family) visits a tourist spot and scans a character panel. The device sends information to the server, which analyzes the family's past history and preferences. The generative AI model suggests "hands-on activities for parents and children to enjoy together" and "nature walks and picnic spots." The device notifies the family of the mission it received, and a guide character uses AR technology to provide guidance.

[0250] Example 2: Solo tourism

[0251] A user (solo traveler) visits a tourist spot and scans a character panel. The device sends the information to the server, which takes into account past visit history and interests. The generative AI model suggests "tours of historical landmarks" and "hidden cafes." As the user travels along the route selected, a guide character will guide them to their next destination when they arrive at a specific spot.

[0252] The components and operation procedures described above allow users to have a more fulfilling experience at tourist spots and brick-and-mortar stores.

[0253] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0254] Step 1:

[0255] The user uses a device to scan AR markers or QR codes installed at tourist spots or brick-and-mortar stores. The device's camera acquires the QR code or AR marker information, and collects location information along with this information. The input is "QR code information" and "location information." This data is then combined into a single communication packet for data processing. The output is "data that is a packet of the user's QR code information and location information."

[0256] Step 2:

[0257] The QR code information and location information collected by the device are sent to the server. The input is "data that is a packet of the user's QR code information and location information." The processing involves sending the data using an HTTP request. The output is "user data sent to the server."

[0258] Step 3:

[0259] The server analyzes the user's QR code information and location information received. The input is "user data sent to the server." Data calculations use database matching and a data analysis engine to obtain the user's past visit history and preference data. The output is "the analysis results of the user information."

[0260] Step 4:

[0261] The server uses the generative AI model to generate individually tailored sightseeing and shopping routes based on the user's past visit history and preference data. The input is the "analysis results of user information." The data is passed to the generative AI model for data processing and calculation, generating multiple personalized options. As a concrete example, the prompt statement used is "Please recommend tourist spots of interest to user 'User123' based on his past visit history." The output is the "generated sightseeing and shopping routes."

[0262] Step 5:

[0263] The server sends the generated sightseeing route or shopping route to the user's device. The input is the "generated sightseeing route or shopping route." The processing involves sending the data using an HTTP response. The output is the "data sent to the user's device."

[0264] Step 6:

[0265] The device analyzes the tourist route and shopping route data received and displays a notification to the user. The input is "data sent to the user's device." Data processing involves reformatting the data into a notification format. The output is "a notification message displayed to the user."

[0266] Step 7:

[0267] When the user arrives at the designated tourist spot or inside the store, the device's camera scans the AR marker again and displays the AR content. The input is "AR marker information." The data is processed by selecting and displaying the appropriate AR content based on the marker information. The output is "AR content displayed on the device."

[0268] Step 8:

[0269] The device periodically acquires the current location and sends it to the server, allowing the user's progress to be analyzed in real time. The input is "current location data." The data is analyzed in real time as part of the process, and information about the next tourist spot and special offers is dynamically generated. The output is "next step and special offer information tailored to the user's progress."

[0270] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0271] This invention is a system that combines generative AI models, AR technology, and an emotion engine to provide users with individually tailored sightseeing tours. The system generates personalized sightseeing routes and missions based on the user's preferences and past visit history, and provides them in real time. It also has the ability to recognize the user's emotional state and dynamically adjust the sightseeing experience based on that.

[0272] System configuration

[0273] The system mainly consists of the following components:

[0274] 1. User's device (smartphone, etc.)

[0275] 2. Server

[0276] 3. Generative AI Models

[0277] 4. Character portrait panels as AR markers

[0278] 5. Emotion Engine

[0279] Explanation of program processing

[0280] 1. Collecting User Information

[0281] The user discovers a standing image panel installed at a tourist spot and launches a dedicated app on their device.

[0282] The device's camera scans the QR code or AR marker on the character portrait panel to obtain location information.

[0283] The device sends the QR code information and location information to the server.

[0284] 2. Analyzing data and applying generative AI models

[0285] The server compares the received user information with a database to obtain the user's past visit history and preference data.

[0286] The server uses a generative AI model to analyze this data and generate sightseeing routes and missions based on the user's interests.

[0287] Generative AI models take into account past data and current trends to create optimal plans.

[0288] The server formats the generated tourist routes and missions and sends them to the terminal.

[0289] 3. Delivering personalized AR content

[0290] The terminal displays the sightseeing route and mission received from the server and notifies the user.

[0291] The user confirms the notification and starts the tour.

[0292] When the device reaches a designated tourist spot, AR technology is used to display a guide character or message from a portrait panel.

[0293] 4. Applying the Emotion Engine

[0294] The device analyzes the user's facial expressions and voice using an emotion engine to recognize their emotional state in real time.

[0295] The device transmits the emotion data to the server.

[0296] The server dynamically adjusts sightseeing routes and missions based on emotion data, and suggests next spots and new activities.

[0297] 5. Real-time feedback

[0298] The device periodically acquires its current location and transmits it to the server.

[0299] The server analyzes the user's progress and emotional state in real time and dynamically generates next steps and recommended tourist spots.

[0300] The server sends the latest instructions and information to the terminal and notifies the user.

[0301] Specific examples

[0302] Example 1: Family sightseeing

[0303] A user (family) visits a tourist spot and scans a portrait panel.

[0304] The device sends information to a server, which then analyzes the family's past history and preferences.

[0305] The generative AI model suggests "experiential activities for parents and children to enjoy together" and "nature walks and picnic spots."

[0306] The device will notify the family of the mission it has received, and a guide character will provide guidance using AR technology.

[0307] The emotion engine recognizes a child's state of excitement and dynamically suggests relaxing spots.

[0308] Example 2: Solo tourism

[0309] The user (traveling alone) visits a tourist spot and scans the portrait panel.

[0310] The device sends the information to the server, which takes into account past visits and interests.

[0311] The generative AI model suggests "tours of historical sites" and "hidden cafes."

[0312] The terminal notifies the received mission and moves along the route selected by the user.

[0313] When you arrive at a specific spot, a guide character will guide you to your next destination.

[0314] The emotion engine recognizes the user's level of satisfaction from their facial expressions and suggests more stimulating spots.

[0315] This concludes the detailed description of the embodiment of the present invention. This system allows users to have new sightseeing experiences and encourages repeat visits to tourist spots. Furthermore, the introduction of an emotion engine makes it possible to provide experiences based on the user's emotions, realizing more personalized services.

[0316] The processing flow will be explained below.

[0317] Step 1:

[0318] The user discovers a standing picture panel installed at a tourist spot.

[0319] The user launches the dedicated app on their device (smartphone, etc.) and points the camera at the character portrait panel.

[0320] Step 2:

[0321] The device uses the camera to scan the QR code or AR marker on the character portrait panel.

[0322] The device acquires location information and sends the QR code information and location information to the server.

[0323] Step 3:

[0324] Based on the user information received by the server, the user's past visit history and preference data are obtained from the database.

[0325] The server analyzes this data and inputs it into a generative AI model.

[0326] Step 4:

[0327] The generative AI model takes into account the analysis results and current trends to generate optimal sightseeing routes and missions for users.

[0328] The server formats the generated tourist routes and missions and sends them to the terminal.

[0329] Step 5:

[0330] The terminal displays the sightseeing route and mission received from the server and notifies the user.

[0331] The user confirms the notification and starts the tour.

[0332] Step 6:

[0333] When the device reaches a designated tourist spot, AR technology is used to display a guide character or message from a portrait panel.

[0334] A guide character will guide you to your next steps and tourist spots.

[0335] Step 7:

[0336] The device analyzes the user's facial expressions and voice data using an emotion engine to recognize their emotional state in real time.

[0337] The device transmits the emotion data to the server.

[0338] Step 8:

[0339] The server dynamically adjusts sightseeing routes and missions based on emotion data, and suggests next spots and new activities.

[0340] Example: The server detects the user's fatigue and suggests a spot where they can relax.

[0341] Step 9:

[0342] The terminal periodically acquires the user's current location and transmits it to the server.

[0343] The server analyzes the user's progress and emotional state in real time and dynamically generates next steps and recommended tourist spots.

[0344] Step 10:

[0345] The server sends the latest instructions and information to the terminal and notifies the user.

[0346] Example: New missions and recommended spots will be announced via push notifications.

[0347] Step 11:

[0348] The user checks the notification from the device and moves to the next spot.

[0349] The device continuously captures the user's location and progress and sends feedback to the server.

[0350] Step 12:

[0351] The server dynamically updates next steps and new tourist spots based on feedback.

[0352] For example: Suggest a special activity at a cafe on the way to your next destination.

[0353] The above is the specific program processing flow of the system that combines the emotion engine. This system allows users to have a sightseeing experience that is tailored to their individual needs, and can encourage them to revisit tourist spots.

[0354] Example 2

[0355] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0356] In recent years, there has been a growing demand for personalized tourism experiences tailored to the preferences and interests of each individual tourist. However, conventional tourist information systems have not been able to adequately address the individual needs of tourists. As a result, it has been difficult to provide tourists with a more fulfilling tourism experience and maximize the appeal of tourist destinations. Furthermore, there has been no system that can recognize the emotional state of users in real time while sightseeing and dynamically adjust sightseeing routes and missions accordingly.

[0357] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0358] In this invention, the server includes a means for a user to scan a character image panel installed at a tourist spot with a mobile communication terminal, a means for the terminal to transmit the user's location information and QR code information to a data center, a means for the data center to analyze the user's interests, preferences, and past visit history based on this information and generate individually suited tourist routes and missions using a generative AI model, a means for the data center to transmit the generated tourist routes and missions to the terminal, and a means for the terminal to display personalized guide characters and messages using augmented reality technology. This allows users to not only experience optimal tourist routes and missions based on their preferences and past visit history, but also to receive services tailored to their emotional state in real time.

[0359] A "user" is an individual who uses this system to visit a tourist spot.

[0360] A "mobile communication terminal" is a device such as a smartphone or tablet that is carried by a user.

[0361] "Standing picture panels" are panels containing QR codes and AR markers that are installed at tourist spots.

[0362] "Augmented reality technology" is a technology that displays virtual information overlaid on real-world images.

[0363] A "data center" is a collection of servers that receives information sent by users and performs analysis and generation processing.

[0364] A "QR code" is a two-dimensional barcode that stores information and can be read by scanning it.

[0365] "Location information" is geographical data that indicates the user's current location.

[0366] A "generative AI model" is an algorithm that analyzes a user's past data and trends to generate optimal sightseeing routes and missions.

[0367] A "tourist route" is a plan showing the route a user will take at a tourist spot.

[0368] A "mission" is a specific task or activity that a user must accomplish while sightseeing.

[0369] A "guide character" is a virtual guide displayed through augmented reality technology.

[0370] The "emotional state" is the psychological state recognized from the user's facial expression and voice.

[0371] "Push notification" is a notification function that delivers information to users in real time.

[0372] The present invention is a system that provides users with personalized sightseeing experiences. This system generates sightseeing routes and missions based on the user's preferences and past visit history, and provides them in real time. It also has the function of recognizing the user's emotional state and dynamically adjusting the sightseeing experience based on that. Specific embodiments of this system are described below.

[0373] Hardware and Software Configuration

[0374] In order to implement the present invention, the following hardware and software are used.

[0375] 1. User's device (mobile communication device)

[0376] Smartphones, tablets, etc.

[0377] 2. Server

[0378] It is equipped with a database of user information, a generative AI model, and an emotion engine.

[0379] 3. Generative AI Models

[0380] An algorithm that analyzes user preferences, past visit history, current trends, etc. to generate optimal tourist routes and missions.

[0381] 4. Augmented Reality Technology (AR Technology)

[0382] Used on the user's device, it displays virtual guide characters and messages.

[0383] 5. Data Center

[0384] It functions as a cloud server or on-premise server and processes user location information, QR code information and emotion data.

[0385] Example of system operation

[0386] Example 1: Family sightseeing

[0387] A user (family) visits a tourist spot and scans a portrait panel.

[0388] The device sends information to a data center, which then analyzes the family's past history and preferences.

[0389] The generative AI model suggests "experiential activities for parents and children to enjoy together" and "nature walks and picnic spots."

[0390] The device will notify the family of the mission it has received, and a guide character will provide guidance using AR technology.

[0391] The emotion engine recognizes a child's state of excitement and dynamically suggests relaxing spots.

[0392] Example 2: Solo tourism

[0393] The user (traveling alone) visits a tourist spot and scans the portrait panel.

[0394] The device sends the information to a data center, which takes into account past visits and interests.

[0395] The generative AI model suggests "tours of historical sites" and "hidden cafes."

[0396] The terminal notifies the received mission and moves along the route selected by the user.

[0397] When you arrive at a specific spot, a guide character will guide you to your next destination.

[0398] The emotion engine recognizes the user's level of satisfaction from their facial expressions and suggests more stimulating spots.

[0399] Prompt Sentence Examples

[0400] "Please explain in detail the process of scanning the portrait panels, sending the obtained user information to the data center, and having the AI ​​model generate the optimal tourist route."

[0401] "Please explain a specific scenario where a family visits a tourist destination and an emotion engine is used to tailor their experience in real time."

[0402] "Please explain how a generative AI model can provide a personalized sightseeing route for a solo traveler."

[0403] The embodiments of the present invention allow users to have new sightseeing experiences and encourage them to revisit tourist destinations. Furthermore, the introduction of an emotion engine allows for experiences based on the user's emotions, enabling more personalized services.

[0404] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0405] Step 1:

[0406] The user scans a portrait panel at a tourist spot using their smartphone camera.

[0407] Specific operation: The user holds the device's camera over the QR code or AR marker and taps "Scan" in the dedicated app.

[0408] Input: QR code and AR marker information on the character portrait panel displayed on the user's smartphone.

[0409] Output: QR code information and location information are obtained.

[0410] Step 2:

[0411] The device sends the acquired QR code information and location information to the data center.

[0412] Specific operation: The device uploads the QR code information and GPS location information to the server via a dedicated app.

[0413] Input: QR code information and GPS location information.

[0414] Output: QR code information and location information are sent to the data center.

[0415] Step 3:

[0416] Based on the information received by the server, the user's preferences and past visit history are retrieved from the database and analyzed using a generative AI model.

[0417] Specific operation: The server queries the database to obtain user information, then inputs that data into the generative AI model to obtain analysis results.

[0418] Input: QR code information, location information, user preference data, past visit history data.

[0419] Output: Suggestions for optimal sightseeing routes and missions for the user.

[0420] Step 4:

[0421] The server formalizes the tourist routes and missions generated and sends them to the device via a dedicated app.

[0422] Specific operation: The server converts the tourist route and missions into a format such as JSON and sends them to the terminal.

[0423] Input: The output of a generative AI model.

[0424] Output: Tourist route and mission data is sent to the terminal.

[0425] Step 5:

[0426] The device notifies the user of the tourist route and missions received from the server, and displays guide characters and messages using AR technology.

[0427] Specific operation: The device will display "New tourist route has arrived" in the notification bar and switch to AR mode to display the virtual character.

[0428] Input: Received tourist route and mission data.

[0429] Output: Notifications and AR guide information.

[0430] Step 6:

[0431] The device periodically obtains the user's current location and sends it to the data center.

[0432] Specific operation: Obtain location information using the device's GPS function and periodically upload it to the server.

[0433] Input: Current location.

[0434] Output: The location information is sent to the server.

[0435] Step 7:

[0436] The device analyzes the user's facial expressions and voice using an emotion engine to recognize their emotional state in real time.

[0437] Specific operation: The device's camera and microphone capture the user's facial expressions and voice, and the emotion engine provides the analysis results.

[0438] Input: User's facial and voice data.

[0439] Output: Recognized emotional state data.

[0440] Step 8:

[0441] The server dynamically adjusts sightseeing routes and missions based on emotion data and current location information, suggesting new activities and next spots.

[0442] Specific operation: The server analyzes the emotion data and location information, generates a new sightseeing route, and sends it to the device.

[0443] Input: Emotional state data and location information.

[0444] Output: Dynamically adjusted tourist routes and new mission data.

[0445] Step 9:

[0446] The terminal notifies the user of the latest instructions and information received from the server.

[0447] Specific operation: The device will display "Please move to the next spot" in the notification bar and provide information about new tourist spots.

[0448] Input: The latest instructions and information from the server.

[0449] Output: Notify the user and display tourist spot information.

[0450] (Application example 2)

[0451] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0452] Conventional tourist information systems have struggled to provide personalized sightseeing routes and missions that fully reflect a user's individual preferences and interests. They also struggled to recognize a user's emotional state in real time and dynamically adjust the sightseeing experience based on that information. This resulted in users being unable to obtain a fully satisfying sightseeing experience, and they were unable to encourage repeat visits to tourist destinations.

[0453] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to scan a portrait panel installed at a tourist attraction with an electronics device; a means for the electronics device to transmit the user's location information and QR code information to a data center; a means for the data center to analyze the user's interests, preferences, and past visit history based on this information and generate individually tailored tourist routes and missions using a generative AI model; a means for the data center to transmit the generated tourist routes and missions to the electronics device; a means for the electronics device to display personalized virtual guide characters and messages using augmented reality technology; a means for the electronics device to analyze the user's facial expressions and voice and recognize the user's emotional state in real time; and a means for the data center to dynamically adjust the tourist routes and missions based on the emotion analysis data. This makes it possible to provide a more satisfying tourist experience tailored to the user's individual preferences and emotional state.

[0454] "User" refers to a person who uses an electronic device to scan the portrait panels installed at tourist spots and experience the tourist routes and missions provided.

[0455] A "tourist destination" refers to an area or place that contains attractions and sights that are of interest to visitors.

[0456] "Standing picture panels" are displays installed at tourist spots that users can scan with their electronic device's camera to read QR codes and AR markers.

[0457] "Electronic devices" refers to electronic devices that users carry and use, such as smartphones and smart glasses.

[0458] "Location information" is data indicating the user's current location within a tourist attraction.

[0459] A "QR code" is a two-dimensional barcode that can be read by an electronic device to obtain the embedded information.

[0460] "Data Center" refers to a service provider that collects and analyzes data and information sent by users and generates personalized tourist routes and missions using generative AI models.

[0461] "Interests and preferences" refers to the interests and preferences that a user has shown in the past.

[0462] "Past visit history" refers to a record of places a user has visited and activities they have participated in in the past.

[0463] A "generative AI model" refers to an artificial intelligence algorithm that generates optimal tourist routes and missions based on a user's preferences and past visit history.

[0464] A "tourist route" refers to the optimal route or directions that a user should follow when sightseeing.

[0465] "Mission" refers to a specific activity or experience offered to a user.

[0466] "Augmented reality technology" is a technology that displays virtual information overlaid on the real world, allowing users to experience it through electronic devices.

[0467] "Virtual guide character" refers to a virtual character that uses augmented reality technology to guide visitors through sightseeing routes and missions.

[0468] "Message" refers to textual or visual information displayed to a user through an electronic device.

[0469] "Facial expression" refers to the expression of emotion shown by the movement of the user's facial muscles.

[0470] "Voice" refers to the tone and manner of speaking of the user.

[0471] "Emotional state" refers to the user's current state of mind and is recognized in real time using an emotion engine.

[0472] "Dynamic adjustment" refers to adapting sightseeing routes and missions on the fly based on the user's real-time emotional state and progress.

[0473] A system embodying this invention provides a personalized sightseeing experience to users using an electronic device when visiting a tourist attraction. The system begins when the user scans a portrait panel installed at the tourist attraction with the electronic device. The electronic device then transmits the user's location information and QR code information to a data center.

[0474] The data center analyzes the user's interests, preferences, and past visit history based on the received information, and uses a generative AI model to generate individually tailored sightseeing routes and missions, which are then sent from the data center to the electronics device.

[0475] The electronics device uses augmented reality technology to display personalized virtual guide characters and messages to the user. As the user moves along the sightseeing route, the electronics device analyzes the user's facial expressions and voice to recognize their current emotional state in real time. The analysis results of the emotional state are sent to a data center, which then dynamically adjusts the sightseeing route and missions based on the emotion analysis data.

[0476] The specific hardware and software used include smartphones and smart glasses as electronic devices, equipped with cameras and AR functions. For emotion analysis, OpenCV and Keras are used, for example. The data center is based on cloud services and servers, and databases and generative AI models are used to store and analyze user data.

[0477] For example, when a user scans a portrait panel installed at a tourist spot with their smartphone and the information is sent to the data center, the data center uses a generative AI model to generate a sightseeing route that matches the user's preferences. For example, for a user with their family, it can suggest "hands-on activities to enjoy together as a family" and "nature walks and picnic spots." It can also dynamically suggest relaxing spots if it detects excitement in the user's facial expression.

[0478] An example prompt is:

[0479] User's past purchase history:

[0480] Product ID: 123, Category: Clothing, Purchase Date: 2023-09-01

[0481] Product ID: 456, Category: Gadgets, Purchase Date: 2023-09-05

[0482] User's current emotional state: Positive

[0483] Based on this information, generate three recommended items for this user.

[0484] In this way, the present invention can provide users with a personalized sightseeing experience in real time, thereby improving user satisfaction and encouraging repeat visits to tourist destinations.

[0485] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0486] Step 1:

[0487] A user scans a standing picture panel installed at a tourist spot with an electronic device. The input is the QR code read by the user's electronic device and location information, and the output is a data packet containing this information. The user scans the standing picture panel with a camera and obtains their current location information along with the QR code.

[0488] Step 2:

[0489] The electronics device transmits the QR code information and location information to the data center. The input is a data packet of the QR code and location information read by the electronics device, and the output is the information transmitted to the data center. The terminal transmits the QR code and location information to the data center via the Internet.

[0490] Step 3:

[0491] The data center analyzes the user's interests, preferences, and past visit history based on the information received. The input is the user's past behavioral data stored along with QR code information and location information, and the output is analyzed user profile data. The server extracts past user data from the database and uses a generative AI model to identify the user's interests and preferences.

[0492] Step 4:

[0493] The data center uses a generative AI model to generate individually tailored sightseeing routes and missions. The input is user profile data and information on current tourist destinations, and the output is personalized sightseeing routes and missions. The generative AI model generates optimal sightseeing routes and missions based on the user's preferences and past history.

[0494] Step 5:

[0495] The data center sends the generated sightseeing routes and missions to the electronics device. The input is the personalized sightseeing route and mission data, and the output is the sightseeing route and mission data sent to the electronics device. The server formats the sightseeing route and mission and sends it to the electronics device.

[0496] Step 6:

[0497] The electronics device uses augmented reality technology to display personalized virtual guide characters and messages. The input is tourist route and mission data received from a data center, and the output is the guide character and message displayed on the user's screen. The device uses its camera and AR function to overlay the guide character and message on the real space.

[0498] Step 7:

[0499] An electronic device analyzes a user's facial expressions and voice to recognize their emotional state in real time. The input is the user's facial image and voice data, and the output is analyzed emotional state data. The device uses a camera and microphone to capture the user's facial expressions and voice, which are then analyzed by an emotion engine.

[0500] Step 8:

[0501] The electronic device transmits emotional state data to the data center, the input is the emotional state data, and the output is the emotional state data transmitted to the data center. The terminal transmits the emotional state to the data center in real time.

[0502] Step 9:

[0503] The data center dynamically adjusts tourist routes and missions based on emotional state data. The input is emotional state data transmitted in real time, and the output is adjusted tourist route and mission data. The server analyzes the emotional data and optimizes tourist routes and missions according to the situation.

[0504] Step 10:

[0505] The data center sends the adjusted sightseeing route and mission data to the electronics device. The input is the adjusted sightseeing route and mission data, and the output is the data sent to the electronics device. The server sends the new plan to the terminal and notifies the user.

[0506] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0507] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0508] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0509] [Second embodiment]

[0510] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0511] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0512] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0513] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0514] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0515] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0516] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0517] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0518] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0519] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0520] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0521] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0522] This invention is a system that utilizes generative AI models and AR technology to provide users with individually tailored sightseeing tours. The system generates personalized sightseeing routes and missions based on the user's preferences, interests, and past visit history, and delivers them in real time.

[0523] System configuration

[0524] The system mainly consists of the following components:

[0525] 1. User's device (smartphone, etc.)

[0526] 2. Server

[0527] 3. Generative AI Models

[0528] 4. Character portrait panels as AR markers

[0529] Explanation of program processing

[0530] 1. Collecting User Information

[0531] The user finds a portrait panel installed at a tourist spot.

[0532] The user starts up the device (smartphone) and opens the dedicated app.

[0533] The device's camera will activate and scan the QR code or AR marker on the character portrait panel.

[0534] The device uses location information services to obtain the current location.

[0535] The device sends the acquired information (QR code information, current location) to the server.

[0536] 2. Analyzing data and applying generative AI models

[0537] The server compares the user's information with a database to obtain the user's past visit history and preference data.

[0538] The server uses a generative AI model to analyze this data and generate sightseeing routes and missions based on the user's interests.

[0539] Generative AI models take into account historical data and current trends to create optimal plans.

[0540] The server sends the generated sightseeing routes and missions to the device.

[0541] 3. Delivering personalized AR content

[0542] The device analyzes the tourist routes and missions received and displays notifications to the user.

[0543] The user travels along a route selected by the user and arrives at the designated tourist spot.

[0544] The device uses AR technology to display personalized guide characters and messages from a portrait panel.

[0545] Example: When you arrive at a specific spot, a guide character will say, "This is your next destination."

[0546] 4. Real-time feedback

[0547] The device periodically acquires its current location and transmits it to the server.

[0548] The server analyzes the user's progress in real time and dynamically generates next steps and recommended tourist spots.

[0549] The server sends the latest instructions to the terminal and notifies the user.

[0550] Example: Send a notification that says, "If you stop by a specific cafe on your way to the next location, a hidden mission will occur."

[0551] Specific examples

[0552] Example 1: Family sightseeing

[0553] A user (family) visits a tourist spot and scans a portrait panel.

[0554] The device sends information to a server, which then analyzes the family's past history and preferences.

[0555] The generative AI model suggests "experiential activities for parents and children to enjoy together" and "nature walks and picnic spots."

[0556] The device will notify the family of the mission it has received, and a guide character will provide guidance using AR technology.

[0557] Example 2: Solo tourism

[0558] The user (traveling alone) visits a tourist spot and scans the portrait panel.

[0559] The device sends the information to the server, which takes into account past visits and interests.

[0560] The generative AI model suggests "tours of historical sites" and "hidden cafes."

[0561] The terminal notifies the received mission and moves along the route selected by the user.

[0562] When you arrive at a specific spot, a guide character will guide you to your next destination.

[0563] The above is a specific description of the embodiment of the present invention. This system allows users to have new sightseeing experiences and encourages them to revisit tourist spots.

[0564] The processing flow will be explained below.

[0565] Step 1:

[0566] The user discovers a standing picture panel installed at a tourist spot.

[0567] The user launches the dedicated app on their device and points the camera at the character portrait panel.

[0568] Step 2:

[0569] The device uses the camera to scan the QR code or AR marker on the character portrait panel.

[0570] The device acquires the location information and sends it to the server along with the QR code information.

[0571] Step 3:

[0572] Based on the user information received by the server, the user's past visit history and preference data are obtained from the database.

[0573] The server analyzes this information and inputs it into the generative AI model.

[0574] Step 4:

[0575] The generative AI model takes into account the analysis results and current trends to generate optimal sightseeing routes and missions for users.

[0576] The server formats the generated tourist routes and missions and sends them to the terminal.

[0577] Step 5:

[0578] The device displays the sightseeing route and missions received from the server,

[0579] Notify the user.

[0580] The user confirms the notification and starts the tour.

[0581] Step 6:

[0582] When the device reaches a designated tourist spot, AR technology is used to display a guide character or message from a portrait panel.

[0583] A guide character will guide you to your next steps and tourist spots.

[0584] Step 7:

[0585] The terminal periodically acquires the user's current location and transmits it to the server.

[0586] The server analyzes your progress in real time and dynamically generates new attractions and missions.

[0587] Step 8:

[0588] The server sends the latest instructions and information to the terminal and notifies the user.

[0589] Example: New missions and recommended spots will be announced via push notifications.

[0590] Step 9:

[0591] The user checks the notification from the device and moves to the next spot.

[0592] The device continuously captures the user's location and progress and sends feedback to the server.

[0593] Step 10:

[0594] The server dynamically updates next steps and new tourist spots based on feedback.

[0595] The above is the specific program processing flow of the system. Through these processing steps, users can have a sightseeing experience that is tailored to their individual needs.

[0596] Example 1

[0597] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0598] Conventional tourism systems provide uniform tourist information, making it difficult to provide a personalized experience tailored to each user's individual preferences and interests. Furthermore, they lack the functionality to track a user's progress in real time and dynamically suggest the next tourist spot or mission, forcing users to follow a predetermined route. Furthermore, they lacked a mechanism to provide new experiences when a user arrives at a specific tourist spot. These issues need to be resolved.

[0599] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0600] In this invention, the server includes: a means for a user to read a display panel installed at a tourist attraction with an information terminal; a means for the terminal to transmit the user's location information and identification information to the server; a means for the server to analyze the user's preferences and past history based on the received information and generate personalized tourist routes and missions using a generative AI model; a means for the server to transmit the generated tourist routes and missions to the terminal; a means for the terminal to display personalized guide characters and messages using augmented reality technology; a means for the terminal to continuously transmit the user's current location to the server, and the server to dynamically analyze the user's progress and generate the next tourist spot and mission; and a means for the generative AI model to generate tourist routes and missions taking past user data and current trends into consideration and to guide the user to the next step and recommended tourist spots via push notifications. This makes it possible to provide a personalized tourist experience for each user, and to dynamically suggest the next step by analyzing the user's progress in real time.

[0601] "User" refers to an individual who uses this system to obtain information at a tourist spot and enjoy sightseeing.

[0602] "Display panel" refers to a panel installed at a tourist spot that contains a QR code or AR marker that users can use to obtain information.

[0603] "Information terminal" refers to an electronic device such as a smartphone or tablet that a user owns.

[0604] "Location Information" refers to information about a user's current location obtained using GPS or other means.

[0605] "Identification information" refers to unique information obtained by scanning the QR code or AR marker printed on the display panel.

[0606] "Server" refers to a computer system for receiving and analyzing data sent by users.

[0607] A "generative AI model" refers to an artificial intelligence system that analyzes user preferences, past history, current trends, etc. to generate tourist routes and missions.

[0608] "Tourist route" refers to a specific route suggested by the server to enable the user to efficiently tour tourist spots.

[0609] A "mission" refers to a specific task or activity that a user must accomplish at a tourist destination.

[0610] "Augmented reality technology" refers to a technology that overlays digital information on a device that displays the real world.

[0611] "Guide character" refers to a virtual character that provides sightseeing guidance to users using augmented reality technology.

[0612] "Message" refers to notifications and guidance information displayed to the user by the guide character or the system.

[0613] "Push notification" refers to a message sent in real time from a server to a user's information terminal.

[0614] "Trends" refers to information about current trends and popular spots.

[0615] "Continuous" means that the information is updated at regular intervals.

[0616] "Dynamic" refers to the way something changes in real time depending on the situation.

[0617] The present invention is a system that provides users with individually tailored tourist experiences at tourist destinations. This system uses generative AI models and augmented reality technology to generate personalized tourist routes and missions based on the user's preferences and past visit history, and provides them in real time. Specific embodiments of this system are described below.

[0618] 1. Collecting User Information

[0619] Users first find a display panel installed at a tourist spot and begin gathering information by scanning it with an information device (e.g., a smartphone). The user then launches a dedicated app on their smartphone and uses the camera to scan the QR code or AR marker on the display panel. The device then uses GPS to obtain the user's current location and sends it to the server along with the QR code information. The location information service uses the Google Maps API.

[0620] 2. Analyzing data and applying generative AI models

[0621] The server compares the user information sent from the device with a database (e.g., MySQL, MongoDB) to obtain the user's past visit history and preference data. The server then uses a generative AI model (e.g., GPT-4) to analyze this data and generate sightseeing routes and missions based on the user's interests. Examples of prompts are as follows:

[0622] "Spots visited in the past: Mount Takao, Interests: Historical landmarks. Based on this, please suggest the next tourist spot and mission."

[0623] The generative AI model takes into account past user data and current trends to create an optimal plan, and the generated sightseeing route and missions are sent from the server to the device.

[0624] 3. Delivering personalized AR content

[0625] The device analyzes the tourist routes and missions received from the server and displays them to the user as push notifications. The user then moves toward the designated tourist spots based on the notifications. Upon arriving at each spot, the device uses augmented reality technology (e.g., ARKit, ARCore) to display a personalized guide character and message on the display panel. For example, when arriving at a specific spot, a guide character will appear saying, "This is your next destination."

[0626] 4. Real-time feedback

[0627] As the user travels along a sightseeing route, the device periodically acquires its current location and sends it to the server. The server analyzes the user's progress in real time and dynamically generates next steps and recommended sightseeing spots. These results are then sent back to the device and displayed as notifications to the user. For example, a notification might say, "If you stop by a specific cafe on your way to the next spot, a hidden mission will occur."

[0628] Specific examples

[0629] For example, suppose a family visits a tourist spot and scans the display panel. The device sends the information to the server, which analyzes the family's past history and preferences. The generative AI model suggests "hands-on activities for parents and children" and "nature walks and picnic spots." The device notifies the family of the mission received, and a guide character uses augmented reality technology to provide guidance.

[0630] For solo travelers, when they visit a tourist spot, they scan the display panel, and the device sends the information to the server. The server takes into account their past visit history and interests, and the generative AI model suggests "tours of historical landmarks" and "hidden cafes." The device notifies the user of the received mission, and as they travel along the route selected by the user, a guide character will guide them to their next destination when they arrive at a specific spot.

[0631] This system allows users to have new sightseeing experiences and encourages them to revisit tourist destinations.

[0632] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0633] The flow of this system's program processing

[0634] Step 1: The user finds the display panel

[0635] Input: Display panels installed in various locations in tourist spots

[0636] Action: The user finds the display panel.

[0637] Output: Your smartphone is ready to get the information from the display panel.

[0638] Specific actions: The user walks around a tourist spot, searches for a display panel, and when they find one, takes out their smartphone.

[0639] Step 2: The user starts the device and opens the dedicated app.

[0640] Input: A smartphone owned by the user

[0641] Process: The user takes out their smartphone and launches the dedicated app.

[0642] Output: The dedicated app will launch and you will be ready to scan.

[0643] Specific operation: The user touches the smartphone screen, selects and opens the dedicated app.

[0644] Step 3: Scan the QR code or AR marker on the character panel with your device

[0645] Input: QR code or AR marker placed on the display panel

[0646] Processing: Activate the smartphone camera and scan the QR code or AR marker.

[0647] Output: QR code and AR marker information is obtained

[0648] Specific operation: The user points the smartphone camera at the display panel and scans the QR code or AR marker.

[0649] Step 4: The device acquires location information and sends it to the server

[0650] Input: Current location information of the device using GPS, as well as information on QR codes and AR markers

[0651] Processing: Send location information and scanned information to the server

[0652] Output: Location information and QR code information are sent to the server

[0653] Specific operation: The device uses location services (e.g., Google Maps API) to obtain the current location and sends the obtained information to the server via HTTPS protocol.

[0654] Step 5: The server checks the user information against the database

[0655] Input: Location information and QR code information sent from the device

[0656] Processing: The server checks the database to obtain the user's past visit history and preference data.

[0657] Output: User's past visit history and preference data

[0658] Specific operation: The server accesses a database (e.g., MySQL, MongoDB) and retrieves the information to be matched.

[0659] Step 6: The server generates tourist routes and missions using the generative AI model

[0660] Input: User's past visit history, preference data, current location information

[0661] Processing: Input prompts into the generative AI model to generate personalized sightseeing routes and missions.

[0662] Output: personalized tourist routes and missions

[0663] Specific operation: An example of a prompt sentence: "Spots visited in the past: Mt. Takao, Interests: Historical buildings. Based on this, please suggest the next tourist spot and mission." is input into a generative AI model (e.g., GPT-4) and the generated results are obtained.

[0664] Step 7: The server sends the generated results to the device.

[0665] Input: Tourist routes and missions generated by generative AI models

[0666] Process: Send the generated results to the terminal

[0667] Output: Tourist route and mission data sent to the device

[0668] Specific operation: The server sends the generated results to the terminal using the HTTPS protocol.

[0669] Step 8: The device analyzes the sightseeing route and mission and displays a notification

[0670] Input: Tourist route and mission data sent from the server

[0671] Processing: The device analyzes the data and displays a notification to the user

[0672] Output: The notification displayed to the user

[0673] Specific operation: The device analyzes the sightseeing route and mission data and displays a notification saying, "Your next destination is XX Park."

[0674] Step 9: User follows notification

[0675] Input: Device notifications

[0676] Action: The user follows the notification and moves to the designated tourist spot.

[0677] Output: The user arrives at the specified location.

[0678] Specific behavior: The user checks the smartphone screen and moves according to the displayed route.

[0679] Step 10: The device uses AR technology to display personalized content.

[0680] Input: User location information, AR content data

[0681] Processing: The device displays a guide character or message using augmented reality technology.

[0682] Output: Personalized AR content displayed to the user

[0683] Specific operation: When the user arrives at the designated spot, a guide character will be displayed using the device's camera, saying, "This is your next destination."

[0684] Step 11: The device periodically acquires location information and sends it to the server.

[0685] Input: current location of the user

[0686] Processing: The device periodically acquires location information using GPS and sends it to the server.

[0687] Output: Location information sent to the server

[0688] Specific operation: The device acquires its current location at regular intervals and sends that information to the server.

[0689] Step 12: The server analyzes the progress and generates the next step

[0690] Input: current location of the user

[0691] Processing: The server analyzes the progress and generates the next attractions and missions.

[0692] Output: Data on the next tourist spot or mission

[0693] Specific operation: The server uses the generative AI model to generate the next route and mission based on the user's current location.

[0694] Step 13: The server sends the latest instructions to the terminal and notifies the user.

[0695] Input: Data for the next tourist spot or mission

[0696] Processing: The server sends the latest instructions to the terminal and notifies the user.

[0697] Output: Notification displayed on the user's device

[0698] Specific operation: The server generates the next step and sends it to the device, which then displays a notification saying, "If you stop by a specific cafe on your way to the next spot, a hidden mission will occur."

[0699] (Application example 1)

[0700] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0701] There is a demand for tourist destinations and brick-and-mortar stores to provide users with personalized information based on their preferences, interests, and past visit history, thereby providing a more fulfilling experience. However, it is difficult for users to efficiently and intuitively obtain information and find optimal routes and recommended products without receiving a large amount of information all at once. Current systems do not adequately provide individually tailored information, making it difficult for users to obtain information that perfectly matches their interests. Therefore, the objective of this invention is to provide a system that can provide users with individually tailored sightseeing routes and shopping routes in real time.

[0702] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0703] In this invention, the server includes means for analyzing the user's interests, preferences, and past visit history based on the user's location information and QR code information, means for generating individually suited sightseeing routes and missions using a generative AI model, and means for displaying AR content based on AR markers detected by the device's camera, thereby enabling the provision of personalized sightseeing routes, shopping routes, and recommended products to the user.

[0704] "User" refers to an individual who uses this system to receive information at tourist spots and brick-and-mortar stores.

[0705] "Terminal" refers to a device carried by a user, such as a smartphone, smart glasses, or head-mounted display, that acquires QR code information and location information and displays AR content.

[0706] "Server" refers to the central processing unit that processes user information and generates sightseeing routes, missions, and shopping routes using generative AI models.

[0707] "Generative AI model" refers to an artificial intelligence model that analyzes a user's interests and preferences, past visit history, and current trends to generate individually appropriate tourist routes and missions.

[0708] "AR technology" refers to augmented reality technology, which displays digital content overlaid on real-world information.

[0709] A "tourist route" refers to the optimal order or route for a user to visit tourist spots.

[0710] A "mission" refers to a specific action or task presented to a user.

[0711] "QR code" refers to a two-dimensional barcode consisting of a rectangular pattern of dots, and is a format for encoding information.

[0712] "AR marker" refers to a printed or digital image that serves as a reference for displaying content using augmented reality technology.

[0713] "Location Information" means a user's current geographic location, which may be obtained using GPS or other location-determining technology.

[0714] "Progress" refers to information indicating how far a user has progressed along a given sightseeing or shopping route.

[0715] A "shopping route" refers to the optimal order or route for a user to efficiently shop in a physical store.

[0716] "Benefit information" refers to information such as discounts, campaigns, and bonuses offered to users.

[0717] This invention is a system for providing users with personalized sightseeing and shopping routes at tourist spots and brick-and-mortar stores. The system utilizes generative AI models and AR technology to provide individually tailored information based on the user's preferences, interests, and past visit history.

[0718] System configuration

[0719] The system mainly consists of the following components:

[0720] 1. User's device (smartphone, smart glasses, head-mounted display, etc.)

[0721] 2. Server

[0722] 3. Generative AI Models

[0723] 4. AR markers (including QR codes)

[0724] User's device

[0725] The user's device may be a smartphone, smart glasses, or a head-mounted display. These devices have a camera function and can scan AR markers and QR codes installed at tourist spots and in brick-and-mortar stores. When the device scans these markers, the acquired data along with location information is sent to a server. Furthermore, the device uses AR technology to display personalized guide characters and messages.

[0726] server

[0727] The server has the ability to analyze the user's interests, preferences, and past visit history based on the user's location information and QR code information. A generative AI model running on the server takes this data and current trends into account to generate optimal sightseeing and shopping routes. The generated routes and missions are then sent to the user's device in real time.

[0728] Generative AI Models

[0729] A generative AI model is an artificial intelligence model that analyzes a user's interests, preferences, and past visit history to personalize sightseeing or shopping routes. This model uses user data and current trends to generate an optimal plan and sends it to the user's device via a server. An example of a specific prompt is, "User 'User123' has the following past purchase history: chairs, tables, and shelves. Taking into account the latest trends, please recommend new furniture that may be of interest to them. Also, please generate the optimal shopping route within the store."

[0730] AR technology

[0731] AR technology is a technology that overlays digital content onto the real world through the user's device. When a user scans an AR marker in a tourist spot or physical store, personalized guide characters, product information, and messages are displayed on the device screen. This plays an important role in improving the user experience.

[0732] Specific examples

[0733] Example 1: Family sightseeing

[0734] A user (family) visits a tourist spot and scans a character panel. The device sends information to the server, which analyzes the family's past history and preferences. The generative AI model suggests "hands-on activities for parents and children to enjoy together" and "nature walks and picnic spots." The device notifies the family of the mission it received, and a guide character uses AR technology to provide guidance.

[0735] Example 2: Solo tourism

[0736] A user (solo traveler) visits a tourist spot and scans a character panel. The device sends the information to the server, which takes into account past visit history and interests. The generative AI model suggests "tours of historical landmarks" and "hidden cafes." As the user travels along the route selected, a guide character will guide them to their next destination when they arrive at a specific spot.

[0737] The components and operation procedures described above allow users to have a more fulfilling experience at tourist spots and brick-and-mortar stores.

[0738] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0739] Step 1:

[0740] The user uses a device to scan AR markers or QR codes installed at tourist spots or brick-and-mortar stores. The device's camera acquires the QR code or AR marker information, and collects location information along with this information. The input is "QR code information" and "location information." This data is then combined into a single communication packet for data processing. The output is "data that is a packet of the user's QR code information and location information."

[0741] Step 2:

[0742] The QR code information and location information collected by the device are sent to the server. The input is "data that is a packet of the user's QR code information and location information." The processing involves sending the data using an HTTP request. The output is "user data sent to the server."

[0743] Step 3:

[0744] The server analyzes the user's QR code information and location information received. The input is "user data sent to the server." Data calculations use database matching and a data analysis engine to obtain the user's past visit history and preference data. The output is "the analysis results of the user information."

[0745] Step 4:

[0746] The server uses the generative AI model to generate individually tailored sightseeing and shopping routes based on the user's past visit history and preference data. The input is the "analysis results of user information." The data is passed to the generative AI model for data processing and calculation, generating multiple personalized options. As a concrete example, the prompt statement used is "Please recommend tourist spots of interest to user 'User123' based on his past visit history." The output is the "generated sightseeing and shopping routes."

[0747] Step 5:

[0748] The server sends the generated sightseeing route or shopping route to the user's device. The input is the "generated sightseeing route or shopping route." The processing involves sending the data using an HTTP response. The output is the "data sent to the user's device."

[0749] Step 6:

[0750] The device analyzes the tourist route and shopping route data received and displays a notification to the user. The input is "data sent to the user's device." Data processing involves reformatting the data into a notification format. The output is "a notification message displayed to the user."

[0751] Step 7:

[0752] When the user arrives at the designated tourist spot or inside the store, the device's camera scans the AR marker again and displays the AR content. The input is "AR marker information." The data is processed by selecting and displaying the appropriate AR content based on the marker information. The output is "AR content displayed on the device."

[0753] Step 8:

[0754] The device periodically acquires the current location and sends it to the server, allowing the user's progress to be analyzed in real time. The input is "current location data." The data is analyzed in real time as part of the process, and information about the next tourist spot and special offers is dynamically generated. The output is "next step and special offer information tailored to the user's progress."

[0755] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0756] This invention is a system that combines generative AI models, AR technology, and an emotion engine to provide users with individually tailored sightseeing tours. The system generates personalized sightseeing routes and missions based on the user's preferences and past visit history, and provides them in real time. It also has the ability to recognize the user's emotional state and dynamically adjust the sightseeing experience based on that.

[0757] System configuration

[0758] The system mainly consists of the following components:

[0759] 1. User's device (smartphone, etc.)

[0760] 2. Server

[0761] 3. Generative AI Models

[0762] 4. Character portrait panels as AR markers

[0763] 5. Emotion Engine

[0764] Explanation of program processing

[0765] 1. Collecting User Information

[0766] The user discovers a standing image panel installed at a tourist spot and launches a dedicated app on their device.

[0767] The device's camera scans the QR code or AR marker on the character portrait panel to obtain location information.

[0768] The device sends the QR code information and location information to the server.

[0769] 2. Analyzing data and applying generative AI models

[0770] The server compares the received user information with a database to obtain the user's past visit history and preference data.

[0771] The server uses a generative AI model to analyze this data and generate sightseeing routes and missions based on the user's interests.

[0772] Generative AI models take into account past data and current trends to create optimal plans.

[0773] The server formats the generated tourist routes and missions and sends them to the terminal.

[0774] 3. Delivering personalized AR content

[0775] The terminal displays the sightseeing route and mission received from the server and notifies the user.

[0776] The user confirms the notification and starts the tour.

[0777] When the device reaches a designated tourist spot, AR technology is used to display a guide character or message from a portrait panel.

[0778] 4. Applying the Emotion Engine

[0779] The device analyzes the user's facial expressions and voice using an emotion engine to recognize their emotional state in real time.

[0780] The device transmits the emotion data to the server.

[0781] The server dynamically adjusts sightseeing routes and missions based on emotion data, and suggests next spots and new activities.

[0782] 5. Real-time feedback

[0783] The device periodically acquires its current location and transmits it to the server.

[0784] The server analyzes the user's progress and emotional state in real time and dynamically generates next steps and recommended tourist spots.

[0785] The server sends the latest instructions and information to the terminal and notifies the user.

[0786] Specific examples

[0787] Example 1: Family sightseeing

[0788] A user (family) visits a tourist spot and scans a portrait panel.

[0789] The device sends information to a server, which then analyzes the family's past history and preferences.

[0790] The generative AI model suggests "experiential activities for parents and children to enjoy together" and "nature walks and picnic spots."

[0791] The device will notify the family of the mission it has received, and a guide character will provide guidance using AR technology.

[0792] The emotion engine recognizes a child's state of excitement and dynamically suggests relaxing spots.

[0793] Example 2: Solo tourism

[0794] The user (traveling alone) visits a tourist spot and scans the portrait panel.

[0795] The device sends the information to the server, which takes into account past visits and interests.

[0796] The generative AI model suggests "tours of historical sites" and "hidden cafes."

[0797] The terminal notifies the received mission and moves along the route selected by the user.

[0798] When you arrive at a specific spot, a guide character will guide you to your next destination.

[0799] The emotion engine recognizes the user's level of satisfaction from their facial expressions and suggests more stimulating spots.

[0800] This concludes the detailed description of the embodiment of the present invention. This system allows users to have new sightseeing experiences and encourages repeat visits to tourist spots. Furthermore, the introduction of an emotion engine makes it possible to provide experiences based on the user's emotions, realizing more personalized services.

[0801] The processing flow will be explained below.

[0802] Step 1:

[0803] The user discovers a standing picture panel installed at a tourist spot.

[0804] The user launches the dedicated app on their device (smartphone, etc.) and points the camera at the character portrait panel.

[0805] Step 2:

[0806] The device uses the camera to scan the QR code or AR marker on the character portrait panel.

[0807] The device acquires location information and sends the QR code information and location information to the server.

[0808] Step 3:

[0809] Based on the user information received by the server, the user's past visit history and preference data are obtained from the database.

[0810] The server analyzes this data and inputs it into a generative AI model.

[0811] Step 4:

[0812] The generative AI model takes into account the analysis results and current trends to generate optimal sightseeing routes and missions for users.

[0813] The server formats the generated tourist routes and missions and sends them to the terminal.

[0814] Step 5:

[0815] The terminal displays the sightseeing route and mission received from the server and notifies the user.

[0816] The user confirms the notification and starts the tour.

[0817] Step 6:

[0818] When the device reaches a designated tourist spot, AR technology is used to display a guide character or message from a portrait panel.

[0819] A guide character will guide you to your next steps and tourist spots.

[0820] Step 7:

[0821] The device analyzes the user's facial expressions and voice data using an emotion engine to recognize their emotional state in real time.

[0822] The device transmits the emotion data to the server.

[0823] Step 8:

[0824] The server dynamically adjusts sightseeing routes and missions based on emotion data, and suggests next spots and new activities.

[0825] Example: The server detects the user's fatigue and suggests a spot where they can relax.

[0826] Step 9:

[0827] The terminal periodically acquires the user's current location and transmits it to the server.

[0828] The server analyzes the user's progress and emotional state in real time and dynamically generates next steps and recommended tourist spots.

[0829] Step 10:

[0830] The server sends the latest instructions and information to the terminal and notifies the user.

[0831] Example: New missions and recommended spots will be announced via push notifications.

[0832] Step 11:

[0833] The user checks the notification from the device and moves to the next spot.

[0834] The device continuously captures the user's location and progress and sends feedback to the server.

[0835] Step 12:

[0836] The server dynamically updates next steps and new tourist spots based on feedback.

[0837] For example: Suggest a special activity at a cafe on the way to your next destination.

[0838] The above is the specific program processing flow of the system that combines the emotion engine. This system allows users to have a sightseeing experience that is tailored to their individual needs, and can encourage them to revisit tourist spots.

[0839] Example 2

[0840] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0841] In recent years, there has been a growing demand for personalized tourism experiences tailored to the preferences and interests of each individual tourist. However, conventional tourist information systems have not been able to adequately address the individual needs of tourists. As a result, it has been difficult to provide tourists with a more fulfilling tourism experience and maximize the appeal of tourist destinations. Furthermore, there has been no system that can recognize the emotional state of users in real time while sightseeing and dynamically adjust sightseeing routes and missions accordingly.

[0842] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0843] In this invention, the server includes a means for a user to scan a character image panel installed at a tourist spot with a mobile communication terminal, a means for the terminal to transmit the user's location information and QR code information to a data center, a means for the data center to analyze the user's interests, preferences, and past visit history based on this information and generate individually suited tourist routes and missions using a generative AI model, a means for the data center to transmit the generated tourist routes and missions to the terminal, and a means for the terminal to display personalized guide characters and messages using augmented reality technology. This allows users to not only experience optimal tourist routes and missions based on their preferences and past visit history, but also to receive services tailored to their emotional state in real time.

[0844] A "user" is an individual who uses this system to visit a tourist spot.

[0845] A "mobile communication terminal" is a device such as a smartphone or tablet that is carried by a user.

[0846] "Standing picture panels" are panels containing QR codes and AR markers that are installed at tourist spots.

[0847] "Augmented reality technology" is a technology that displays virtual information overlaid on real-world images.

[0848] A "data center" is a collection of servers that receives information sent by users and performs analysis and generation processing.

[0849] A "QR code" is a two-dimensional barcode that stores information and can be read by scanning it.

[0850] "Location information" is geographical data that indicates the user's current location.

[0851] A "generative AI model" is an algorithm that analyzes a user's past data and trends to generate optimal sightseeing routes and missions.

[0852] A "tourist route" is a plan showing the route a user will take at a tourist spot.

[0853] A "mission" is a specific task or activity that a user must accomplish while sightseeing.

[0854] A "guide character" is a virtual guide displayed through augmented reality technology.

[0855] The "emotional state" is the psychological state recognized from the user's facial expression and voice.

[0856] "Push notification" is a notification function that delivers information to users in real time.

[0857] The present invention is a system that provides users with personalized sightseeing experiences. This system generates sightseeing routes and missions based on the user's preferences and past visit history, and provides them in real time. It also has the function of recognizing the user's emotional state and dynamically adjusting the sightseeing experience based on that. Specific embodiments of this system are described below.

[0858] Hardware and Software Configuration

[0859] In order to implement the present invention, the following hardware and software are used.

[0860] 1. User's device (mobile communication device)

[0861] Smartphones, tablets, etc.

[0862] 2. Server

[0863] It is equipped with a database of user information, a generative AI model, and an emotion engine.

[0864] 3. Generative AI Models

[0865] An algorithm that analyzes user preferences, past visit history, current trends, etc. to generate optimal tourist routes and missions.

[0866] 4. Augmented Reality Technology (AR Technology)

[0867] Used on the user's device, it displays virtual guide characters and messages.

[0868] 5. Data Center

[0869] It functions as a cloud server or on-premise server and processes user location information, QR code information and emotion data.

[0870] Example of system operation

[0871] Example 1: Family sightseeing

[0872] A user (family) visits a tourist spot and scans a portrait panel.

[0873] The device sends information to a data center, which then analyzes the family's past history and preferences.

[0874] The generative AI model suggests "experiential activities for parents and children to enjoy together" and "nature walks and picnic spots."

[0875] The device will notify the family of the mission it has received, and a guide character will provide guidance using AR technology.

[0876] The emotion engine recognizes a child's state of excitement and dynamically suggests relaxing spots.

[0877] Example 2: Solo tourism

[0878] The user (traveling alone) visits a tourist spot and scans the portrait panel.

[0879] The device sends the information to a data center, which takes into account past visits and interests.

[0880] The generative AI model suggests "tours of historical sites" and "hidden cafes."

[0881] The terminal notifies the received mission and moves along the route selected by the user.

[0882] When you arrive at a specific spot, a guide character will guide you to your next destination.

[0883] The emotion engine recognizes the user's level of satisfaction from their facial expressions and suggests more stimulating spots.

[0884] Prompt Sentence Examples

[0885] "Please explain in detail the process of scanning the portrait panels, sending the obtained user information to the data center, and having the AI ​​model generate the optimal tourist route."

[0886] "Please explain a specific scenario where a family visits a tourist destination and an emotion engine is used to tailor their experience in real time."

[0887] "Please explain how a generative AI model can provide a personalized sightseeing route for a solo traveler."

[0888] The embodiments of the present invention allow users to have new sightseeing experiences and encourage them to revisit tourist destinations. Furthermore, the introduction of an emotion engine allows for experiences based on the user's emotions, enabling more personalized services.

[0889] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0890] Step 1:

[0891] The user scans a portrait panel at a tourist spot using their smartphone camera.

[0892] Specific operation: The user holds the device's camera over the QR code or AR marker and taps "Scan" in the dedicated app.

[0893] Input: QR code and AR marker information on the character portrait panel displayed on the user's smartphone.

[0894] Output: QR code information and location information are obtained.

[0895] Step 2:

[0896] The device sends the acquired QR code information and location information to the data center.

[0897] Specific operation: The device uploads the QR code information and GPS location information to the server via a dedicated app.

[0898] Input: QR code information and GPS location information.

[0899] Output: QR code information and location information are sent to the data center.

[0900] Step 3:

[0901] Based on the information received by the server, the user's preferences and past visit history are retrieved from the database and analyzed using a generative AI model.

[0902] Specific operation: The server queries the database to obtain user information, then inputs that data into the generative AI model to obtain analysis results.

[0903] Input: QR code information, location information, user preference data, past visit history data.

[0904] Output: Suggestions for optimal sightseeing routes and missions for the user.

[0905] Step 4:

[0906] The server formalizes the tourist routes and missions generated and sends them to the device via a dedicated app.

[0907] Specific operation: The server converts the tourist route and missions into a format such as JSON and sends them to the terminal.

[0908] Input: The output of a generative AI model.

[0909] Output: Tourist route and mission data is sent to the terminal.

[0910] Step 5:

[0911] The device notifies the user of the tourist route and missions received from the server, and displays guide characters and messages using AR technology.

[0912] Specific operation: The device will display "New tourist route has arrived" in the notification bar and switch to AR mode to display the virtual character.

[0913] Input: Received tourist route and mission data.

[0914] Output: Notifications and AR guide information.

[0915] Step 6:

[0916] The device periodically obtains the user's current location and sends it to the data center.

[0917] Specific operation: Obtain location information using the device's GPS function and periodically upload it to the server.

[0918] Input: Current location.

[0919] Output: The location information is sent to the server.

[0920] Step 7:

[0921] The device analyzes the user's facial expressions and voice using an emotion engine to recognize their emotional state in real time.

[0922] Specific operation: The device's camera and microphone capture the user's facial expressions and voice, and the emotion engine provides the analysis results.

[0923] Input: User's facial and voice data.

[0924] Output: Recognized emotional state data.

[0925] Step 8:

[0926] The server dynamically adjusts sightseeing routes and missions based on emotion data and current location information, suggesting new activities and next spots.

[0927] Specific operation: The server analyzes the emotion data and location information, generates a new sightseeing route, and sends it to the device.

[0928] Input: Emotional state data and location information.

[0929] Output: Dynamically adjusted tourist routes and new mission data.

[0930] Step 9:

[0931] The terminal notifies the user of the latest instructions and information received from the server.

[0932] Specific operation: The device will display "Please move to the next spot" in the notification bar and provide information about new tourist spots.

[0933] Input: The latest instructions and information from the server.

[0934] Output: Notify the user and display tourist spot information.

[0935] (Application example 2)

[0936] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0937] Conventional tourist information systems have struggled to provide personalized sightseeing routes and missions that fully reflect a user's individual preferences and interests. They also struggled to recognize a user's emotional state in real time and dynamically adjust the sightseeing experience based on that information. This resulted in users being unable to obtain a fully satisfying sightseeing experience, and they were unable to encourage repeat visits to tourist destinations.

[0938] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to scan a portrait panel installed at a tourist attraction with an electronics device; a means for the electronics device to transmit the user's location information and QR code information to a data center; a means for the data center to analyze the user's interests, preferences, and past visit history based on this information and generate individually tailored tourist routes and missions using a generative AI model; a means for the data center to transmit the generated tourist routes and missions to the electronics device; a means for the electronics device to display personalized virtual guide characters and messages using augmented reality technology; a means for the electronics device to analyze the user's facial expressions and voice and recognize the user's emotional state in real time; and a means for the data center to dynamically adjust the tourist routes and missions based on the emotion analysis data. This makes it possible to provide a more satisfying tourist experience tailored to the user's individual preferences and emotional state.

[0939] "User" refers to a person who uses an electronic device to scan the portrait panels installed at tourist spots and experience the tourist routes and missions provided.

[0940] A "tourist destination" refers to an area or place that contains attractions and sights that are of interest to visitors.

[0941] "Standing picture panels" are displays installed at tourist spots that users can scan with their electronic device's camera to read QR codes and AR markers.

[0942] "Electronic devices" refers to electronic devices that users carry and use, such as smartphones and smart glasses.

[0943] "Location information" is data indicating the user's current location within a tourist attraction.

[0944] A "QR code" is a two-dimensional barcode that can be read by an electronic device to obtain the embedded information.

[0945] "Data Center" refers to a service provider that collects and analyzes data and information sent by users and generates personalized tourist routes and missions using generative AI models.

[0946] "Interests and preferences" refers to the interests and preferences that a user has shown in the past.

[0947] "Past visit history" refers to a record of places a user has visited and activities they have participated in in the past.

[0948] A "generative AI model" refers to an artificial intelligence algorithm that generates optimal tourist routes and missions based on a user's preferences and past visit history.

[0949] A "tourist route" refers to the optimal route or directions that a user should follow when sightseeing.

[0950] "Mission" refers to a specific activity or experience offered to a user.

[0951] "Augmented reality technology" is a technology that displays virtual information overlaid on the real world, allowing users to experience it through electronic devices.

[0952] "Virtual guide character" refers to a virtual character that uses augmented reality technology to guide visitors through sightseeing routes and missions.

[0953] "Message" refers to textual or visual information displayed to a user through an electronic device.

[0954] "Facial expression" refers to the expression of emotion shown by the movement of the user's facial muscles.

[0955] "Voice" refers to the tone and manner of speaking of the user.

[0956] "Emotional state" refers to the user's current state of mind and is recognized in real time using an emotion engine.

[0957] "Dynamic adjustment" refers to adapting sightseeing routes and missions on the fly based on the user's real-time emotional state and progress.

[0958] A system embodying this invention provides a personalized sightseeing experience to users using an electronic device when visiting a tourist attraction. The system begins when the user scans a portrait panel installed at the tourist attraction with the electronic device. The electronic device then transmits the user's location information and QR code information to a data center.

[0959] The data center analyzes the user's interests, preferences, and past visit history based on the received information, and uses a generative AI model to generate individually tailored sightseeing routes and missions, which are then sent from the data center to the electronics device.

[0960] The electronics device uses augmented reality technology to display personalized virtual guide characters and messages to the user. As the user moves along the sightseeing route, the electronics device analyzes the user's facial expressions and voice to recognize their current emotional state in real time. The analysis results of the emotional state are sent to a data center, which then dynamically adjusts the sightseeing route and missions based on the emotion analysis data.

[0961] The specific hardware and software used include smartphones and smart glasses as electronic devices, equipped with cameras and AR functions. For emotion analysis, OpenCV and Keras are used, for example. The data center is based on cloud services and servers, and databases and generative AI models are used to store and analyze user data.

[0962] For example, when a user scans a portrait panel installed at a tourist spot with their smartphone and the information is sent to the data center, the data center uses a generative AI model to generate a sightseeing route that matches the user's preferences. For example, for a user with their family, it can suggest "hands-on activities to enjoy together as a family" and "nature walks and picnic spots." It can also dynamically suggest relaxing spots if it detects excitement in the user's facial expression.

[0963] An example prompt is:

[0964] User's past purchase history:

[0965] Product ID: 123, Category: Clothing, Purchase Date: 2023-09-01

[0966] Product ID: 456, Category: Gadgets, Purchase Date: 2023-09-05

[0967] User's current emotional state: Positive

[0968] Based on this information, generate three recommended items for this user.

[0969] In this way, the present invention can provide users with a personalized sightseeing experience in real time, thereby improving user satisfaction and encouraging repeat visits to tourist destinations.

[0970] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0971] Step 1:

[0972] A user scans a standing picture panel installed at a tourist spot with an electronic device. The input is the QR code read by the user's electronic device and location information, and the output is a data packet containing this information. The user scans the standing picture panel with a camera and obtains their current location information along with the QR code.

[0973] Step 2:

[0974] The electronics device transmits the QR code information and location information to the data center. The input is a data packet of the QR code and location information read by the electronics device, and the output is the information transmitted to the data center. The terminal transmits the QR code and location information to the data center via the Internet.

[0975] Step 3:

[0976] The data center analyzes the user's interests, preferences, and past visit history based on the information received. The input is the user's past behavioral data stored along with QR code information and location information, and the output is analyzed user profile data. The server extracts past user data from the database and uses a generative AI model to identify the user's interests and preferences.

[0977] Step 4:

[0978] The data center uses a generative AI model to generate individually tailored sightseeing routes and missions. The input is user profile data and information on current tourist destinations, and the output is personalized sightseeing routes and missions. The generative AI model generates optimal sightseeing routes and missions based on the user's preferences and past history.

[0979] Step 5:

[0980] The data center sends the generated sightseeing routes and missions to the electronics device. The input is the personalized sightseeing route and mission data, and the output is the sightseeing route and mission data sent to the electronics device. The server formats the sightseeing route and mission and sends it to the electronics device.

[0981] Step 6:

[0982] The electronics device uses augmented reality technology to display personalized virtual guide characters and messages. The input is tourist route and mission data received from a data center, and the output is the guide character and message displayed on the user's screen. The device uses its camera and AR function to overlay the guide character and message on the real space.

[0983] Step 7:

[0984] An electronic device analyzes a user's facial expressions and voice to recognize their emotional state in real time. The input is the user's facial image and voice data, and the output is analyzed emotional state data. The device uses a camera and microphone to capture the user's facial expressions and voice, which are then analyzed by an emotion engine.

[0985] Step 8:

[0986] The electronic device transmits emotional state data to the data center, the input is the emotional state data, and the output is the emotional state data transmitted to the data center. The terminal transmits the emotional state to the data center in real time.

[0987] Step 9:

[0988] The data center dynamically adjusts tourist routes and missions based on emotional state data. The input is emotional state data transmitted in real time, and the output is adjusted tourist route and mission data. The server analyzes the emotional data and optimizes tourist routes and missions according to the situation.

[0989] Step 10:

[0990] The data center sends the adjusted sightseeing route and mission data to the electronics device. The input is the adjusted sightseeing route and mission data, and the output is the data sent to the electronics device. The server sends the new plan to the terminal and notifies the user.

[0991] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0992] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0993] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0994] [Third embodiment]

[0995] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0996] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0997] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0998] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0999] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1000] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1001] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1002] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1003] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1004] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1005] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1006] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1007] This invention is a system that utilizes generative AI models and AR technology to provide users with individually tailored sightseeing tours. The system generates personalized sightseeing routes and missions based on the user's preferences, interests, and past visit history, and delivers them in real time.

[1008] System configuration

[1009] The system mainly consists of the following components:

[1010] 1. User's device (smartphone, etc.)

[1011] 2. Server

[1012] 3. Generative AI Models

[1013] 4. Character portrait panels as AR markers

[1014] Explanation of program processing

[1015] 1. Collecting User Information

[1016] The user finds a portrait panel installed at a tourist spot.

[1017] The user starts up the device (smartphone) and opens the dedicated app.

[1018] The device's camera will activate and scan the QR code or AR marker on the character portrait panel.

[1019] The device uses location information services to obtain the current location.

[1020] The device sends the acquired information (QR code information, current location) to the server.

[1021] 2. Analyzing data and applying generative AI models

[1022] The server compares the user's information with a database to obtain the user's past visit history and preference data.

[1023] The server uses a generative AI model to analyze this data and generate sightseeing routes and missions based on the user's interests.

[1024] Generative AI models take into account historical data and current trends to create optimal plans.

[1025] The server sends the generated sightseeing routes and missions to the device.

[1026] 3. Delivering personalized AR content

[1027] The device analyzes the tourist routes and missions received and displays notifications to the user.

[1028] The user travels along a route selected by the user and arrives at the designated tourist spot.

[1029] The device uses AR technology to display personalized guide characters and messages from a portrait panel.

[1030] Example: When you arrive at a specific spot, a guide character will say, "This is your next destination."

[1031] 4. Real-time feedback

[1032] The device periodically acquires its current location and transmits it to the server.

[1033] The server analyzes the user's progress in real time and dynamically generates next steps and recommended tourist spots.

[1034] The server sends the latest instructions to the terminal and notifies the user.

[1035] Example: Send a notification that says, "If you stop by a specific cafe on your way to the next location, a hidden mission will occur."

[1036] Specific examples

[1037] Example 1: Family sightseeing

[1038] A user (family) visits a tourist spot and scans a portrait panel.

[1039] The device sends information to a server, which then analyzes the family's past history and preferences.

[1040] The generative AI model suggests "experiential activities for parents and children to enjoy together" and "nature walks and picnic spots."

[1041] The device will notify the family of the mission it has received, and a guide character will provide guidance using AR technology.

[1042] Example 2: Solo tourism

[1043] The user (traveling alone) visits a tourist spot and scans the portrait panel.

[1044] The device sends the information to the server, which takes into account past visits and interests.

[1045] The generative AI model suggests "tours of historical sites" and "hidden cafes."

[1046] The terminal notifies the received mission and moves along the route selected by the user.

[1047] When you arrive at a specific spot, a guide character will guide you to your next destination.

[1048] The above is a specific description of the embodiment of the present invention. This system allows users to have new sightseeing experiences and encourages them to revisit tourist spots.

[1049] The processing flow will be explained below.

[1050] Step 1:

[1051] The user discovers a standing picture panel installed at a tourist spot.

[1052] The user launches the dedicated app on their device and points the camera at the character portrait panel.

[1053] Step 2:

[1054] The device uses the camera to scan the QR code or AR marker on the character portrait panel.

[1055] The device acquires the location information and sends it to the server along with the QR code information.

[1056] Step 3:

[1057] Based on the user information received by the server, the user's past visit history and preference data are obtained from the database.

[1058] The server analyzes this information and inputs it into the generative AI model.

[1059] Step 4:

[1060] The generative AI model takes into account the analysis results and current trends to generate optimal sightseeing routes and missions for users.

[1061] The server formats the generated tourist routes and missions and sends them to the terminal.

[1062] Step 5:

[1063] The device displays the sightseeing route and missions received from the server,

[1064] Notify the user.

[1065] The user confirms the notification and starts the tour.

[1066] Step 6:

[1067] When the device reaches a designated tourist spot, AR technology is used to display a guide character or message from a portrait panel.

[1068] A guide character will guide you to your next steps and tourist spots.

[1069] Step 7:

[1070] The terminal periodically acquires the user's current location and transmits it to the server.

[1071] The server analyzes your progress in real time and dynamically generates new attractions and missions.

[1072] Step 8:

[1073] The server sends the latest instructions and information to the terminal and notifies the user.

[1074] Example: New missions and recommended spots will be announced via push notifications.

[1075] Step 9:

[1076] The user checks the notification from the device and moves to the next spot.

[1077] The device continuously captures the user's location and progress and sends feedback to the server.

[1078] Step 10:

[1079] The server dynamically updates next steps and new tourist spots based on feedback.

[1080] The above is the specific program processing flow of the system. Through these processing steps, users can have a sightseeing experience that is tailored to their individual needs.

[1081] Example 1

[1082] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1083] Conventional tourism systems provide uniform tourist information, making it difficult to provide a personalized experience tailored to each user's individual preferences and interests. Furthermore, they lack the functionality to track a user's progress in real time and dynamically suggest the next tourist spot or mission, forcing users to follow a predetermined route. Furthermore, they lacked a mechanism to provide new experiences when a user arrives at a specific tourist spot. These issues need to be resolved.

[1084] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1085] In this invention, the server includes: a means for a user to read a display panel installed at a tourist attraction with an information terminal; a means for the terminal to transmit the user's location information and identification information to the server; a means for the server to analyze the user's preferences and past history based on the received information and generate personalized tourist routes and missions using a generative AI model; a means for the server to transmit the generated tourist routes and missions to the terminal; a means for the terminal to display personalized guide characters and messages using augmented reality technology; a means for the terminal to continuously transmit the user's current location to the server, and the server to dynamically analyze the user's progress and generate the next tourist spot and mission; and a means for the generative AI model to generate tourist routes and missions taking past user data and current trends into consideration and to guide the user to the next step and recommended tourist spots via push notifications. This makes it possible to provide a personalized tourist experience for each user, and to dynamically suggest the next step by analyzing the user's progress in real time.

[1086] "User" refers to an individual who uses this system to obtain information at a tourist spot and enjoy sightseeing.

[1087] "Display panel" refers to a panel installed at a tourist spot that contains a QR code or AR marker that users can use to obtain information.

[1088] "Information terminal" refers to an electronic device such as a smartphone or tablet that a user owns.

[1089] "Location Information" refers to information about a user's current location obtained using GPS or other means.

[1090] "Identification information" refers to unique information obtained by scanning the QR code or AR marker printed on the display panel.

[1091] "Server" refers to a computer system for receiving and analyzing data sent by users.

[1092] A "generative AI model" refers to an artificial intelligence system that analyzes user preferences, past history, current trends, etc. to generate tourist routes and missions.

[1093] "Tourist route" refers to a specific route suggested by the server to enable the user to efficiently tour tourist spots.

[1094] A "mission" refers to a specific task or activity that a user must accomplish at a tourist destination.

[1095] "Augmented reality technology" refers to a technology that overlays digital information on a device that displays the real world.

[1096] "Guide character" refers to a virtual character that provides sightseeing guidance to users using augmented reality technology.

[1097] "Message" refers to notifications and guidance information displayed to the user by the guide character or the system.

[1098] "Push notification" refers to a message sent in real time from a server to a user's information terminal.

[1099] "Trends" refers to information about current trends and popular spots.

[1100] "Continuous" means that the information is updated at regular intervals.

[1101] "Dynamic" refers to the way something changes in real time depending on the situation.

[1102] The present invention is a system that provides users with individually tailored tourist experiences at tourist destinations. This system uses generative AI models and augmented reality technology to generate personalized tourist routes and missions based on the user's preferences and past visit history, and provides them in real time. Specific embodiments of this system are described below.

[1103] 1. Collecting User Information

[1104] Users first find a display panel installed at a tourist spot and begin gathering information by scanning it with an information device (e.g., a smartphone). The user then launches a dedicated app on their smartphone and uses the camera to scan the QR code or AR marker on the display panel. The device then uses GPS to obtain the user's current location and sends it to the server along with the QR code information. The location information service uses the Google Maps API.

[1105] 2. Analyzing data and applying generative AI models

[1106] The server compares the user information sent from the device with a database (e.g., MySQL, MongoDB) to obtain the user's past visit history and preference data. The server then uses a generative AI model (e.g., GPT-4) to analyze this data and generate sightseeing routes and missions based on the user's interests. Examples of prompts are as follows:

[1107] "Spots visited in the past: Mount Takao, Interests: Historical landmarks. Based on this, please suggest the next tourist spot and mission."

[1108] The generative AI model takes into account past user data and current trends to create an optimal plan, and the generated sightseeing route and missions are sent from the server to the device.

[1109] 3. Delivering personalized AR content

[1110] The device analyzes the tourist routes and missions received from the server and displays them to the user as push notifications. The user then moves toward the designated tourist spots based on the notifications. Upon arriving at each spot, the device uses augmented reality technology (e.g., ARKit, ARCore) to display a personalized guide character and message on the display panel. For example, when arriving at a specific spot, a guide character will appear saying, "This is your next destination."

[1111] 4. Real-time feedback

[1112] As the user travels along a sightseeing route, the device periodically acquires its current location and sends it to the server. The server analyzes the user's progress in real time and dynamically generates next steps and recommended sightseeing spots. These results are then sent back to the device and displayed as notifications to the user. For example, a notification might say, "If you stop by a specific cafe on your way to the next spot, a hidden mission will occur."

[1113] Specific examples

[1114] For example, suppose a family visits a tourist spot and scans the display panel. The device sends the information to the server, which analyzes the family's past history and preferences. The generative AI model suggests "hands-on activities for parents and children" and "nature walks and picnic spots." The device notifies the family of the mission received, and a guide character uses augmented reality technology to provide guidance.

[1115] For solo travelers, when they visit a tourist spot, they scan the display panel, and the device sends the information to the server. The server takes into account their past visit history and interests, and the generative AI model suggests "tours of historical landmarks" and "hidden cafes." The device notifies the user of the received mission, and as they travel along the route selected by the user, a guide character will guide them to their next destination when they arrive at a specific spot.

[1116] This system allows users to have new sightseeing experiences and encourages them to revisit tourist destinations.

[1117] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1118] The flow of this system's program processing

[1119] Step 1: The user finds the display panel

[1120] Input: Display panels installed in various locations in tourist spots

[1121] Action: The user finds the display panel.

[1122] Output: Your smartphone is ready to get the information from the display panel.

[1123] Specific actions: The user walks around a tourist spot, searches for a display panel, and when they find one, takes out their smartphone.

[1124] Step 2: The user starts the device and opens the dedicated app.

[1125] Input: A smartphone owned by the user

[1126] Process: The user takes out their smartphone and launches the dedicated app.

[1127] Output: The dedicated app will launch and you will be ready to scan.

[1128] Specific operation: The user touches the smartphone screen, selects and opens the dedicated app.

[1129] Step 3: Scan the QR code or AR marker on the character panel with your device

[1130] Input: QR code or AR marker placed on the display panel

[1131] Processing: Activate the smartphone camera and scan the QR code or AR marker.

[1132] Output: QR code and AR marker information is obtained

[1133] Specific operation: The user points the smartphone camera at the display panel and scans the QR code or AR marker.

[1134] Step 4: The device acquires location information and sends it to the server

[1135] Input: Current location information of the device using GPS, as well as information on QR codes and AR markers

[1136] Processing: Send location information and scanned information to the server

[1137] Output: Location information and QR code information are sent to the server

[1138] Specific operation: The device uses location services (e.g., Google Maps API) to obtain the current location and sends the obtained information to the server via HTTPS protocol.

[1139] Step 5: The server checks the user information against the database

[1140] Input: Location information and QR code information sent from the device

[1141] Processing: The server checks the database to obtain the user's past visit history and preference data.

[1142] Output: User's past visit history and preference data

[1143] Specific operation: The server accesses a database (e.g., MySQL, MongoDB) and retrieves the information to be matched.

[1144] Step 6: The server generates tourist routes and missions using the generative AI model

[1145] Input: User's past visit history, preference data, current location information

[1146] Processing: Input prompts into the generative AI model to generate personalized sightseeing routes and missions.

[1147] Output: personalized tourist routes and missions

[1148] Specific operation: An example of a prompt sentence: "Spots visited in the past: Mt. Takao, Interests: Historical buildings. Based on this, please suggest the next tourist spot and mission." is input into a generative AI model (e.g., GPT-4) and the generated results are obtained.

[1149] Step 7: The server sends the generated results to the device.

[1150] Input: Tourist routes and missions generated by generative AI models

[1151] Process: Send the generated results to the terminal

[1152] Output: Tourist route and mission data sent to the device

[1153] Specific operation: The server sends the generated results to the terminal using the HTTPS protocol.

[1154] Step 8: The device analyzes the sightseeing route and mission and displays a notification

[1155] Input: Tourist route and mission data sent from the server

[1156] Processing: The device analyzes the data and displays a notification to the user

[1157] Output: The notification displayed to the user

[1158] Specific operation: The device analyzes the sightseeing route and mission data and displays a notification saying, "Your next destination is XX Park."

[1159] Step 9: User follows notification

[1160] Input: Device notifications

[1161] Action: The user follows the notification and moves to the designated tourist spot.

[1162] Output: The user arrives at the specified location.

[1163] Specific behavior: The user checks the smartphone screen and moves according to the displayed route.

[1164] Step 10: The device uses AR technology to display personalized content.

[1165] Input: User location information, AR content data

[1166] Processing: The device displays a guide character or message using augmented reality technology.

[1167] Output: Personalized AR content displayed to the user

[1168] Specific operation: When the user arrives at the designated spot, a guide character will be displayed using the device's camera, saying, "This is your next destination."

[1169] Step 11: The device periodically acquires location information and sends it to the server.

[1170] Input: current location of the user

[1171] Processing: The device periodically acquires location information using GPS and sends it to the server.

[1172] Output: Location information sent to the server

[1173] Specific operation: The device acquires its current location at regular intervals and sends that information to the server.

[1174] Step 12: The server analyzes the progress and generates the next step

[1175] Input: current location of the user

[1176] Processing: The server analyzes the progress and generates the next attractions and missions.

[1177] Output: Data on the next tourist spot or mission

[1178] Specific operation: The server uses the generative AI model to generate the next route and mission based on the user's current location.

[1179] Step 13: The server sends the latest instructions to the terminal and notifies the user.

[1180] Input: Data for the next tourist spot or mission

[1181] Processing: The server sends the latest instructions to the terminal and notifies the user.

[1182] Output: Notification displayed on the user's device

[1183] Specific operation: The server generates the next step and sends it to the device, which then displays a notification saying, "If you stop by a specific cafe on your way to the next spot, a hidden mission will occur."

[1184] (Application example 1)

[1185] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1186] There is a demand for tourist destinations and brick-and-mortar stores to provide users with personalized information based on their preferences, interests, and past visit history, thereby providing a more fulfilling experience. However, it is difficult for users to efficiently and intuitively obtain information and find optimal routes and recommended products without receiving a large amount of information all at once. Current systems do not adequately provide individually tailored information, making it difficult for users to obtain information that perfectly matches their interests. Therefore, the objective of this invention is to provide a system that can provide users with individually tailored sightseeing routes and shopping routes in real time.

[1187] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1188] In this invention, the server includes means for analyzing the user's interests, preferences, and past visit history based on the user's location information and QR code information, means for generating individually suited sightseeing routes and missions using a generative AI model, and means for displaying AR content based on AR markers detected by the device's camera, thereby enabling the provision of personalized sightseeing routes, shopping routes, and recommended products to the user.

[1189] "User" refers to an individual who uses this system to receive information at tourist spots and brick-and-mortar stores.

[1190] "Terminal" refers to a device carried by a user, such as a smartphone, smart glasses, or head-mounted display, that acquires QR code information and location information and displays AR content.

[1191] "Server" refers to the central processing unit that processes user information and generates sightseeing routes, missions, and shopping routes using generative AI models.

[1192] "Generative AI model" refers to an artificial intelligence model that analyzes a user's interests and preferences, past visit history, and current trends to generate individually appropriate tourist routes and missions.

[1193] "AR technology" refers to augmented reality technology, which displays digital content overlaid on real-world information.

[1194] A "tourist route" refers to the optimal order or route for a user to visit tourist spots.

[1195] A "mission" refers to a specific action or task presented to a user.

[1196] "QR code" refers to a two-dimensional barcode consisting of a rectangular pattern of dots, and is a format for encoding information.

[1197] "AR marker" refers to a printed or digital image that serves as a reference for displaying content using augmented reality technology.

[1198] "Location Information" means a user's current geographic location, which may be obtained using GPS or other location-determining technology.

[1199] "Progress" refers to information indicating how far a user has progressed along a given sightseeing or shopping route.

[1200] A "shopping route" refers to the optimal order or route for a user to efficiently shop in a physical store.

[1201] "Benefit information" refers to information such as discounts, campaigns, and bonuses offered to users.

[1202] This invention is a system for providing users with personalized sightseeing and shopping routes at tourist spots and brick-and-mortar stores. The system utilizes generative AI models and AR technology to provide individually tailored information based on the user's preferences, interests, and past visit history.

[1203] System configuration

[1204] The system mainly consists of the following components:

[1205] 1. User's device (smartphone, smart glasses, head-mounted display, etc.)

[1206] 2. Server

[1207] 3. Generative AI Models

[1208] 4. AR markers (including QR codes)

[1209] User's device

[1210] The user's device may be a smartphone, smart glasses, or a head-mounted display. These devices have a camera function and can scan AR markers and QR codes installed at tourist spots and in brick-and-mortar stores. When the device scans these markers, the acquired data along with location information is sent to a server. Furthermore, the device uses AR technology to display personalized guide characters and messages.

[1211] server

[1212] The server has the ability to analyze the user's interests, preferences, and past visit history based on the user's location information and QR code information. A generative AI model running on the server takes this data and current trends into account to generate optimal sightseeing and shopping routes. The generated routes and missions are then sent to the user's device in real time.

[1213] Generative AI Models

[1214] A generative AI model is an artificial intelligence model that analyzes a user's interests, preferences, and past visit history to personalize sightseeing or shopping routes. This model uses user data and current trends to generate an optimal plan and sends it to the user's device via a server. An example of a specific prompt is, "User 'User123' has the following past purchase history: chairs, tables, and shelves. Taking into account the latest trends, please recommend new furniture that may be of interest to them. Also, please generate the optimal shopping route within the store."

[1215] AR technology

[1216] AR technology is a technology that overlays digital content onto the real world through the user's device. When a user scans an AR marker in a tourist spot or physical store, personalized guide characters, product information, and messages are displayed on the device screen. This plays an important role in improving the user experience.

[1217] Specific examples

[1218] Example 1: Family sightseeing

[1219] A user (family) visits a tourist spot and scans a character panel. The device sends information to the server, which analyzes the family's past history and preferences. The generative AI model suggests "hands-on activities for parents and children to enjoy together" and "nature walks and picnic spots." The device notifies the family of the mission it received, and a guide character uses AR technology to provide guidance.

[1220] Example 2: Solo tourism

[1221] A user (solo traveler) visits a tourist spot and scans a character panel. The device sends the information to the server, which takes into account past visit history and interests. The generative AI model suggests "tours of historical landmarks" and "hidden cafes." As the user travels along the route selected, a guide character will guide them to their next destination when they arrive at a specific spot.

[1222] The components and operation procedures described above allow users to have a more fulfilling experience at tourist spots and brick-and-mortar stores.

[1223] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1224] Step 1:

[1225] The user uses a device to scan AR markers or QR codes installed at tourist spots or brick-and-mortar stores. The device's camera acquires the QR code or AR marker information, and collects location information along with this information. The input is "QR code information" and "location information." This data is then combined into a single communication packet for data processing. The output is "data that is a packet of the user's QR code information and location information."

[1226] Step 2:

[1227] The QR code information and location information collected by the device are sent to the server. The input is "data that is a packet of the user's QR code information and location information." The processing involves sending the data using an HTTP request. The output is "user data sent to the server."

[1228] Step 3:

[1229] The server analyzes the user's QR code information and location information received. The input is "user data sent to the server." Data calculations use database matching and a data analysis engine to obtain the user's past visit history and preference data. The output is "the analysis results of the user information."

[1230] Step 4:

[1231] The server uses the generative AI model to generate individually tailored sightseeing and shopping routes based on the user's past visit history and preference data. The input is the "analysis results of user information." The data is passed to the generative AI model for data processing and calculation, generating multiple personalized options. As a concrete example, the prompt statement used is "Please recommend tourist spots of interest to user 'User123' based on his past visit history." The output is the "generated sightseeing and shopping routes."

[1232] Step 5:

[1233] The server sends the generated sightseeing route or shopping route to the user's device. The input is the "generated sightseeing route or shopping route." The processing involves sending the data using an HTTP response. The output is the "data sent to the user's device."

[1234] Step 6:

[1235] The device analyzes the tourist route and shopping route data received and displays a notification to the user. The input is "data sent to the user's device." Data processing involves reformatting the data into a notification format. The output is "a notification message displayed to the user."

[1236] Step 7:

[1237] When the user arrives at the designated tourist spot or inside the store, the device's camera scans the AR marker again and displays the AR content. The input is "AR marker information." The data is processed by selecting and displaying the appropriate AR content based on the marker information. The output is "AR content displayed on the device."

[1238] Step 8:

[1239] The device periodically acquires the current location and sends it to the server, allowing the user's progress to be analyzed in real time. The input is "current location data." The data is analyzed in real time as part of the process, and information about the next tourist spot and special offers is dynamically generated. The output is "next step and special offer information tailored to the user's progress."

[1240] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1241] This invention is a system that combines generative AI models, AR technology, and an emotion engine to provide users with individually tailored sightseeing tours. The system generates personalized sightseeing routes and missions based on the user's preferences and past visit history, and provides them in real time. It also has the ability to recognize the user's emotional state and dynamically adjust the sightseeing experience based on that.

[1242] System configuration

[1243] The system mainly consists of the following components:

[1244] 1. User's device (smartphone, etc.)

[1245] 2. Server

[1246] 3. Generative AI Models

[1247] 4. Character portrait panels as AR markers

[1248] 5. Emotion Engine

[1249] Explanation of program processing

[1250] 1. Collecting User Information

[1251] The user discovers a standing image panel installed at a tourist spot and launches a dedicated app on their device.

[1252] The device's camera scans the QR code or AR marker on the character portrait panel to obtain location information.

[1253] The device sends the QR code information and location information to the server.

[1254] 2. Analyzing data and applying generative AI models

[1255] The server compares the received user information with a database to obtain the user's past visit history and preference data.

[1256] The server uses a generative AI model to analyze this data and generate sightseeing routes and missions based on the user's interests.

[1257] Generative AI models take into account past data and current trends to create optimal plans.

[1258] The server formats the generated tourist routes and missions and sends them to the terminal.

[1259] 3. Delivering personalized AR content

[1260] The terminal displays the sightseeing route and mission received from the server and notifies the user.

[1261] The user confirms the notification and starts the tour.

[1262] When the device reaches a designated tourist spot, AR technology is used to display a guide character or message from a portrait panel.

[1263] 4. Applying the Emotion Engine

[1264] The device analyzes the user's facial expressions and voice using an emotion engine to recognize their emotional state in real time.

[1265] The device transmits the emotion data to the server.

[1266] The server dynamically adjusts sightseeing routes and missions based on emotion data, and suggests next spots and new activities.

[1267] 5. Real-time feedback

[1268] The device periodically acquires its current location and transmits it to the server.

[1269] The server analyzes the user's progress and emotional state in real time and dynamically generates next steps and recommended tourist spots.

[1270] The server sends the latest instructions and information to the terminal and notifies the user.

[1271] Specific examples

[1272] Example 1: Family sightseeing

[1273] A user (family) visits a tourist spot and scans a portrait panel.

[1274] The device sends information to a server, which then analyzes the family's past history and preferences.

[1275] The generative AI model suggests "experiential activities for parents and children to enjoy together" and "nature walks and picnic spots."

[1276] The device will notify the family of the mission it has received, and a guide character will provide guidance using AR technology.

[1277] The emotion engine recognizes a child's state of excitement and dynamically suggests relaxing spots.

[1278] Example 2: Solo tourism

[1279] The user (traveling alone) visits a tourist spot and scans the portrait panel.

[1280] The device sends the information to the server, which takes into account past visits and interests.

[1281] The generative AI model suggests "tours of historical sites" and "hidden cafes."

[1282] The terminal notifies the received mission and moves along the route selected by the user.

[1283] When you arrive at a specific spot, a guide character will guide you to your next destination.

[1284] The emotion engine recognizes the user's level of satisfaction from their facial expressions and suggests more stimulating spots.

[1285] This concludes the detailed description of the embodiment of the present invention. This system allows users to have new sightseeing experiences and encourages repeat visits to tourist spots. Furthermore, the introduction of an emotion engine makes it possible to provide experiences based on the user's emotions, realizing more personalized services.

[1286] The processing flow will be explained below.

[1287] Step 1:

[1288] The user discovers a standing picture panel installed at a tourist spot.

[1289] The user launches the dedicated app on their device (smartphone, etc.) and points the camera at the character portrait panel.

[1290] Step 2:

[1291] The device uses the camera to scan the QR code or AR marker on the character portrait panel.

[1292] The device acquires location information and sends the QR code information and location information to the server.

[1293] Step 3:

[1294] Based on the user information received by the server, the user's past visit history and preference data are obtained from the database.

[1295] The server analyzes this data and inputs it into a generative AI model.

[1296] Step 4:

[1297] The generative AI model takes into account the analysis results and current trends to generate optimal sightseeing routes and missions for users.

[1298] The server formats the generated tourist routes and missions and sends them to the terminal.

[1299] Step 5:

[1300] The terminal displays the sightseeing route and mission received from the server and notifies the user.

[1301] The user confirms the notification and starts the tour.

[1302] Step 6:

[1303] When the device reaches a designated tourist spot, AR technology is used to display a guide character or message from a portrait panel.

[1304] A guide character will guide you to your next steps and tourist spots.

[1305] Step 7:

[1306] The device analyzes the user's facial expressions and voice data using an emotion engine to recognize their emotional state in real time.

[1307] The device transmits the emotion data to the server.

[1308] Step 8:

[1309] The server dynamically adjusts sightseeing routes and missions based on emotion data, and suggests next spots and new activities.

[1310] Example: The server detects the user's fatigue and suggests a spot where they can relax.

[1311] Step 9:

[1312] The terminal periodically acquires the user's current location and transmits it to the server.

[1313] The server analyzes the user's progress and emotional state in real time and dynamically generates next steps and recommended tourist spots.

[1314] Step 10:

[1315] The server sends the latest instructions and information to the terminal and notifies the user.

[1316] Example: New missions and recommended spots will be announced via push notifications.

[1317] Step 11:

[1318] The user checks the notification from the device and moves to the next spot.

[1319] The device continuously captures the user's location and progress and sends feedback to the server.

[1320] Step 12:

[1321] The server dynamically updates next steps and new tourist spots based on feedback.

[1322] For example: Suggest a special activity at a cafe on the way to your next destination.

[1323] The above is the specific program processing flow of the system that combines the emotion engine. This system allows users to have a sightseeing experience that is tailored to their individual needs, and can encourage them to revisit tourist spots.

[1324] Example 2

[1325] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1326] In recent years, there has been a growing demand for personalized tourism experiences tailored to the preferences and interests of each individual tourist. However, conventional tourist information systems have not been able to adequately address the individual needs of tourists. As a result, it has been difficult to provide tourists with a more fulfilling tourism experience and maximize the appeal of tourist destinations. Furthermore, there has been no system that can recognize the emotional state of users in real time while sightseeing and dynamically adjust sightseeing routes and missions accordingly.

[1327] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1328] In this invention, the server includes a means for a user to scan a character image panel installed at a tourist spot with a mobile communication terminal, a means for the terminal to transmit the user's location information and QR code information to a data center, a means for the data center to analyze the user's interests, preferences, and past visit history based on this information and generate individually suited tourist routes and missions using a generative AI model, a means for the data center to transmit the generated tourist routes and missions to the terminal, and a means for the terminal to display personalized guide characters and messages using augmented reality technology. This allows users to not only experience optimal tourist routes and missions based on their preferences and past visit history, but also to receive services tailored to their emotional state in real time.

[1329] A "user" is an individual who uses this system to visit a tourist spot.

[1330] A "mobile communication terminal" is a device such as a smartphone or tablet that is carried by a user.

[1331] "Standing picture panels" are panels containing QR codes and AR markers that are installed at tourist spots.

[1332] "Augmented reality technology" is a technology that displays virtual information overlaid on real-world images.

[1333] A "data center" is a collection of servers that receives information sent by users and performs analysis and generation processing.

[1334] A "QR code" is a two-dimensional barcode that stores information and can be read by scanning it.

[1335] "Location information" is geographical data that indicates the user's current location.

[1336] A "generative AI model" is an algorithm that analyzes a user's past data and trends to generate optimal sightseeing routes and missions.

[1337] A "tourist route" is a plan showing the route a user will take at a tourist spot.

[1338] A "mission" is a specific task or activity that a user must accomplish while sightseeing.

[1339] A "guide character" is a virtual guide displayed through augmented reality technology.

[1340] The "emotional state" is the psychological state recognized from the user's facial expression and voice.

[1341] "Push notification" is a notification function that delivers information to users in real time.

[1342] The present invention is a system that provides users with personalized sightseeing experiences. This system generates sightseeing routes and missions based on the user's preferences and past visit history, and provides them in real time. It also has the function of recognizing the user's emotional state and dynamically adjusting the sightseeing experience based on that. Specific embodiments of this system are described below.

[1343] Hardware and Software Configuration

[1344] In order to implement the present invention, the following hardware and software are used.

[1345] 1. User's device (mobile communication device)

[1346] Smartphones, tablets, etc.

[1347] 2. Server

[1348] It is equipped with a database of user information, a generative AI model, and an emotion engine.

[1349] 3. Generative AI Models

[1350] An algorithm that analyzes user preferences, past visit history, current trends, etc. to generate optimal tourist routes and missions.

[1351] 4. Augmented Reality Technology (AR Technology)

[1352] Used on the user's device, it displays virtual guide characters and messages.

[1353] 5. Data Center

[1354] It functions as a cloud server or on-premise server and processes user location information, QR code information and emotion data.

[1355] Example of system operation

[1356] Example 1: Family sightseeing

[1357] A user (family) visits a tourist spot and scans a portrait panel.

[1358] The device sends information to a data center, which then analyzes the family's past history and preferences.

[1359] The generative AI model suggests "experiential activities for parents and children to enjoy together" and "nature walks and picnic spots."

[1360] The device will notify the family of the mission it has received, and a guide character will provide guidance using AR technology.

[1361] The emotion engine recognizes a child's state of excitement and dynamically suggests relaxing spots.

[1362] Example 2: Solo tourism

[1363] The user (traveling alone) visits a tourist spot and scans the portrait panel.

[1364] The device sends the information to a data center, which takes into account past visits and interests.

[1365] The generative AI model suggests "tours of historical sites" and "hidden cafes."

[1366] The terminal notifies the received mission and moves along the route selected by the user.

[1367] When you arrive at a specific spot, a guide character will guide you to your next destination.

[1368] The emotion engine recognizes the user's level of satisfaction from their facial expressions and suggests more stimulating spots.

[1369] Prompt Sentence Examples

[1370] "Please explain in detail the process of scanning the portrait panels, sending the obtained user information to the data center, and having the AI ​​model generate the optimal tourist route."

[1371] "Please explain a specific scenario where a family visits a tourist destination and an emotion engine is used to tailor their experience in real time."

[1372] "Please explain how a generative AI model can provide a personalized sightseeing route for a solo traveler."

[1373] The embodiments of the present invention allow users to have new sightseeing experiences and encourage them to revisit tourist destinations. Furthermore, the introduction of an emotion engine allows for experiences based on the user's emotions, enabling more personalized services.

[1374] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1375] Step 1:

[1376] The user scans a portrait panel at a tourist spot using their smartphone camera.

[1377] Specific operation: The user holds the device's camera over the QR code or AR marker and taps "Scan" in the dedicated app.

[1378] Input: QR code and AR marker information on the character portrait panel displayed on the user's smartphone.

[1379] Output: QR code information and location information are obtained.

[1380] Step 2:

[1381] The device sends the acquired QR code information and location information to the data center.

[1382] Specific operation: The device uploads the QR code information and GPS location information to the server via a dedicated app.

[1383] Input: QR code information and GPS location information.

[1384] Output: QR code information and location information are sent to the data center.

[1385] Step 3:

[1386] Based on the information received by the server, the user's preferences and past visit history are retrieved from the database and analyzed using a generative AI model.

[1387] Specific operation: The server queries the database to obtain user information, then inputs that data into the generative AI model to obtain analysis results.

[1388] Input: QR code information, location information, user preference data, past visit history data.

[1389] Output: Suggestions for optimal sightseeing routes and missions for the user.

[1390] Step 4:

[1391] The server formalizes the tourist routes and missions generated and sends them to the device via a dedicated app.

[1392] Specific operation: The server converts the tourist route and missions into a format such as JSON and sends them to the terminal.

[1393] Input: The output of a generative AI model.

[1394] Output: Tourist route and mission data is sent to the terminal.

[1395] Step 5:

[1396] The device notifies the user of the tourist route and missions received from the server, and displays guide characters and messages using AR technology.

[1397] Specific operation: The device will display "New tourist route has arrived" in the notification bar and switch to AR mode to display the virtual character.

[1398] Input: Received tourist route and mission data.

[1399] Output: Notifications and AR guide information.

[1400] Step 6:

[1401] The device periodically obtains the user's current location and sends it to the data center.

[1402] Specific operation: Obtain location information using the device's GPS function and periodically upload it to the server.

[1403] Input: Current location.

[1404] Output: The location information is sent to the server.

[1405] Step 7:

[1406] The device analyzes the user's facial expressions and voice using an emotion engine to recognize their emotional state in real time.

[1407] Specific operation: The device's camera and microphone capture the user's facial expressions and voice, and the emotion engine provides the analysis results.

[1408] Input: User's facial and voice data.

[1409] Output: Recognized emotional state data.

[1410] Step 8:

[1411] The server dynamically adjusts sightseeing routes and missions based on emotion data and current location information, suggesting new activities and next spots.

[1412] Specific operation: The server analyzes the emotion data and location information, generates a new sightseeing route, and sends it to the device.

[1413] Input: Emotional state data and location information.

[1414] Output: Dynamically adjusted tourist routes and new mission data.

[1415] Step 9:

[1416] The terminal notifies the user of the latest instructions and information received from the server.

[1417] Specific operation: The device will display "Please move to the next spot" in the notification bar and provide information about new tourist spots.

[1418] Input: The latest instructions and information from the server.

[1419] Output: Notify the user and display tourist spot information.

[1420] (Application example 2)

[1421] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1422] Conventional tourist information systems have struggled to provide personalized sightseeing routes and missions that fully reflect a user's individual preferences and interests. They also struggled to recognize a user's emotional state in real time and dynamically adjust the sightseeing experience based on that information. This resulted in users being unable to obtain a fully satisfying sightseeing experience, and they were unable to encourage repeat visits to tourist destinations.

[1423] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to scan a portrait panel installed at a tourist attraction with an electronics device; a means for the electronics device to transmit the user's location information and QR code information to a data center; a means for the data center to analyze the user's interests, preferences, and past visit history based on this information and generate individually tailored tourist routes and missions using a generative AI model; a means for the data center to transmit the generated tourist routes and missions to the electronics device; a means for the electronics device to display personalized virtual guide characters and messages using augmented reality technology; a means for the electronics device to analyze the user's facial expressions and voice and recognize the user's emotional state in real time; and a means for the data center to dynamically adjust the tourist routes and missions based on the emotion analysis data. This makes it possible to provide a more satisfying tourist experience tailored to the user's individual preferences and emotional state.

[1424] "User" refers to a person who uses an electronic device to scan the portrait panels installed at tourist spots and experience the tourist routes and missions provided.

[1425] A "tourist destination" refers to an area or place that contains attractions and sights that are of interest to visitors.

[1426] "Standing picture panels" are displays installed at tourist spots that users can scan with their electronic device's camera to read QR codes and AR markers.

[1427] "Electronic devices" refers to electronic devices that users carry and use, such as smartphones and smart glasses.

[1428] "Location information" is data indicating the user's current location within a tourist attraction.

[1429] A "QR code" is a two-dimensional barcode that can be read by an electronic device to obtain the embedded information.

[1430] "Data Center" refers to a service provider that collects and analyzes data and information sent by users and generates personalized tourist routes and missions using generative AI models.

[1431] "Interests and preferences" refers to the interests and preferences that a user has shown in the past.

[1432] "Past visit history" refers to a record of places a user has visited and activities they have participated in in the past.

[1433] A "generative AI model" refers to an artificial intelligence algorithm that generates optimal tourist routes and missions based on a user's preferences and past visit history.

[1434] A "tourist route" refers to the optimal route or directions that a user should follow when sightseeing.

[1435] "Mission" refers to a specific activity or experience offered to a user.

[1436] "Augmented reality technology" is a technology that displays virtual information overlaid on the real world, allowing users to experience it through electronic devices.

[1437] "Virtual guide character" refers to a virtual character that uses augmented reality technology to guide visitors through sightseeing routes and missions.

[1438] "Message" refers to textual or visual information displayed to a user through an electronic device.

[1439] "Facial expression" refers to the expression of emotion shown by the movement of the user's facial muscles.

[1440] "Voice" refers to the tone and manner of speaking of the user.

[1441] "Emotional state" refers to the user's current state of mind and is recognized in real time using an emotion engine.

[1442] "Dynamic adjustment" refers to adapting sightseeing routes and missions on the fly based on the user's real-time emotional state and progress.

[1443] A system embodying this invention provides a personalized sightseeing experience to users using an electronic device when visiting a tourist attraction. The system begins when the user scans a portrait panel installed at the tourist attraction with the electronic device. The electronic device then transmits the user's location information and QR code information to a data center.

[1444] The data center analyzes the user's interests, preferences, and past visit history based on the received information, and uses a generative AI model to generate individually tailored sightseeing routes and missions, which are then sent from the data center to the electronics device.

[1445] The electronics device uses augmented reality technology to display personalized virtual guide characters and messages to the user. As the user moves along the sightseeing route, the electronics device analyzes the user's facial expressions and voice to recognize their current emotional state in real time. The analysis results of the emotional state are sent to a data center, which then dynamically adjusts the sightseeing route and missions based on the emotion analysis data.

[1446] The specific hardware and software used include smartphones and smart glasses as electronic devices, equipped with cameras and AR functions. For emotion analysis, OpenCV and Keras are used, for example. The data center is based on cloud services and servers, and databases and generative AI models are used to store and analyze user data.

[1447] For example, when a user scans a portrait panel installed at a tourist spot with their smartphone and the information is sent to the data center, the data center uses a generative AI model to generate a sightseeing route that matches the user's preferences. For example, for a user with their family, it can suggest "hands-on activities to enjoy together as a family" and "nature walks and picnic spots." It can also dynamically suggest relaxing spots if it detects excitement in the user's facial expression.

[1448] An example prompt is:

[1449] User's past purchase history:

[1450] Product ID: 123, Category: Clothing, Purchase Date: 2023-09-01

[1451] Product ID: 456, Category: Gadgets, Purchase Date: 2023-09-05

[1452] User's current emotional state: Positive

[1453] Based on this information, generate three recommended items for this user.

[1454] In this way, the present invention can provide users with a personalized sightseeing experience in real time, thereby improving user satisfaction and encouraging repeat visits to tourist destinations.

[1455] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1456] Step 1:

[1457] A user scans a standing picture panel installed at a tourist spot with an electronic device. The input is the QR code read by the user's electronic device and location information, and the output is a data packet containing this information. The user scans the standing picture panel with a camera and obtains their current location information along with the QR code.

[1458] Step 2:

[1459] The electronics device transmits the QR code information and location information to the data center. The input is a data packet of the QR code and location information read by the electronics device, and the output is the information transmitted to the data center. The terminal transmits the QR code and location information to the data center via the Internet.

[1460] Step 3:

[1461] The data center analyzes the user's interests, preferences, and past visit history based on the information received. The input is the user's past behavioral data stored along with QR code information and location information, and the output is analyzed user profile data. The server extracts past user data from the database and uses a generative AI model to identify the user's interests and preferences.

[1462] Step 4:

[1463] The data center uses a generative AI model to generate individually tailored sightseeing routes and missions. The input is user profile data and information on current tourist destinations, and the output is personalized sightseeing routes and missions. The generative AI model generates optimal sightseeing routes and missions based on the user's preferences and past history.

[1464] Step 5:

[1465] The data center sends the generated sightseeing routes and missions to the electronics device. The input is the personalized sightseeing route and mission data, and the output is the sightseeing route and mission data sent to the electronics device. The server formats the sightseeing route and mission and sends it to the electronics device.

[1466] Step 6:

[1467] The electronics device uses augmented reality technology to display personalized virtual guide characters and messages. The input is tourist route and mission data received from a data center, and the output is the guide character and message displayed on the user's screen. The device uses its camera and AR function to overlay the guide character and message on the real space.

[1468] Step 7:

[1469] An electronic device analyzes a user's facial expressions and voice to recognize their emotional state in real time. The input is the user's facial image and voice data, and the output is analyzed emotional state data. The device uses a camera and microphone to capture the user's facial expressions and voice, which are then analyzed by an emotion engine.

[1470] Step 8:

[1471] The electronic device transmits emotional state data to the data center, the input is the emotional state data, and the output is the emotional state data transmitted to the data center. The terminal transmits the emotional state to the data center in real time.

[1472] Step 9:

[1473] The data center dynamically adjusts tourist routes and missions based on emotional state data. The input is emotional state data transmitted in real time, and the output is adjusted tourist route and mission data. The server analyzes the emotional data and optimizes tourist routes and missions according to the situation.

[1474] Step 10:

[1475] The data center sends the adjusted sightseeing route and mission data to the electronics device. The input is the adjusted sightseeing route and mission data, and the output is the data sent to the electronics device. The server sends the new plan to the terminal and notifies the user.

[1476] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1477] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1478] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1479] [Fourth embodiment]

[1480] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1481] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1482] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1483] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1484] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1485] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1486] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1487] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1488] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1489] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1490] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1491] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1492] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1493] This invention is a system that utilizes generative AI models and AR technology to provide users with individually tailored sightseeing tours. The system generates personalized sightseeing routes and missions based on the user's preferences, interests, and past visit history, and delivers them in real time.

[1494] System configuration

[1495] The system mainly consists of the following components:

[1496] 1. User's device (smartphone, etc.)

[1497] 2. Server

[1498] 3. Generative AI Models

[1499] 4. Character portrait panels as AR markers

[1500] Explanation of program processing

[1501] 1. Collecting User Information

[1502] The user finds a portrait panel installed at a tourist spot.

[1503] The user starts up the device (smartphone) and opens the dedicated app.

[1504] The device's camera will activate and scan the QR code or AR marker on the character portrait panel.

[1505] The device uses location information services to obtain the current location.

[1506] The device sends the acquired information (QR code information, current location) to the server.

[1507] 2. Analyzing data and applying generative AI models

[1508] The server compares the user's information with a database to obtain the user's past visit history and preference data.

[1509] The server uses a generative AI model to analyze this data and generate sightseeing routes and missions based on the user's interests.

[1510] Generative AI models take into account historical data and current trends to create optimal plans.

[1511] The server sends the generated sightseeing routes and missions to the device.

[1512] 3. Delivering personalized AR content

[1513] The device analyzes the tourist routes and missions received and displays notifications to the user.

[1514] The user travels along a route selected by the user and arrives at the designated tourist spot.

[1515] The device uses AR technology to display personalized guide characters and messages from a portrait panel.

[1516] Example: When you arrive at a specific spot, a guide character will say, "This is your next destination."

[1517] 4. Real-time feedback

[1518] The device periodically acquires its current location and transmits it to the server.

[1519] The server analyzes the user's progress in real time and dynamically generates next steps and recommended tourist spots.

[1520] The server sends the latest instructions to the terminal and notifies the user.

[1521] Example: Send a notification that says, "If you stop by a specific cafe on your way to the next location, a hidden mission will occur."

[1522] Specific examples

[1523] Example 1: Family sightseeing

[1524] A user (family) visits a tourist spot and scans a portrait panel.

[1525] The device sends information to a server, which then analyzes the family's past history and preferences.

[1526] The generative AI model suggests "experiential activities for parents and children to enjoy together" and "nature walks and picnic spots."

[1527] The device will notify the family of the mission it has received, and a guide character will provide guidance using AR technology.

[1528] Example 2: Solo tourism

[1529] The user (traveling alone) visits a tourist spot and scans the portrait panel.

[1530] The device sends the information to the server, which takes into account past visits and interests.

[1531] The generative AI model suggests "tours of historical sites" and "hidden cafes."

[1532] The terminal notifies the received mission and moves along the route selected by the user.

[1533] When you arrive at a specific spot, a guide character will guide you to your next destination.

[1534] The above is a specific description of the embodiment of the present invention. This system allows users to have new sightseeing experiences and encourages them to revisit tourist spots.

[1535] The processing flow will be explained below.

[1536] Step 1:

[1537] The user discovers a standing picture panel installed at a tourist spot.

[1538] The user launches the dedicated app on their device and points the camera at the character portrait panel.

[1539] Step 2:

[1540] The device uses the camera to scan the QR code or AR marker on the character portrait panel.

[1541] The device acquires the location information and sends it to the server along with the QR code information.

[1542] Step 3:

[1543] Based on the user information received by the server, the user's past visit history and preference data are obtained from the database.

[1544] The server analyzes this information and inputs it into the generative AI model.

[1545] Step 4:

[1546] The generative AI model takes into account the analysis results and current trends to generate optimal sightseeing routes and missions for users.

[1547] The server formats the generated tourist routes and missions and sends them to the terminal.

[1548] Step 5:

[1549] The device displays the sightseeing route and missions received from the server,

[1550] Notify the user.

[1551] The user confirms the notification and starts the tour.

[1552] Step 6:

[1553] When the device reaches a designated tourist spot, AR technology is used to display a guide character or message from a portrait panel.

[1554] A guide character will guide you to your next steps and tourist spots.

[1555] Step 7:

[1556] The terminal periodically acquires the user's current location and transmits it to the server.

[1557] The server analyzes your progress in real time and dynamically generates new attractions and missions.

[1558] Step 8:

[1559] The server sends the latest instructions and information to the terminal and notifies the user.

[1560] Example: New missions and recommended spots will be announced via push notifications.

[1561] Step 9:

[1562] The user checks the notification from the device and moves to the next spot.

[1563] The device continuously captures the user's location and progress and sends feedback to the server.

[1564] Step 10:

[1565] The server dynamically updates next steps and new tourist spots based on feedback.

[1566] The above is the specific program processing flow of the system. Through these processing steps, users can have a sightseeing experience that is tailored to their individual needs.

[1567] Example 1

[1568] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1569] Conventional tourism systems provide uniform tourist information, making it difficult to provide a personalized experience tailored to each user's individual preferences and interests. Furthermore, they lack the functionality to track a user's progress in real time and dynamically suggest the next tourist spot or mission, forcing users to follow a predetermined route. Furthermore, they lacked a mechanism to provide new experiences when a user arrives at a specific tourist spot. These issues need to be resolved.

[1570] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1571] In this invention, the server includes: a means for a user to read a display panel installed at a tourist attraction with an information terminal; a means for the terminal to transmit the user's location information and identification information to the server; a means for the server to analyze the user's preferences and past history based on the received information and generate personalized tourist routes and missions using a generative AI model; a means for the server to transmit the generated tourist routes and missions to the terminal; a means for the terminal to display personalized guide characters and messages using augmented reality technology; a means for the terminal to continuously transmit the user's current location to the server, and the server to dynamically analyze the user's progress and generate the next tourist spot and mission; and a means for the generative AI model to generate tourist routes and missions taking past user data and current trends into consideration and to guide the user to the next step and recommended tourist spots via push notifications. This makes it possible to provide a personalized tourist experience for each user, and to dynamically suggest the next step by analyzing the user's progress in real time.

[1572] "User" refers to an individual who uses this system to obtain information at a tourist spot and enjoy sightseeing.

[1573] "Display panel" refers to a panel installed at a tourist spot that contains a QR code or AR marker that users can use to obtain information.

[1574] "Information terminal" refers to an electronic device such as a smartphone or tablet that a user owns.

[1575] "Location Information" refers to information about a user's current location obtained using GPS or other means.

[1576] "Identification information" refers to unique information obtained by scanning the QR code or AR marker printed on the display panel.

[1577] "Server" refers to a computer system for receiving and analyzing data sent by users.

[1578] A "generative AI model" refers to an artificial intelligence system that analyzes user preferences, past history, current trends, etc. to generate tourist routes and missions.

[1579] "Tourist route" refers to a specific route suggested by the server to enable the user to efficiently tour tourist spots.

[1580] A "mission" refers to a specific task or activity that a user must accomplish at a tourist destination.

[1581] "Augmented reality technology" refers to a technology that overlays digital information on a device that displays the real world.

[1582] "Guide character" refers to a virtual character that provides sightseeing guidance to users using augmented reality technology.

[1583] "Message" refers to notifications and guidance information displayed to the user by the guide character or the system.

[1584] "Push notification" refers to a message sent in real time from a server to a user's information terminal.

[1585] "Trends" refers to information about current trends and popular spots.

[1586] "Continuous" means that the information is updated at regular intervals.

[1587] "Dynamic" refers to the way something changes in real time depending on the situation.

[1588] The present invention is a system that provides users with individually tailored tourist experiences at tourist destinations. This system uses generative AI models and augmented reality technology to generate personalized tourist routes and missions based on the user's preferences and past visit history, and provides them in real time. Specific embodiments of this system are described below.

[1589] 1. Collecting User Information

[1590] Users first find a display panel installed at a tourist spot and begin gathering information by scanning it with an information device (e.g., a smartphone). The user then launches a dedicated app on their smartphone and uses the camera to scan the QR code or AR marker on the display panel. The device then uses GPS to obtain the user's current location and sends it to the server along with the QR code information. The location information service uses the Google Maps API.

[1591] 2. Analyzing data and applying generative AI models

[1592] The server compares the user information sent from the device with a database (e.g., MySQL, MongoDB) to obtain the user's past visit history and preference data. The server then uses a generative AI model (e.g., GPT-4) to analyze this data and generate sightseeing routes and missions based on the user's interests. Examples of prompts are as follows:

[1593] "Spots visited in the past: Mount Takao, Interests: Historical landmarks. Based on this, please suggest the next tourist spot and mission."

[1594] The generative AI model takes into account past user data and current trends to create an optimal plan, and the generated sightseeing route and missions are sent from the server to the device.

[1595] 3. Delivering personalized AR content

[1596] The device analyzes the tourist routes and missions received from the server and displays them to the user as push notifications. The user then moves toward the designated tourist spots based on the notifications. Upon arriving at each spot, the device uses augmented reality technology (e.g., ARKit, ARCore) to display a personalized guide character and message on the display panel. For example, when arriving at a specific spot, a guide character will appear saying, "This is your next destination."

[1597] 4. Real-time feedback

[1598] As the user travels along a sightseeing route, the device periodically acquires its current location and sends it to the server. The server analyzes the user's progress in real time and dynamically generates next steps and recommended sightseeing spots. These results are then sent back to the device and displayed as notifications to the user. For example, a notification might say, "If you stop by a specific cafe on your way to the next spot, a hidden mission will occur."

[1599] Specific examples

[1600] For example, suppose a family visits a tourist spot and scans the display panel. The device sends the information to the server, which analyzes the family's past history and preferences. The generative AI model suggests "hands-on activities for parents and children" and "nature walks and picnic spots." The device notifies the family of the mission received, and a guide character uses augmented reality technology to provide guidance.

[1601] For solo travelers, when they visit a tourist spot, they scan the display panel, and the device sends the information to the server. The server takes into account their past visit history and interests, and the generative AI model suggests "tours of historical landmarks" and "hidden cafes." The device notifies the user of the received mission, and as they travel along the route selected by the user, a guide character will guide them to their next destination when they arrive at a specific spot.

[1602] This system allows users to have new sightseeing experiences and encourages them to revisit tourist destinations.

[1603] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1604] The flow of this system's program processing

[1605] Step 1: The user finds the display panel

[1606] Input: Display panels installed in various locations in tourist spots

[1607] Action: The user finds the display panel.

[1608] Output: Your smartphone is ready to get the information from the display panel.

[1609] Specific actions: The user walks around a tourist spot, searches for a display panel, and when they find one, takes out their smartphone.

[1610] Step 2: The user starts the device and opens the dedicated app.

[1611] Input: A smartphone owned by the user

[1612] Process: The user takes out their smartphone and launches the dedicated app.

[1613] Output: The dedicated app will launch and you will be ready to scan.

[1614] Specific operation: The user touches the smartphone screen, selects and opens the dedicated app.

[1615] Step 3: Scan the QR code or AR marker on the character panel with your device

[1616] Input: QR code or AR marker placed on the display panel

[1617] Processing: Activate the smartphone camera and scan the QR code or AR marker.

[1618] Output: QR code and AR marker information is obtained

[1619] Specific operation: The user points the smartphone camera at the display panel and scans the QR code or AR marker.

[1620] Step 4: The device acquires location information and sends it to the server

[1621] Input: Current location information of the device using GPS, as well as information on QR codes and AR markers

[1622] Processing: Send location information and scanned information to the server

[1623] Output: Location information and QR code information are sent to the server

[1624] Specific operation: The device uses location services (e.g., Google Maps API) to obtain the current location and sends the obtained information to the server via HTTPS protocol.

[1625] Step 5: The server checks the user information against the database

[1626] Input: Location information and QR code information sent from the device

[1627] Processing: The server checks the database to obtain the user's past visit history and preference data.

[1628] Output: User's past visit history and preference data

[1629] Specific operation: The server accesses a database (e.g., MySQL, MongoDB) and retrieves the information to be matched.

[1630] Step 6: The server generates tourist routes and missions using the generative AI model

[1631] Input: User's past visit history, preference data, current location information

[1632] Processing: Input prompts into the generative AI model to generate personalized sightseeing routes and missions.

[1633] Output: personalized tourist routes and missions

[1634] Specific operation: An example of a prompt sentence: "Spots visited in the past: Mt. Takao, Interests: Historical buildings. Based on this, please suggest the next tourist spot and mission." is input into a generative AI model (e.g., GPT-4) and the generated results are obtained.

[1635] Step 7: The server sends the generated results to the device.

[1636] Input: Tourist routes and missions generated by generative AI models

[1637] Process: Send the generated results to the terminal

[1638] Output: Tourist route and mission data sent to the device

[1639] Specific operation: The server sends the generated results to the terminal using the HTTPS protocol.

[1640] Step 8: The device analyzes the sightseeing route and mission and displays a notification

[1641] Input: Tourist route and mission data sent from the server

[1642] Processing: The device analyzes the data and displays a notification to the user

[1643] Output: The notification displayed to the user

[1644] Specific operation: The device analyzes the sightseeing route and mission data and displays a notification saying, "Your next destination is XX Park."

[1645] Step 9: User follows notification

[1646] Input: Device notifications

[1647] Action: The user follows the notification and moves to the designated tourist spot.

[1648] Output: The user arrives at the specified location.

[1649] Specific behavior: The user checks the smartphone screen and moves according to the displayed route.

[1650] Step 10: The device uses AR technology to display personalized content.

[1651] Input: User location information, AR content data

[1652] Processing: The device displays a guide character or message using augmented reality technology.

[1653] Output: Personalized AR content displayed to the user

[1654] Specific operation: When the user arrives at the designated spot, a guide character will be displayed using the device's camera, saying, "This is your next destination."

[1655] Step 11: The device periodically acquires location information and sends it to the server.

[1656] Input: current location of the user

[1657] Processing: The device periodically acquires location information using GPS and sends it to the server.

[1658] Output: Location information sent to the server

[1659] Specific operation: The device acquires its current location at regular intervals and sends that information to the server.

[1660] Step 12: The server analyzes the progress and generates the next step

[1661] Input: current location of the user

[1662] Processing: The server analyzes the progress and generates the next attractions and missions.

[1663] Output: Data on the next tourist spot or mission

[1664] Specific operation: The server uses the generative AI model to generate the next route and mission based on the user's current location.

[1665] Step 13: The server sends the latest instructions to the terminal and notifies the user.

[1666] Input: Data for the next tourist spot or mission

[1667] Processing: The server sends the latest instructions to the terminal and notifies the user.

[1668] Output: Notification displayed on the user's device

[1669] Specific operation: The server generates the next step and sends it to the device, which then displays a notification saying, "If you stop by a specific cafe on your way to the next spot, a hidden mission will occur."

[1670] (Application example 1)

[1671] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1672] There is a demand for tourist destinations and brick-and-mortar stores to provide users with personalized information based on their preferences, interests, and past visit history, thereby providing a more fulfilling experience. However, it is difficult for users to efficiently and intuitively obtain information and find optimal routes and recommended products without receiving a large amount of information all at once. Current systems do not adequately provide individually tailored information, making it difficult for users to obtain information that perfectly matches their interests. Therefore, the objective of this invention is to provide a system that can provide users with individually tailored sightseeing routes and shopping routes in real time.

[1673] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1674] In this invention, the server includes means for analyzing the user's interests, preferences, and past visit history based on the user's location information and QR code information, means for generating individually suited sightseeing routes and missions using a generative AI model, and means for displaying AR content based on AR markers detected by the device's camera, thereby enabling the provision of personalized sightseeing routes, shopping routes, and recommended products to the user.

[1675] "User" refers to an individual who uses this system to receive information at tourist spots and brick-and-mortar stores.

[1676] "Terminal" refers to a device carried by a user, such as a smartphone, smart glasses, or head-mounted display, that acquires QR code information and location information and displays AR content.

[1677] "Server" refers to the central processing unit that processes user information and generates sightseeing routes, missions, and shopping routes using generative AI models.

[1678] "Generative AI model" refers to an artificial intelligence model that analyzes a user's interests and preferences, past visit history, and current trends to generate individually appropriate tourist routes and missions.

[1679] "AR technology" refers to augmented reality technology, which displays digital content overlaid on real-world information.

[1680] A "tourist route" refers to the optimal order or route for a user to visit tourist spots.

[1681] A "mission" refers to a specific action or task presented to a user.

[1682] "QR code" refers to a two-dimensional barcode consisting of a rectangular pattern of dots, and is a format for encoding information.

[1683] "AR marker" refers to a printed or digital image that serves as a reference for displaying content using augmented reality technology.

[1684] "Location Information" means a user's current geographic location, which may be obtained using GPS or other location-determining technology.

[1685] "Progress" refers to information indicating how far a user has progressed along a given sightseeing or shopping route.

[1686] A "shopping route" refers to the optimal order or route for a user to efficiently shop in a physical store.

[1687] "Benefit information" refers to information such as discounts, campaigns, and bonuses offered to users.

[1688] This invention is a system for providing users with personalized sightseeing and shopping routes at tourist spots and brick-and-mortar stores. The system utilizes generative AI models and AR technology to provide individually tailored information based on the user's preferences, interests, and past visit history.

[1689] System configuration

[1690] The system mainly consists of the following components:

[1691] 1. User's device (smartphone, smart glasses, head-mounted display, etc.)

[1692] 2. Server

[1693] 3. Generative AI Models

[1694] 4. AR markers (including QR codes)

[1695] User's device

[1696] The user's device may be a smartphone, smart glasses, or a head-mounted display. These devices have a camera function and can scan AR markers and QR codes installed at tourist spots and in brick-and-mortar stores. When the device scans these markers, the acquired data along with location information is sent to a server. Furthermore, the device uses AR technology to display personalized guide characters and messages.

[1697] server

[1698] The server has the ability to analyze the user's interests, preferences, and past visit history based on the user's location information and QR code information. A generative AI model running on the server takes this data and current trends into account to generate optimal sightseeing and shopping routes. The generated routes and missions are then sent to the user's device in real time.

[1699] Generative AI Models

[1700] A generative AI model is an artificial intelligence model that analyzes a user's interests, preferences, and past visit history to personalize sightseeing or shopping routes. This model uses user data and current trends to generate an optimal plan and sends it to the user's device via a server. An example of a specific prompt is, "User 'User123' has the following past purchase history: chairs, tables, and shelves. Taking into account the latest trends, please recommend new furniture that may be of interest to them. Also, please generate the optimal shopping route within the store."

[1701] AR technology

[1702] AR technology is a technology that overlays digital content onto the real world through the user's device. When a user scans an AR marker in a tourist spot or physical store, personalized guide characters, product information, and messages are displayed on the device screen. This plays an important role in improving the user experience.

[1703] Specific examples

[1704] Example 1: Family sightseeing

[1705] A user (family) visits a tourist spot and scans a character panel. The device sends information to the server, which analyzes the family's past history and preferences. The generative AI model suggests "hands-on activities for parents and children to enjoy together" and "nature walks and picnic spots." The device notifies the family of the mission it received, and a guide character uses AR technology to provide guidance.

[1706] Example 2: Solo tourism

[1707] A user (solo traveler) visits a tourist spot and scans a character panel. The device sends the information to the server, which takes into account past visit history and interests. The generative AI model suggests "tours of historical landmarks" and "hidden cafes." As the user travels along the route selected, a guide character will guide them to their next destination when they arrive at a specific spot.

[1708] The components and operation procedures described above allow users to have a more fulfilling experience at tourist spots and brick-and-mortar stores.

[1709] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1710] Step 1:

[1711] The user uses a device to scan AR markers or QR codes installed at tourist spots or brick-and-mortar stores. The device's camera acquires the QR code or AR marker information, and collects location information along with this information. The input is "QR code information" and "location information." This data is then combined into a single communication packet for data processing. The output is "data that is a packet of the user's QR code information and location information."

[1712] Step 2:

[1713] The QR code information and location information collected by the device are sent to the server. The input is "data that is a packet of the user's QR code information and location information." The processing involves sending the data using an HTTP request. The output is "user data sent to the server."

[1714] Step 3:

[1715] The server analyzes the user's QR code information and location information received. The input is "user data sent to the server." Data calculations use database matching and a data analysis engine to obtain the user's past visit history and preference data. The output is "the analysis results of the user information."

[1716] Step 4:

[1717] The server uses the generative AI model to generate individually tailored sightseeing and shopping routes based on the user's past visit history and preference data. The input is the "analysis results of user information." The data is passed to the generative AI model for data processing and calculation, generating multiple personalized options. As a concrete example, the prompt statement used is "Please recommend tourist spots of interest to user 'User123' based on his past visit history." The output is the "generated sightseeing and shopping routes."

[1718] Step 5:

[1719] The server sends the generated sightseeing route or shopping route to the user's device. The input is the "generated sightseeing route or shopping route." The processing involves sending the data using an HTTP response. The output is the "data sent to the user's device."

[1720] Step 6:

[1721] The device analyzes the tourist route and shopping route data received and displays a notification to the user. The input is "data sent to the user's device." Data processing involves reformatting the data into a notification format. The output is "a notification message displayed to the user."

[1722] Step 7:

[1723] When the user arrives at the designated tourist spot or inside the store, the device's camera scans the AR marker again and displays the AR content. The input is "AR marker information." The data is processed by selecting and displaying the appropriate AR content based on the marker information. The output is "AR content displayed on the device."

[1724] Step 8:

[1725] The device periodically acquires the current location and sends it to the server, allowing the user's progress to be analyzed in real time. The input is "current location data." The data is analyzed in real time as part of the process, and information about the next tourist spot and special offers is dynamically generated. The output is "next step and special offer information tailored to the user's progress."

[1726] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1727] This invention is a system that combines generative AI models, AR technology, and an emotion engine to provide users with individually tailored sightseeing tours. The system generates personalized sightseeing routes and missions based on the user's preferences and past visit history, and provides them in real time. It also has the ability to recognize the user's emotional state and dynamically adjust the sightseeing experience based on that.

[1728] System configuration

[1729] The system mainly consists of the following components:

[1730] 1. User's device (smartphone, etc.)

[1731] 2. Server

[1732] 3. Generative AI Models

[1733] 4. Character portrait panels as AR markers

[1734] 5. Emotion Engine

[1735] Explanation of program processing

[1736] 1. Collecting User Information

[1737] The user discovers a standing image panel installed at a tourist spot and launches a dedicated app on their device.

[1738] The device's camera scans the QR code or AR marker on the character portrait panel to obtain location information.

[1739] The device sends the QR code information and location information to the server.

[1740] 2. Analyzing data and applying generative AI models

[1741] The server compares the received user information with a database to obtain the user's past visit history and preference data.

[1742] The server uses a generative AI model to analyze this data and generate sightseeing routes and missions based on the user's interests.

[1743] Generative AI models take into account past data and current trends to create optimal plans.

[1744] The server formats the generated tourist routes and missions and sends them to the terminal.

[1745] 3. Delivering personalized AR content

[1746] The terminal displays the sightseeing route and mission received from the server and notifies the user.

[1747] The user confirms the notification and starts the tour.

[1748] When the device reaches a designated tourist spot, AR technology is used to display a guide character or message from a portrait panel.

[1749] 4. Applying the Emotion Engine

[1750] The device analyzes the user's facial expressions and voice using an emotion engine to recognize their emotional state in real time.

[1751] The device transmits the emotion data to the server.

[1752] The server dynamically adjusts sightseeing routes and missions based on emotion data, and suggests next spots and new activities.

[1753] 5. Real-time feedback

[1754] The device periodically acquires its current location and transmits it to the server.

[1755] The server analyzes the user's progress and emotional state in real time and dynamically generates next steps and recommended tourist spots.

[1756] The server sends the latest instructions and information to the terminal and notifies the user.

[1757] Specific examples

[1758] Example 1: Family sightseeing

[1759] A user (family) visits a tourist spot and scans a portrait panel.

[1760] The device sends information to a server, which then analyzes the family's past history and preferences.

[1761] The generative AI model suggests "experiential activities for parents and children to enjoy together" and "nature walks and picnic spots."

[1762] The device will notify the family of the mission it has received, and a guide character will provide guidance using AR technology.

[1763] The emotion engine recognizes a child's state of excitement and dynamically suggests relaxing spots.

[1764] Example 2: Solo tourism

[1765] The user (traveling alone) visits a tourist spot and scans the portrait panel.

[1766] The device sends the information to the server, which takes into account past visits and interests.

[1767] The generative AI model suggests "tours of historical sites" and "hidden cafes."

[1768] The terminal notifies the received mission and moves along the route selected by the user.

[1769] When you arrive at a specific spot, a guide character will guide you to your next destination.

[1770] The emotion engine recognizes the user's level of satisfaction from their facial expressions and suggests more stimulating spots.

[1771] This concludes the detailed description of the embodiment of the present invention. This system allows users to have new sightseeing experiences and encourages repeat visits to tourist spots. Furthermore, the introduction of an emotion engine makes it possible to provide experiences based on the user's emotions, realizing more personalized services.

[1772] The processing flow will be explained below.

[1773] Step 1:

[1774] The user discovers a standing picture panel installed at a tourist spot.

[1775] The user launches the dedicated app on their device (smartphone, etc.) and points the camera at the character portrait panel.

[1776] Step 2:

[1777] The device uses the camera to scan the QR code or AR marker on the character portrait panel.

[1778] The device acquires location information and sends the QR code information and location information to the server.

[1779] Step 3:

[1780] Based on the user information received by the server, the user's past visit history and preference data are obtained from the database.

[1781] The server analyzes this data and inputs it into a generative AI model.

[1782] Step 4:

[1783] The generative AI model takes into account the analysis results and current trends to generate optimal sightseeing routes and missions for users.

[1784] The server formats the generated tourist routes and missions and sends them to the terminal.

[1785] Step 5:

[1786] The terminal displays the sightseeing route and mission received from the server and notifies the user.

[1787] The user confirms the notification and starts the tour.

[1788] Step 6:

[1789] When the device reaches a designated tourist spot, AR technology is used to display a guide character or message from a portrait panel.

[1790] A guide character will guide you to your next steps and tourist spots.

[1791] Step 7:

[1792] The device analyzes the user's facial expressions and voice data using an emotion engine to recognize their emotional state in real time.

[1793] The device transmits the emotion data to the server.

[1794] Step 8:

[1795] The server dynamically adjusts sightseeing routes and missions based on emotion data, and suggests next spots and new activities.

[1796] Example: The server detects the user's fatigue and suggests a spot where they can relax.

[1797] Step 9:

[1798] The terminal periodically acquires the user's current location and transmits it to the server.

[1799] The server analyzes the user's progress and emotional state in real time and dynamically generates next steps and recommended tourist spots.

[1800] Step 10:

[1801] The server sends the latest instructions and information to the terminal and notifies the user.

[1802] Example: New missions and recommended spots will be announced via push notifications.

[1803] Step 11:

[1804] The user checks the notification from the device and moves to the next spot.

[1805] The device continuously captures the user's location and progress and sends feedback to the server.

[1806] Step 12:

[1807] The server dynamically updates next steps and new tourist spots based on feedback.

[1808] For example: Suggest a special activity at a cafe on the way to your next destination.

[1809] The above is the specific program processing flow of the system that combines the emotion engine. This system allows users to have a sightseeing experience that is tailored to their individual needs, and can encourage them to revisit tourist spots.

[1810] Example 2

[1811] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1812] In recent years, there has been a growing demand for personalized tourism experiences tailored to the preferences and interests of each individual tourist. However, conventional tourist information systems have not been able to adequately address the individual needs of tourists. As a result, it has been difficult to provide tourists with a more fulfilling tourism experience and maximize the appeal of tourist destinations. Furthermore, there has been no system that can recognize the emotional state of users in real time while sightseeing and dynamically adjust sightseeing routes and missions accordingly.

[1813] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1814] In this invention, the server includes a means for a user to scan a character image panel installed at a tourist spot with a mobile communication terminal, a means for the terminal to transmit the user's location information and QR code information to a data center, a means for the data center to analyze the user's interests, preferences, and past visit history based on this information and generate individually suited tourist routes and missions using a generative AI model, a means for the data center to transmit the generated tourist routes and missions to the terminal, and a means for the terminal to display personalized guide characters and messages using augmented reality technology. This allows users to not only experience optimal tourist routes and missions based on their preferences and past visit history, but also to receive services tailored to their emotional state in real time.

[1815] A "user" is an individual who uses this system to visit a tourist spot.

[1816] A "mobile communication terminal" is a device such as a smartphone or tablet that is carried by a user.

[1817] "Standing picture panels" are panels containing QR codes and AR markers that are installed at tourist spots.

[1818] "Augmented reality technology" is a technology that displays virtual information overlaid on real-world images.

[1819] A "data center" is a collection of servers that receives information sent by users and performs analysis and generation processing.

[1820] A "QR code" is a two-dimensional barcode that stores information and can be read by scanning it.

[1821] "Location information" is geographical data that indicates the user's current location.

[1822] A "generative AI model" is an algorithm that analyzes a user's past data and trends to generate optimal sightseeing routes and missions.

[1823] A "tourist route" is a plan showing the route a user will take at a tourist spot.

[1824] A "mission" is a specific task or activity that a user must accomplish while sightseeing.

[1825] A "guide character" is a virtual guide displayed through augmented reality technology.

[1826] The "emotional state" is the psychological state recognized from the user's facial expression and voice.

[1827] "Push notification" is a notification function that delivers information to users in real time.

[1828] The present invention is a system that provides users with personalized sightseeing experiences. This system generates sightseeing routes and missions based on the user's preferences and past visit history, and provides them in real time. It also has the function of recognizing the user's emotional state and dynamically adjusting the sightseeing experience based on that. Specific embodiments of this system are described below.

[1829] Hardware and Software Configuration

[1830] In order to implement the present invention, the following hardware and software are used.

[1831] 1. User's device (mobile communication device)

[1832] Smartphones, tablets, etc.

[1833] 2. Server

[1834] It is equipped with a database of user information, a generative AI model, and an emotion engine.

[1835] 3. Generative AI Models

[1836] An algorithm that analyzes user preferences, past visit history, current trends, etc. to generate optimal tourist routes and missions.

[1837] 4. Augmented Reality Technology (AR Technology)

[1838] Used on the user's device, it displays virtual guide characters and messages.

[1839] 5. Data Center

[1840] It functions as a cloud server or on-premise server and processes user location information, QR code information and emotion data.

[1841] Example of system operation

[1842] Example 1: Family sightseeing

[1843] A user (family) visits a tourist spot and scans a portrait panel.

[1844] The device sends information to a data center, which then analyzes the family's past history and preferences.

[1845] The generative AI model suggests "experiential activities for parents and children to enjoy together" and "nature walks and picnic spots."

[1846] The device will notify the family of the mission it has received, and a guide character will provide guidance using AR technology.

[1847] The emotion engine recognizes a child's state of excitement and dynamically suggests relaxing spots.

[1848] Example 2: Solo tourism

[1849] The user (traveling alone) visits a tourist spot and scans the portrait panel.

[1850] The device sends the information to a data center, which takes into account past visits and interests.

[1851] The generative AI model suggests "tours of historical sites" and "hidden cafes."

[1852] The terminal notifies the received mission and moves along the route selected by the user.

[1853] When you arrive at a specific spot, a guide character will guide you to your next destination.

[1854] The emotion engine recognizes the user's level of satisfaction from their facial expressions and suggests more stimulating spots.

[1855] Prompt Sentence Examples

[1856] "Please explain in detail the process of scanning the portrait panels, sending the obtained user information to the data center, and having the AI ​​model generate the optimal tourist route."

[1857] "Please explain a specific scenario where a family visits a tourist destination and an emotion engine is used to tailor their experience in real time."

[1858] "Please explain how a generative AI model can provide a personalized sightseeing route for a solo traveler."

[1859] The embodiments of the present invention allow users to have new sightseeing experiences and encourage them to revisit tourist destinations. Furthermore, the introduction of an emotion engine allows for experiences based on the user's emotions, enabling more personalized services.

[1860] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1861] Step 1:

[1862] The user scans a portrait panel at a tourist spot using their smartphone camera.

[1863] Specific operation: The user holds the device's camera over the QR code or AR marker and taps "Scan" in the dedicated app.

[1864] Input: QR code and AR marker information on the character portrait panel displayed on the user's smartphone.

[1865] Output: QR code information and location information are obtained.

[1866] Step 2:

[1867] The device sends the acquired QR code information and location information to the data center.

[1868] Specific operation: The device uploads the QR code information and GPS location information to the server via a dedicated app.

[1869] Input: QR code information and GPS location information.

[1870] Output: QR code information and location information are sent to the data center.

[1871] Step 3:

[1872] Based on the information received by the server, the user's preferences and past visit history are retrieved from the database and analyzed using a generative AI model.

[1873] Specific operation: The server queries the database to obtain user information, then inputs that data into the generative AI model to obtain analysis results.

[1874] Input: QR code information, location information, user preference data, past visit history data.

[1875] Output: Suggestions for optimal sightseeing routes and missions for the user.

[1876] Step 4:

[1877] The server formalizes the tourist routes and missions generated and sends them to the device via a dedicated app.

[1878] Specific operation: The server converts the tourist route and missions into a format such as JSON and sends them to the terminal.

[1879] Input: The output of a generative AI model.

[1880] Output: Tourist route and mission data is sent to the terminal.

[1881] Step 5:

[1882] The device notifies the user of the tourist route and missions received from the server, and displays guide characters and messages using AR technology.

[1883] Specific operation: The device will display "New tourist route has arrived" in the notification bar and switch to AR mode to display the virtual character.

[1884] Input: Received tourist route and mission data.

[1885] Output: Notifications and AR guide information.

[1886] Step 6:

[1887] The device periodically obtains the user's current location and sends it to the data center.

[1888] Specific operation: Obtain location information using the device's GPS function and periodically upload it to the server.

[1889] Input: Current location.

[1890] Output: The location information is sent to the server.

[1891] Step 7:

[1892] The device analyzes the user's facial expressions and voice using an emotion engine to recognize their emotional state in real time.

[1893] Specific operation: The device's camera and microphone capture the user's facial expressions and voice, and the emotion engine provides the analysis results.

[1894] Input: User's facial and voice data.

[1895] Output: Recognized emotional state data.

[1896] Step 8:

[1897] The server dynamically adjusts sightseeing routes and missions based on emotion data and current location information, suggesting new activities and next spots.

[1898] Specific operation: The server analyzes the emotion data and location information, generates a new sightseeing route, and sends it to the device.

[1899] Input: Emotional state data and location information.

[1900] Output: Dynamically adjusted tourist routes and new mission data.

[1901] Step 9:

[1902] The terminal notifies the user of the latest instructions and information received from the server.

[1903] Specific operation: The device will display "Please move to the next spot" in the notification bar and provide information about new tourist spots.

[1904] Input: The latest instructions and information from the server.

[1905] Output: Notify the user and display tourist spot information.

[1906] (Application example 2)

[1907] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1908] Conventional tourist information systems have struggled to provide personalized sightseeing routes and missions that fully reflect a user's individual preferences and interests. They also struggled to recognize a user's emotional state in real time and dynamically adjust the sightseeing experience based on that information. This resulted in users being unable to obtain a fully satisfying sightseeing experience, and they were unable to encourage repeat visits to tourist destinations.

[1909] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to scan a portrait panel installed at a tourist attraction with an electronics device; a means for the electronics device to transmit the user's location information and QR code information to a data center; a means for the data center to analyze the user's interests, preferences, and past visit history based on this information and generate individually tailored tourist routes and missions using a generative AI model; a means for the data center to transmit the generated tourist routes and missions to the electronics device; a means for the electronics device to display personalized virtual guide characters and messages using augmented reality technology; a means for the electronics device to analyze the user's facial expressions and voice and recognize the user's emotional state in real time; and a means for the data center to dynamically adjust the tourist routes and missions based on the emotion analysis data. This makes it possible to provide a more satisfying tourist experience tailored to the user's individual preferences and emotional state.

[1910] "User" refers to a person who uses an electronic device to scan the portrait panels installed at tourist spots and experience the tourist routes and missions provided.

[1911] A "tourist destination" refers to an area or place that contains attractions and sights that are of interest to visitors.

[1912] "Standing picture panels" are displays installed at tourist spots that users can scan with their electronic device's camera to read QR codes and AR markers.

[1913] "Electronic devices" refers to electronic devices that users carry and use, such as smartphones and smart glasses.

[1914] "Location information" is data indicating the user's current location within a tourist attraction.

[1915] A "QR code" is a two-dimensional barcode that can be read by an electronic device to obtain the embedded information.

[1916] "Data Center" refers to a service provider that collects and analyzes data and information sent by users and generates personalized tourist routes and missions using generative AI models.

[1917] "Interests and preferences" refers to the interests and preferences that a user has shown in the past.

[1918] "Past visit history" refers to a record of places a user has visited and activities they have participated in in the past.

[1919] A "generative AI model" refers to an artificial intelligence algorithm that generates optimal tourist routes and missions based on a user's preferences and past visit history.

[1920] A "tourist route" refers to the optimal route or directions that a user should follow when sightseeing.

[1921] "Mission" refers to a specific activity or experience offered to a user.

[1922] "Augmented reality technology" is a technology that displays virtual information overlaid on the real world, allowing users to experience it through electronic devices.

[1923] "Virtual guide character" refers to a virtual character that uses augmented reality technology to guide visitors through sightseeing routes and missions.

[1924] "Message" refers to textual or visual information displayed to a user through an electronic device.

[1925] "Facial expression" refers to the expression of emotion shown by the movement of the user's facial muscles.

[1926] "Voice" refers to the tone and manner of speaking of the user.

[1927] "Emotional state" refers to the user's current state of mind and is recognized in real time using an emotion engine.

[1928] "Dynamic adjustment" refers to adapting sightseeing routes and missions on the fly based on the user's real-time emotional state and progress.

[1929] A system embodying this invention provides a personalized sightseeing experience to users using an electronic device when visiting a tourist attraction. The system begins when the user scans a portrait panel installed at the tourist attraction with the electronic device. The electronic device then transmits the user's location information and QR code information to a data center.

[1930] The data center analyzes the user's interests, preferences, and past visit history based on the received information, and uses a generative AI model to generate individually tailored sightseeing routes and missions, which are then sent from the data center to the electronics device.

[1931] The electronics device uses augmented reality technology to display personalized virtual guide characters and messages to the user. As the user moves along the sightseeing route, the electronics device analyzes the user's facial expressions and voice to recognize their current emotional state in real time. The analysis results of the emotional state are sent to a data center, which then dynamically adjusts the sightseeing route and missions based on the emotion analysis data.

[1932] The specific hardware and software used include smartphones and smart glasses as electronic devices, equipped with cameras and AR functions. For emotion analysis, OpenCV and Keras are used, for example. The data center is based on cloud services and servers, and databases and generative AI models are used to store and analyze user data.

[1933] For example, when a user scans a portrait panel installed at a tourist spot with their smartphone and the information is sent to the data center, the data center uses a generative AI model to generate a sightseeing route that matches the user's preferences. For example, for a user with their family, it can suggest "hands-on activities to enjoy together as a family" and "nature walks and picnic spots." It can also dynamically suggest relaxing spots if it detects excitement in the user's facial expression.

[1934] An example prompt is:

[1935] User's past purchase history:

[1936] Product ID: 123, Category: Clothing, Purchase Date: 2023-09-01

[1937] Product ID: 456, Category: Gadgets, Purchase Date: 2023-09-05

[1938] User's current emotional state: Positive

[1939] Based on this information, generate three recommended items for this user.

[1940] In this way, the present invention can provide users with a personalized sightseeing experience in real time, thereby improving user satisfaction and encouraging repeat visits to tourist destinations.

[1941] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1942] Step 1:

[1943] A user scans a standing picture panel installed at a tourist spot with an electronic device. The input is the QR code read by the user's electronic device and location information, and the output is a data packet containing this information. The user scans the standing picture panel with a camera and obtains their current location information along with the QR code.

[1944] Step 2:

[1945] The electronics device transmits the QR code information and location information to the data center. The input is a data packet of the QR code and location information read by the electronics device, and the output is the information transmitted to the data center. The terminal transmits the QR code and location information to the data center via the Internet.

[1946] Step 3:

[1947] The data center analyzes the user's interests, preferences, and past visit history based on the information received. The input is the user's past behavioral data stored along with QR code information and location information, and the output is analyzed user profile data. The server extracts past user data from the database and uses a generative AI model to identify the user's interests and preferences.

[1948] Step 4:

[1949] The data center uses a generative AI model to generate individually tailored sightseeing routes and missions. The input is user profile data and information on current tourist destinations, and the output is personalized sightseeing routes and missions. The generative AI model generates optimal sightseeing routes and missions based on the user's preferences and past history.

[1950] Step 5:

[1951] The data center sends the generated sightseeing routes and missions to the electronics device. The input is the personalized sightseeing route and mission data, and the output is the sightseeing route and mission data sent to the electronics device. The server formats the sightseeing route and mission and sends it to the electronics device.

[1952] Step 6:

[1953] The electronics device uses augmented reality technology to display personalized virtual guide characters and messages. The input is tourist route and mission data received from a data center, and the output is the guide character and message displayed on the user's screen. The device uses its camera and AR function to overlay the guide character and message on the real space.

[1954] Step 7:

[1955] An electronic device analyzes a user's facial expressions and voice to recognize their emotional state in real time. The input is the user's facial image and voice data, and the output is analyzed emotional state data. The device uses a camera and microphone to capture the user's facial expressions and voice, which are then analyzed by an emotion engine.

[1956] Step 8:

[1957] The electronic device transmits emotional state data to the data center, the input is the emotional state data, and the output is the emotional state data transmitted to the data center. The terminal transmits the emotional state to the data center in real time.

[1958] Step 9:

[1959] The data center dynamically adjusts tourist routes and missions based on emotional state data. The input is emotional state data transmitted in real time, and the output is adjusted tourist route and mission data. The server analyzes the emotional data and optimizes tourist routes and missions according to the situation.

[1960] Step 10:

[1961] The data center sends the adjusted sightseeing route and mission data to the electronics device. The input is the adjusted sightseeing route and mission data, and the output is the data sent to the electronics device. The server sends the new plan to the terminal and notifies the user.

[1962] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1963] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1964] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1965] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1966] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1967] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1968] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1969] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1970] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1971] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1972] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1973] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1974] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1975] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1976] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1977] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1978] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1979] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1980] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1981] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1982] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1983] The following is further disclosed regarding the above embodiment.

[1984] (Claim 1)

[1985] A means for a user to scan a portrait panel installed at a tourist spot with a device such as a smartphone;

[1986] A means for the terminal to transmit user location information and QR code information to a server;

[1987] The server analyzes the user's interests, preferences, and past visit history based on this information, and generates individually appropriate sightseeing routes and missions using a generative AI model.

[1988] A means for transmitting the tourist routes and missions generated by the server to the terminal;

[1989] A system including a means for a terminal to display personalized guide characters and messages using AR technology.

[1990] (Claim 2)

[1991] The device sends the user's current location to the server in real time,

[1992] The server analyzes the user's progress in real time and dynamically generates the next tourist spot or mission.

[1993] 10. The system of claim 1.

[1994] (Claim 3)

[1995] The generative AI model generates sightseeing routes and missions based on past user data and current trends.

[1996] This includes providing users with push notifications to guide them to next steps and recommended tourist spots.

[1997] 10. The system of claim 1.

[1998] "Example 1"

[1999] (Claim 1)

[2000] A means for a user to read a display panel installed at a tourist spot using an information terminal;

[2001] A means for the terminal to transmit user location information and identification information to a server;

[2002] A means for analyzing the user's preferences and past history based on the information received by the server, and generating personalized sightseeing routes and missions using a generative AI model;

[2003] A means for transmitting the tourist routes and missions generated by the server to the terminal;

[2004] A system including a means for a terminal to display personalized guide characters and messages using augmented reality technology.

[2005] (Claim 2)

[2006] The device continuously transmits the user's current location to the server,

[2007] The server dynamically analyzes the user's progress and generates the next tourist spot or mission.

[2008] 10. The system of claim 1.

[2009] (Claim 3)

[2010] The generative AI model generates sightseeing routes and missions by taki...

Claims

1. A means for a user to scan a portrait panel installed at a tourist spot with a device such as a smartphone; A means for the terminal to transmit user location information and QR code information to a server; The server analyzes the user's interests, preferences, and past visit history based on this information, and generates individually appropriate sightseeing routes and missions using a generative AI model. A means for transmitting the tourist routes and missions generated by the server to the terminal; A system including a means for a terminal to display personalized guide characters and messages using AR technology.

2. The device sends the user's current location to the server in real time, The server analyzes the user's progress in real time and dynamically generates the next tourist spot or mission. The system of claim 1 .

3. The generative AI model generates sightseeing routes and missions based on past user data and current trends. This includes providing users with push notifications to guide them to next steps and recommended tourist spots. The system of claim 1 .

Images