system

Abstract

We provide the system. [Solution] A means for inputting user preferences, Means for collecting users' movement history, A means of collecting and analyzing local spot data, A means of generating routes based on user preferences and spot data, A means of sending the generated route to the terminal, A means by which the terminal displays and guides the route, A system that includes means for collecting user feedback and incorporating it into future route suggestions.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional navigation systems tend to prioritize the shortest route to the destination, and have the problem that daily travel is monotonous and there are few new discoveries and pleasures. As a result, users have low satisfaction with the travel itself, and the opportunity to fully utilize the charm of the region is decreasing. Furthermore, since it is impossible to provide a travel experience tailored to the individual preferences of users, it is difficult to meet the needs of users who seek a personalized experience. It is an object to solve the above problems and transform daily travel into a rich experience.

Means for Solving the Problems

[0005] The present invention provides a system comprising means for inputting user preferences, means for collecting user travel history, means for collecting and analyzing local spot data, means for generating a route based on user preferences and spot data, means for transmitting the generated route to a terminal, means for the terminal to display and guide the user along the route, and means for collecting user feedback and reflecting it in future route suggestions.

[0006] Specifically, natural language processing and machine learning are used to collect and classify local spot data, which is then matched with user preference data. A route is then generated to provide the user with the optimal entertainment experience, and voice guidance is provided via a navigation device based on that route. This allows users to discover new things and enjoy themselves, and to enjoy a fulfilling travel experience that leverages the charm of the region.

[0007] "User preferences" refer to information about the specific scenery, places, foods, hobbies, and other tastes that users are interested in.

[0008] "User travel history" refers to historical data such as the places a user has traveled to in the past, the routes they have taken, and the time it has taken to travel.

[0009] "Local spot data" refers to information about places within a region, such as parks, restaurants, and tourist attractions.

[0010] "Collection methods" refer to the means of incorporating user preferences, travel history, and local spot data into the system.

[0011] "Analysis methods" refer to the means used to analyze collected data and derive specific behavioral patterns and preferences of users.

[0012] A "route generation method" is a means for calculating and suggesting the optimal travel route based on user preferences and spot data.

[0013] "Transmission means" refers to the means for sending the generated route to the user's terminal.

[0014] A "device" refers to a device owned by a user, such as a smartphone, tablet, or navigation device.

[0015] "Display and guidance means" refers to means for visually displaying the route generated on the terminal and providing guidance via voice or text.

[0016] A "feedback method" is a means of collecting evaluations and opinions about users' travel experiences and incorporating them into future route suggestions.

[0017] "Natural language processing" is a technology for analyzing text data, understanding its meaning, and processing it.

[0018] "Machine learning" is a technique that uses large amounts of data to build models and make predictions and decisions based on new data.

[0019] Navigation is the process of guiding someone from their current location to their destination.

[0020] "Voice guidance" refers to a function that conveys route information to the user through sound in addition to visual means. [Brief explanation of the drawing]

[0021] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0022] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0025] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0026] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0027] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0029] [First Embodiment]

[0030] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0031] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0034] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0037] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0041] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0042] This invention relates to a navigation system that suggests routes with scenic spots and good restaurants based on the user's preferences and travel history, in order to make everyday travel a more fulfilling experience. The form of this system is described below.

[0043] 1. Overview of the entire system

[0044] This navigation system mainly includes the following components:

[0045] 1. A means for users to input their preferences.

[0046] 2. Means for collecting user movement history

[0047] 3. Means for collecting and analyzing local spot data

[0048] 4. Means for generating routes based on user preferences and spot data

[0049] 5. Means for sending the generated route to the terminal

[0050] 6. Means by which the terminal displays and guides the route.

[0051] 7. A means of collecting user feedback and incorporating it into future route suggestions.

[0052] 2. Collection and analysis of user data

[0053] The server inputs user preferences (e.g., loves nature, wants to discover new restaurants) through initial user setup and periodic interactions, and stores this information in a database. This data is frequently updated to reflect the user's latest preferences. The server also periodically collects and stores the user's travel history to analyze past travel patterns.

[0054] 3. Collection and analysis of local spot data

[0055] The server collects local spot data from the internet and various databases. During this process, natural language processing technology is used to analyze the collected text data (reviews, ratings, etc.) and classify the characteristics and ratings of each spot. Furthermore, machine learning algorithms are used to identify spots that match the user's preferences.

[0056] 4. Route generation and sending

[0057] Based on the collected and analyzed data, the server generates an optimal route tailored to the user's preferences, taking into account the user's current location and destination. The generated route is designed to pass through scenic spots and highly-rated restaurants. This route information is then transmitted to the user's device.

[0058] 5. Route display and guidance

[0059] The device displays the route received from the server on a map application and provides visual and audio guidance to the user. For example, it can provide specific guidance such as, "There is a beautiful lake on your right 100 meters ahead, so please be sure to stop by." It can also suggest a new route in real time if the user needs to change their route while traveling.

[0060] 6. Gathering feedback and incorporating it into future proposals.

[0061] After completing a trip, users provide feedback within the app, including ratings and comments on the provided route. The device sends this feedback data to a server, which then updates the user's preference database based on this data. This updated data is then reflected in future route suggestions, providing a more personalized experience.

[0062] Specific example

[0063] Use Case 1: For users who want to enjoy nature

[0064] 1. The user enters their preference for nature into the app.

[0065] 2. Based on these preferences and past travel history, the server generates a route that includes parks and natural tourist attractions.

[0066] 3. The server sends the route it generated to the terminal.

[0067] 4. The device provides route information to the user via map and voice guidance.

[0068] 5. After the user has traveled, they provide feedback on the route through the app.

[0069] 6. The server collects feedback and incorporates it into the next route suggestion.

[0070] Use Case 2: For users who love food

[0071] 1. The user enters their preference for "discovering new restaurants" into the app.

[0072] 2. Based on these preferences, the server generates a route that includes new, highly-rated restaurants.

[0073] 3. The server sends the route it generated to the terminal.

[0074] 4. The device provides route information to the user via map and voice guidance.

[0075] 5. After the user has traveled to the restaurant, they provide feedback on the restaurant's rating through the app.

[0076] 6. The server collects feedback and incorporates it into the next route suggestion.

[0077] Thus, the navigation system of the present invention is designed to transform the user's daily travel into an opportunity for new discoveries and enjoyment. By providing a route optimized based on the user's preferences, the journey itself becomes an enjoyable experience.

[0078] The following describes the processing flow.

[0079] Step 1:

[0080] The user installs the app and completes the initial setup. Here, they enter their preferences (e.g., they like nature, they want to discover new restaurants, etc.).

[0081] Step 2:

[0082] The device collects user preference information and sends it to the server. Examples of data sent include the user ID and preferences (e.g., "User ID: 12345, Preferences: Nature, Gourmet").

[0083] Step 3:

[0084] The server stores user preference data in a database. This data forms the basis for future route suggestions.

[0085] Step 4:

[0086] The device periodically sends the user's current location and movement history to the server. Examples of data sent include location information and time information (e.g., "User ID: 12345, Location: (35.6895, 139.6917), Time: 2023-10-01 08:00").

[0087] Step 5:

[0088] The server collects local spot data from the internet and various databases. This data includes information on parks, restaurants, tourist attractions, and more.

[0089] Step 6:

[0090] The server analyzes the collected regional spot data using natural language processing and machine learning techniques. This allows for the classification of the characteristics and evaluation of each spot.

[0091] Step 7:

[0092] The server matches user preference data with local spot data to generate the optimal route for the user. For example, a user who likes nature will be given a route that passes through nearby parks and natural tourist spots.

[0093] Step 8:

[0094] The server sends the generated route to the terminal. An example of the data sent is the route details (e.g., "Route: Starting point (35.6895, 139.6917) → Park (35.6850, 139.6920) → Destination (35.7000, 139.7000)").

[0095] Step 9:

[0096] The device displays route information on a map and provides visual and audio guidance to the user. For example, it might say, "There's a beautiful lake 100 meters ahead, so please stop by."

[0097] Step 10:

[0098] The user confirms the displayed route and begins moving according to the instructions. During the journey, the device will notify the user in real time if any changes or corrections to the route are necessary.

[0099] Step 11:

[0100] After users have traveled, they can rate and provide feedback on the route within the app. For example, they can enter comments such as, "This route was great," or "I loved the new park."

[0101] Step 12:

[0102] The device sends user feedback data to the server. Examples of data sent include user ID, rating, and comment (e.g., "User ID: 12345, Rating: 5, Comment: The new park was great").

[0103] Step 13:

[0104] The server analyzes feedback data, updates user preference data, and incorporates it into future route suggestions. This improves the accuracy of the algorithm, enabling more personalized route suggestions.

[0105] Through these steps, this system can transform users' daily commutes into opportunities for new discoveries and enjoyment. Based on user preferences, it always provides the optimal route, changing travel from a mere means to an end in itself.

[0106] (Example 1)

[0107] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0108] Existing navigation systems often simply guide users to the shortest or fastest routes without adequately considering their diverse preferences or travel history. As a result, the journey itself may not be a satisfying experience for users, and may lack a sense of fulfillment. Furthermore, real-time route changes are difficult, making it difficult to respond to unexpected situations that occur during travel. In addition, there is a lack of mechanisms to incorporate user feedback into future route suggestions, making it difficult to provide routes optimized for individual users.

[0109] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0110] In this invention, the server includes means for inputting user preferences, means for collecting user travel history, means for collecting and analyzing local spot information, means for generating a route based on user preferences and spot information, means for transmitting the generated route to the terminal, and means for suggesting a new route in real time if a route change is necessary during travel. This enables individually optimized route guidance that takes into account the diverse preferences and travel history of the user. Furthermore, it can respond quickly to unexpected situations during travel, providing users with a more fulfilling travel experience.

[0111] "User preferences" refer to themes and activities that users are particularly interested in, such as appreciating the natural environment or discovering new restaurants.

[0112] "Travel history" refers to a record of routes a user has traveled and places they have visited in the past.

[0113] "Local spot information" refers to data on the evaluation and characteristics of tourist attractions, restaurants, parks, and other places located within a specific region.

[0114] "Natural language processing" is a technology that enables computers to understand and analyze human language.

[0115] "Machine learning" is an algorithm that allows computer systems to learn from data and automatically discover patterns and rules.

[0116] "Route generation" is the process of calculating and setting the optimal route based on the user's current location, destination, preferences, and point-of-interest information.

[0117] "Real-time route suggestions" refers to providing users with instant access to updated route guidance based on new information and circumstances while they are on the move.

[0118] "Feedback" refers to users providing opinions and evaluations regarding the services and routes they have experienced.

[0119] "Terminal" refers to an electronic device used to display navigation information and provide voice guidance, such as a smartphone or tablet.

[0120] This invention relates to a navigation system that suggests routes with scenic spots and good restaurants based on the user's preferences and travel history, in order to make everyday travel a more fulfilling experience. The form of this system is described below.

[0121] 1. System Configuration

[0122] This navigation system includes the following hardware and software configuration:

[0123] Servers: Cloud servers are used for data processing and storage. Specifically, Amazon Web Services (AWS®) and Google Cloud Platform can be used.

[0124] Device: A mobile device such as a smartphone or tablet will be used as the device for user operation. These devices must have GPS functionality and an internet connection.

[0125] Software: This involves using application software for inputting user preferences and travel history, and for navigation. Specifically, this includes mobile apps for iOS and Android®.

[0126] 2. Collection and Analysis of User Data

[0127] The server inputs user preferences (e.g., loves nature, wants to discover new restaurants) through initial user setup and regular interactions, and stores this information in a database. This data is frequently updated to reflect the user's latest preferences. The server also periodically collects and stores the user's travel history to analyze past travel patterns.

[0128] 3. Collection and analysis of local spot data

[0129] The server collects local spot data from the internet and various databases. During this process, natural language processing technology is used to analyze the collected text data (reviews, ratings, etc.) and classify the characteristics of each spot. Furthermore, machine learning algorithms are used to identify spots that match the user's preferences.

[0130] 4. Route generation

[0131] Based on the collected and analyzed data, the server considers the user's current location and destination to generate an optimal route tailored to the user's preferences. The generated route is designed to pass through scenic spots, highly-rated restaurants, and other points of interest.

[0132] 5. Send the route

[0133] The server sends the generated route information to the user's terminal. HTTP or WebSocket are used as the communication protocol.

[0134] 6. Route display and guidance

[0135] The device displays the route received from the server on a map application and provides visual and audio guidance to the user. For example, it can provide specific guidance such as, "There is a beautiful lake on your right 100 meters ahead, so please be sure to stop by." Furthermore, if a route change is necessary during travel, it can suggest a new route in real time.

[0136] 7. Gathering feedback and incorporating it into future proposals.

[0137] After completing a trip, users provide feedback within the app, including ratings and comments on the provided route. The device sends this feedback data to a server, which then updates the user's preference database based on this data. This updated data is then reflected in future route suggestions, providing a more personalized experience.

[0138] Specific example

[0139] Use Case 1: For users who want to enjoy nature

[0140] The user enters their preference for nature into the app.

[0141] Based on these preferences and past travel history, the server generates a route that includes parks and natural tourist attractions.

[0142] The server sends the generated route to the terminal.

[0143] The device provides route information to the user via map and voice guidance.

[0144] After the user has traveled, they provide feedback on the route through the app.

[0145] The server collects feedback and incorporates it into the next route suggestion.

[0146] Use Case 2: For users who love food

[0147] Users input their preference for "discovering new restaurants" into the app.

[0148] Based on these preferences, the server generates a route that includes new, highly-rated restaurants.

[0149] The server sends the generated route to the terminal.

[0150] The device provides route information to the user via map and voice guidance.

[0151] After the user has visited the restaurant, they provide feedback on their experience through the app.

[0152] The server collects feedback and incorporates it into the next route suggestion.

[0153] Examples of input prompts for a generative AI model

[0154] "Please explain the specifications of the program that suggests the best routes for users who want to enjoy nature."

[0155] "Please explain how the navigation system works for users who want to discover new restaurants."

[0156] This allows the generating AI model to dynamically generate optimal routes based on the user's diverse preferences and travel history, and to generate a detailed description of the system that provides a rich travel experience.

[0157] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0158] Step 1:

[0159] The user launches the application and enters their preferences. When the user selects preferences such as "I like nature" or "I want to discover new restaurants," this input data is generated. Specifically, the user accesses the app's preference settings screen and taps on the options to set their preferences. This preference information is sent to the server in JSON format. The server parses this data and stores it in a database.

[0160] Input: User preference data (e.g., "I like nature," "I want to discover new restaurants")

[0161] Output: User preference information stored in the database

[0162] Step 2:

[0163] The device collects the user's movement history in real time. It uses GPS to acquire location information at regular intervals and sends it to the server. The server analyzes the received location information and stores the user's movement patterns in a database. Specifically, the device acquires location information every 10 seconds and sends it to the server using an HTTP request.

[0164] Input: User's real-time location information (e.g., latitude and longitude data)

[0165] Output: Movement history information stored in the database

[0166] Step 3:

[0167] The server collects local spot information from the internet and various databases. It analyzes the collected text data (such as reviews and ratings) using natural language processing technology and classifies the characteristics of each spot. For example, the server collects reviews from review sites via an API and calculates a spot rating score using a natural language processing algorithm. Furthermore, it uses machine learning algorithms to identify spots that match the user's preferences.

[0168] Input: Spot information collected from the internet and databases.

[0169] Output: Characteristic data and evaluation scores of the analyzed spots

[0170] Step 4:

[0171] The server generates the optimal route based on the user's current location, destination, preferences, and analyzed spot data. It uses algorithms to calculate paths between points and design routes that include spots tailored to the user's preferences. For example, the server might obtain the user's current location and destination and generate a route that includes parks and highly-rated restaurants. The generated route is sent to the terminal in JSON format.

[0172] Input: User's current location, destination, preference data, and location data.

[0173] Output: Optimal route data sent to the terminal

[0174] Step 5:

[0175] The device analyzes route information received from the server and displays it on the map application. Specifically, it analyzes the JSON data received by the device and renders the route information on the map screen. Furthermore, it initiates voice guidance and provides guidance to the user at designated points. For example, it might announce, "There is a highly-rated cafe at the next right turn."

[0176] Input: Optimal route data received from the server

[0177] Output: Route display and voice guidance on the map app.

[0178] Step 6:

[0179] After completing a trip, users provide feedback within the app, including ratings and comments on the provided route. The device sends this feedback to the server in JSON format. The server analyzes the feedback data and updates the user preference database. For example, a user might rate a route they experienced as "excellent" and enter specific comments.

[0180] Input: User feedback data (ratings and comments)

[0181] Output: Updated preference database

[0182] Step 7:

[0183] The server suggests new routes in real time if new conditions arise during travel. For example, when traffic congestion or road construction occurs, it receives new information and sends a recalculated route to the terminal, providing the user with the latest route information.

[0184] Input: Real-time traffic information, user's current location, destination

[0185] Output: Updated optimal route data

[0186] (Application Example 1)

[0187] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0188] Conventional navigation systems can suggest routes based on user preferences and tastes, but they are limited to real-world travel environments, making it difficult to provide access to remote locations or virtual sightseeing experiences. Furthermore, optimization based on travel history and preferences is insufficient, limiting their ability to provide a truly satisfying experience. Therefore, a new system is needed to offer a more enriching sightseeing experience.

[0189] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0190] In this invention, the server includes means for inputting user preferences, means for collecting the user's travel history, and means for collecting and analyzing local spot data. This makes it possible to generate an optimal virtual sightseeing route based on the user's preferences.

[0191] "User preferences" refer to elements that users are interested in or concerned with, and include categories such as historical buildings, nature, and shopping.

[0192] "User travel history" refers to information about places a user has visited and routes they have used in the past.

[0193] "Local spot data" refers to data that includes detailed information and evaluations of tourist destinations, facilities, restaurants, natural landscapes, and other similar locations.

[0194] "Methods for generating routes" refer to systems that design the optimal route based on user preferences and location data.

[0195] "Terminal" refers to a device used by a user, and includes smartphones, smart glasses, and head-mounted displays.

[0196] "Means of visual display" refers to a system that presents information visually through a screen or display.

[0197] "Virtual movement history" refers to a record of the movements a user has made in a virtual environment.

[0198] A "virtual sightseeing route" refers to a route that includes paths and spots for sightseeing in a virtual space.

[0199] "Means of collecting feedback" refers to a system for obtaining ratings and comments from users.

[0200] "Natural language processing" refers to computational techniques for understanding and analyzing human language.

[0201] "Machine learning" refers to the technology of learning patterns from data to perform predictions and classifications.

[0202] A "navigation terminal" refers to a device that provides route guidance, including voice guidance functionality.

[0203] This invention is a navigation system that provides an optimal virtual sightseeing experience based on the user's preferences and travel history. This system consists of the following various means.

[0204] 1. User's preferred input method

[0205] Users can input their travel preferences via devices such as smartphones or head-mounted displays. For example, this can be done by selecting categories such as historical buildings, natural landscapes, and shopping.

[0206] 2. Means for collecting user movement history

[0207] The device collects a history of places visited and routes traveled by the user during virtual sightseeing and sends this information to a server. This allows past virtual travel history to be stored in a database.

[0208] 3. Means for collecting and analyzing local spot data

[0209] The server retrieves tourist spot information, reviews, and image data from the internet and various databases, and analyzes this data using natural language processing and machine learning algorithms. As a result, the characteristics and ratings of each spot are classified.

[0210] 4. Means for generating routes

[0211] The server generates an optimal virtual sightseeing route based on the user's preferences and collected spot data. The generated route is optimized to include spots that the user is interested in.

[0212] 5. Means for transmitting route information to the terminal

[0213] The server sends the generated tourist route information to the user's device. This allows the user to receive route information in real time.

[0214] 6. Means by which the terminal visually displays and guides the route.

[0215] The device (smart glasses or head-mounted display) visually displays the received route information in a VR environment and provides specific guidance to the user. For example, it might say, "There's a famous historical site 300 meters ahead, so please stop by."

[0216] 7. Means of collecting user feedback

[0217] After users complete their virtual tour, they provide ratings and comments on the provided routes and spots. This feedback is sent from the device to the server and used to improve future route suggestions.

[0218] Specific example

[0219] Example 1: Virtual sightseeing to enjoy nature

[0220] If a user prefers "natural scenery," the server generates a sightseeing route based on the user's preferences, including natural spots such as mountains, lakes, and forests. The device visually displays this route in a VR environment and provides audio guidance. After completing the tour, the user sends feedback such as, "The scenery was very beautiful."

[0221] Example 2: Virtual sightseeing enjoying shopping in the city

[0222] If a user enjoys shopping, the server generates a sightseeing route based on the user's preferences, including shopping areas and highly-rated stores. The device visually displays this route in a VR environment, showcasing store information and reviews. After completing the tour, the user sends feedback such as "I discovered a new brand."

[0223] Example of a prompt

[0224] "Create an application that generates optimal virtual sightseeing routes based on user preferences. Use user preference data, tourist spot information, and rating data to suggest routes tailored to the user's preferences and visually display them in a VR environment. Implement a function to collect user feedback and incorporate it into future suggestions."

[0225] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0226] Step 1: Enter user preferences

[0227] Users input their travel preferences (e.g., historical buildings, natural landscapes, shopping) using devices such as smartphones or head-mounted displays. This input data is stored on the device and sent to the server in the next step.

[0228] Input: User preference data (historical buildings, natural landscapes, shopping, etc.)

[0229] Output: User preference data stored on the device

[0230] Step 2: Collect the user's movement history.

[0231] The device automatically collects the user's history of virtual tourist destinations and travel routes. This data is sent to the server as a list of the spots and routes the user has visited.

[0232] Input: Data on virtual tourist destinations and travel routes previously visited by the user.

[0233] Output: User movement history data sent to the server

[0234] Step 3: Collect and analyze local spot data.

[0235] The server collects spot information, reviews, and image data related to tourist destinations from the internet and various databases, and analyzes them using natural language processing technology and machine learning algorithms. As a result of the analysis, the characteristics and evaluations of each spot are classified.

[0236] Input: Information and reviews of tourist attractions collected from the internet and databases.

[0237] Output: Spot data categorized by features and evaluations.

[0238] Step 4: Route Generation

[0239] The server generates the optimal virtual sightseeing route based on the user's preferences and collected and analyzed spot data. This process prioritizes the types of spots the user wants to visit and optimizes the route accordingly.

[0240] Input: User preference data and categorized spot data

[0241] Output: Optimal virtual sightseeing route

[0242] Step 5: Send route information to the device.

[0243] The server sends the generated tourist route information to the user's device. This allows the user to receive route information in real time.

[0244] Input: Optimal virtual sightseeing route

[0245] Output: Route information sent to the terminal

[0246] Step 6: Visual representation and guidance of the route

[0247] The device (smart glasses or head-mounted display) visually displays the received route information in a VR environment and provides specific guidance to the user. For example, it might provide voice guidance such as, "There's a famous historical site 300 meters ahead, so please stop by."

[0248] Input: Received route information

[0249] Output: Visually displayed route and voice guidance in a VR environment

[0250] Step 7: Gathering Feedback

[0251] After the user completes their virtual tour, they enter their evaluation and comments on the provided route and spots. This feedback data is sent from the device to the server and used to improve future route suggestions.

[0252] Input: User feedback (ratings and comments on routes and spots)

[0253] Output: Feedback data sent to the server

[0254] Through these steps, the system can provide an optimal virtual sightseeing experience based on the user's preferences and travel history, thereby increasing user satisfaction.

[0255] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0256] This invention combines an emotion engine with a navigation system that suggests routes based on the user's preferences and travel history, in order to enhance the user's daily travel experience. This system can recognize the user's emotional state in real time and suggest routes that take this into consideration. The embodiments for carrying out this invention will be described in detail below.

[0257] 1. Overview of the entire system

[0258] This navigation system has the following main components:

[0259] 1. A means for users to input their preferences.

[0260] 2. Means for collecting user movement history

[0261] 3. Means for collecting and analyzing local spot data

[0262] 4. Means for generating routes based on user preferences and spot data

[0263] 5. Means for sending the generated route to the terminal

[0264] 6. Means by which the terminal displays and guides the route.

[0265] 7. A means of collecting user feedback and incorporating it into future route suggestions.

[0266] 8. A means of collecting emotional data using an emotion engine that recognizes user emotions.

[0267] 2. Collection and Analysis of User Data

[0268] The server inputs and stores the user's preferences (e.g., loves nature, wants to discover new restaurants) during the user's initial setup and through periodic visits. Furthermore, the server periodically collects and stores the user's travel history.

[0269] 3. Collection and analysis of local spot data

[0270] The server collects local spot data from the internet and various databases. During this process, it analyzes text data (such as reviews and ratings) using natural language processing and machine learning techniques to classify the characteristics and ratings of each spot.

[0271] 4. Route generation and sending

[0272] The server, based on collected and analyzed data, considers the user's current location and destination to generate an optimal route tailored to the user's preferences. The generated route includes scenic spots and highly-rated restaurants. This route information is then transmitted to the user's device.

[0273] 5. Route display and guidance

[0274] The device displays the route received from the server on a map application and provides visual and audio guidance to the user. For example, it provides specific guidance such as, "There is a beautiful lake on your right 100 meters ahead, so please be sure to stop by."

[0275] 6. Collection and Analysis of Emotional Data

[0276] When users are on the move or providing feedback on route guidance, the device uses an emotion engine to analyze the user's facial expressions and voice tone, collecting emotional data. Examples include reading the user's facial expressions through the camera and analyzing their voice tone through the microphone.

[0277] 7. Optimizing route suggestions based on emotional data

[0278] Based on the emotional data collected by the server, analyze the user's current emotional state and regenerate a corresponding route. For example, if the user is feeling stressed, propose a route that passes through a more relaxing and scenic park.

[0279] 8. Collection of Feedback and Reflection in Next Proposal

[0280] After the user moves, the user provides feedback on the evaluation and comments of the route within the app. This feedback also includes the user's emotional state. The terminal sends this feedback data to the server, and the server analyzes the data and reflects it in the route proposals for subsequent times.

[0281] Specific Example

[0282] Use Case 1: For Users Who Want to Enjoy Nature

[0283] 1. The user enters the preference of "likes nature" into the app.

[0284] 2. Based on this preference and the past movement history, the server generates a route that includes parks and natural tourist attractions.

[0285] 3. The server sends the generated route to the terminal.

[0286] 4. The terminal guides the user about the route information on the map and by voice.

[0287] 5. If the user recognizes their emotion with the emotion engine during movement and finds that they are feeling stressed, propose a new route that includes relaxation spots.

[0288] 6. After the user moves, the user conducts an evaluation of the route and provides emotional feedback in the app.

[0289] 7. The server collects feedback and incorporates it into the next route suggestion.

[0290] Use Case 2: For users who love food

[0291] 1. The user enters their preference for "discovering new restaurants" into the app.

[0292] 2. Based on these preferences, the server generates a route that includes new, highly-rated restaurants.

[0293] 3. The server sends the route it generated to the terminal.

[0294] 4. The device provides route information to the user via map and voice guidance.

[0295] 5. The emotion engine recognizes the user's emotions while they are on the move, and if they express excitement or satisfaction, it will continue to suggest similar new locations.

[0296] 6. After visiting the restaurant, users rate it and provide sentimental feedback through the app.

[0297] 7. The server collects feedback and incorporates it into the next route suggestion.

[0298] Thus, the navigation system of the present invention can provide the optimal route based on the user's preferences and emotional state, transforming travel from a mere means of transportation into entertainment.

[0299] The following describes the processing flow.

[0300] Step 1:

[0301] The user installs the app and completes the initial setup. Here, they enter their preferences (e.g., they like nature, want to discover new restaurants, etc.) and permission settings.

[0302] Step 2:

[0303] The terminal collects the user's preference information and permission settings and sends them to the server. Examples of the data to be sent include the user ID and preferences (e.g., "User ID: 12345, Preferences: Nature, Gourmet").

[0304] Step 3:

[0305] The server saves the user's preference data in the database. This data will serve as the basis for future route proposals.

[0306] Step 4:

[0307] The terminal periodically sends the user's current location and movement history to the server. Examples of the data to be sent include location information and time information (e.g., "User ID: 12345, Location Information: (35.6895, 139.6917), Time: 2023-10-01 08:00").

[0308] Step 5:

[0309] The server collects spot data of the area from the Internet and various databases. This data includes information such as parks, restaurants, and tourist attractions.

[0310] Step 6:

[0311] The server analyzes the spot data of the area it has collected using natural language processing and machine learning techniques. As a result, the characteristics and evaluations of each spot are classified.

[0312] Step 7:

[0313] The server matches the user's preference data with the spot data of the area and generates an optimal route for the user. For example, for a user with a preference of "liking nature", a route passing through nearby parks and natural tourist attractions will be selected.

[0314] Step 8:

[0315] The server sends the generated route to the terminal. An example of the data sent is the route details (e.g., "Route: Starting point (35.6895, 139.6917) → Park (35.6850, 139.6920) → Destination (35.7000, 139.7000)").

[0316] Step 9:

[0317] The device displays route information on a map and provides visual and audio guidance to the user. For example, it might say, "There's a beautiful lake 100 meters ahead, so please stop by."

[0318] Step 10:

[0319] The user confirms the displayed route and begins moving according to the instructions. During the journey, the device will notify the user in real time if any changes or corrections to the route are necessary.

[0320] Step 11:

[0321] The device activates an emotion engine while in motion, analyzing the user's facial expressions and voice tone to collect emotional data. For example, it uses a camera for facial recognition and a microphone to analyze voice tone.

[0322] Step 12:

[0323] The device sends collected emotional data to the server. Examples of data sent include the user ID and emotional state (e.g., "User ID: 12345, Emotion: Stress").

[0324] Step 13:

[0325] The server analyzes emotional data and optimizes route suggestions based on the user's current emotional state. For example, if the user is feeling stressed, it will regenerate a route that includes relaxation spots.

[0326] Step 14:

[0327] The server sends the newly generated route to the terminal, and the terminal guides the user along it. For example, it might say, "There's a relaxation spot nearby; we recommend you stop by."

[0328] Step 15:

[0329] After users have traveled, they can rate and provide feedback on the route within the app. For example, they can enter comments such as, "This route was great," or "I loved the new park."

[0330] Step 16:

[0331] The device sends user feedback data to the server. Examples of data sent include user ID, rating, and comment (e.g., "User ID: 12345, Rating: 5, Comment: The new park was great").

[0332] Step 17:

[0333] The server analyzes feedback data, updates user preference and sentiment data, and incorporates these into future route suggestions. This improves the accuracy of the algorithm, enabling more personalized route suggestions.

[0334] Through these steps, this system can transform users' daily commutes into opportunities for new discoveries and enjoyment. By considering users' preferences and emotions and always providing the optimal route, the journey itself becomes an enjoyable experience.

[0335] (Example 2)

[0336] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0337] Traditional navigation systems viewed user travel merely as a means to an end, failing to consider individual user preferences and emotional states when suggesting routes. As a result, users often found it difficult to experience satisfaction, enjoyment, or relaxation during their journeys. Furthermore, current systems have limited ability to incorporate user feedback into future suggestions. Against this backdrop, there is a need for a navigation system that enhances the user's travel experience and provides optimal routes for each individual user.

[0338] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting user preferences, means for collecting the user's travel history, means for collecting and analyzing local spot data, means for generating a route based on the user's preferences and spot data, means for transmitting the generated route to a terminal, means for the terminal to display and guide the route, means for collecting user feedback and reflecting it in subsequent route suggestions, means for recognizing the user's emotional state and regenerating the route taking it into consideration, means for collecting user emotional data, and means for making route suggestions based on the user's emotional data. This makes it possible to reflect the user's preferences and emotional state in real time and enhance the travel experience.

[0339] "Means for inputting user preferences" refers to an interface and related functions for users to input their interests and preferences (e.g., liking nature, liking historical places, liking food, etc.).

[0340] "Means for collecting user movement history" refers to a function that periodically acquires historical data of routes and locations that a user has traveled in the past and transmits it to the system.

[0341] "Means for collecting and analyzing local spot data" refers to a function that collects spot data such as tourist destinations, restaurants, and parks related to a region from the internet and various databases, and then classifies and analyzes that data using natural language processing and machine learning techniques.

[0342] "Means for generating routes based on user preferences and spot data" refers to a function that combines user-inputted preferences with collected spot data to calculate and generate the optimal travel route.

[0343] "Means for sending generated routes to the terminal" refers to a function for sending travel route information generated on the server to the user's terminal.

[0344] "Means for a terminal to display and guide a route" refers to a function on the user's terminal that displays the received travel route on a map and provides visual and audio guidance.

[0345] "Means for collecting user feedback and reflecting it in future route suggestions" refers to a function that stores feedback data such as ratings and comments collected from users after their trip in the system and reflects it in subsequent route suggestions.

[0346] "Means for recognizing the user's emotional state and regenerating the route taking it into consideration" refers to a function that analyzes the user's emotions in real time and generates a more appropriate route based on that analysis.

[0347] "Means for collecting user emotional data" refers to functions that use cameras and microphones to analyze the user's facial expressions and voice tone in order to collect user emotional data.

[0348] "Methods for providing route suggestions based on user emotional data" refers to a function that uses collected emotional data to inform future route suggestions and optimizes routes according to the user's emotional state.

[0349] This invention is a navigation system that enhances the user's daily travel experience by suggesting routes based on the user's preferences and travel history, and further recognizing the user's emotional state in real time to suggest routes accordingly. Specific embodiments are described in detail below.

[0350] 1. Components of the entire system

[0351] This navigation system includes the following components:

[0352] A way to input user preferences

[0353] Means of collecting user movement history

[0354] A means of collecting and analyzing local spot data.

[0355] A means of generating routes based on user preferences and spot data.

[0356] A means of sending the generated route to the terminal.

[0357] Means by which the terminal displays and guides the route

[0358] A means of collecting user feedback and incorporating it into future route suggestions.

[0359] Means of recognizing a user's emotional state

[0360] Means of collecting user sentiment data

[0361] A method for providing route suggestions based on user sentiment data.

[0362] 2. Program processing and the hardware and software used

[0363] 2.1 Collection of User Data

[0364] The server inputs and stores user preferences in a database during initial setup and through periodic user interactions. It also periodically sends the user's movement history to the server. Specifically, if a user enters "I like nature" on the app's initial setup screen, the device sends this information to the server, which then stores it in the database. Movement history is obtained using the device's GPS function.

[0365] 2.2 Collection and Analysis of Local Spot Data

[0366] The server retrieves spot data from internet APIs and databases (e.g., Google Places API, Yelp API) and analyzes it using natural language processing (NLP) and machine learning techniques. This allows it to classify the characteristics and ratings of each spot. For example, it analyzes text reviews to quantify the spot's rating and stores it in the database.

[0367] 2.3 Route generation and transmission

[0368] The server obtains the user's current location and destination, and calculates and generates the optimal route based on the user's preferences and spot data. The route includes spots the user can enjoy (e.g., parks, cafes, tourist attractions, etc.). This information is sent to the device.

[0369] 2.4 Route display and guidance

[0370] The device displays route information received from the server on a map application (e.g., Google Maps, Apple Maps) and provides visual and audio guidance to the user. For example, it might say, "There's a beautiful lake 100 meters ahead on your right, so be sure to stop by."

[0371] 2.5 Collection and Analysis of Emotional Data

[0372] The device uses an emotion engine to analyze the user's facial expressions and voice tone, collecting emotional data. For example, it uses a camera to read the user's facial expressions and a microphone to analyze their voice tone. This data is transmitted to a server in real time.

[0373] 2.6 Route optimization based on emotional data

[0374] The server receives emotional data and regenerates the route to match the user's current emotional state. For example, if the user is feeling stressed, it will suggest a route that includes many relaxation spots. The new route information is then sent back to the device.

[0375] 2.7 Gathering Feedback and Incorporating it into Future Proposals

[0376] After traveling, users rate and comment on their route within the app. The device sends this feedback data to a server, which analyzes the data and uses it to improve future route suggestions. This continuously improves the user's travel experience.

[0377] 3. Specific Examples

[0378] Use Case 1: For users who want to enjoy nature

[0379] 1. The user enters "I like nature" into the app.

[0380] 2. The device sends this data to the server.

[0381] 3. The server generates the optimal route, including parks and natural tourist spots, and sends it to the terminal.

[0382] 4. The device displays the route on a map and provides voice guidance.

[0383] 5. If the server detects that the user is experiencing stress through the camera or microphone while traveling, it will generate a new route that includes relaxation spots and send it to the device.

[0384] 6. After the user has traveled, they can leave a rating in the app, such as "This route was very good."

[0385] 7. The device sends this feedback to the server, which will then be used to improve the next route suggestion.

[0386] Use Case 2: For users who love food

[0387] 1. The user enters "I want to discover a new restaurant" into the app.

[0388] 2. The device sends this data to the server.

[0389] 3. The server generates a route that includes new restaurants with high ratings and sends it to the terminal.

[0390] 4. The device displays the route on a map and provides voice guidance.

[0391] 5. If the server detects that the user is expressing satisfaction through the camera or microphone while on the move, it will suggest similar new locations.

[0392] 6. After the user has visited the restaurant, they can leave a review on the app, such as "This restaurant was very good."

[0393] 7. The device sends this feedback to the server, which will then be used to improve the next route suggestion.

[0394] Examples of prompts for generative AI models

[0395] "I love nature. Please suggest some recommended routes that include natural spots."

[0396] This system reflects the user's preferences and emotional state in real time, enriching the travel experience. This allows users to enjoy an experience that goes beyond mere transportation and becomes a form of entertainment.

[0397] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0398] Specific processing steps

[0399] Step 1: Collecting User Data

[0400] The server collects the initial user data.

[0401] Input: The user enters their preferences, such as "I like nature," on the initial setup screen of the app.

[0402] Specific action: The device sends this preference data to the server.

[0403] Data processing: The server receives this data and saves it to the database along with the user's ID.

[0404] Output: User preference data stored in the database.

[0405] Step 2: Collecting movement history

[0406] The device collects the user's movement history.

[0407] Input: The user launches the app and begins moving. The GPS function is enabled.

[0408] Specific operation: The device periodically obtains the user's current location via GPS and generates movement history data.

[0409] Data processing: Acquired current location data is saved in chronological order to construct a movement history.

[0410] Output: Send the constructed movement history data to the server.

[0411] Step 3: Collect and analyze local spot data.

[0412] The server collects and analyzes local spot data from the internet and databases.

[0413] Input: Spot data obtained from APIs or online databases.

[0414] Specific operation: The server uses the Google Places API and Yelp API to collect information about local spots (e.g., reviews, ratings, categories).

[0415] Data Processing: Using natural language processing (NLP) and machine learning, we analyze text data to extract the characteristics and evaluations of the locations.

[0416] Output: Store spot data classified by characteristics and evaluations in a database.

[0417] Step 4: Route Generation

[0418] The server generates the optimal route based on user preferences and spot data.

[0419] Input: User's current location, destination, user preference data, and location data.

[0420] Specific operation: The server uses an algorithm to calculate the optimal route between the user's current location and destination.

[0421] Data processing: Optimize the route to include spots that match your preferences.

[0422] Output: Sends optimized route data to the terminal.

[0423] Step 5: Route display and guidance

[0424] The terminal displays and guides the user through the generated route.

[0425] Input: Route data received from the server.

[0426] Specific actions: The device draws the route on the map app and provides voice guidance such as, "There is a beautiful lake on the right 100 meters ahead."

[0427] Data calculation: Calculate the timing of map display updates and voice guidance.

[0428] Output: A user interface that provides visual and auditory guidance.

[0429] Step 6: Collect and analyze emotional data

[0430] The device collects and analyzes user emotional data.

[0431] Input: User's facial expressions and voice.

[0432] Specific operation: The device uses its camera and microphone to collect the user's facial expressions and voice in real time.

[0433] Data processing: The emotion engine analyzes facial expressions and voice tone to determine the emotional state (e.g., happy, relaxed, stressed).

[0434] Output: Send the determined emotion data to the server.

[0435] Step 7: Route optimization based on emotional data

[0436] The server regenerates the route based on sentiment data.

[0437] Input: Emotional data sent from the device.

[0438] Specific operation: The server analyzes the user's emotional state and optimizes the route.

[0439] Data calculation: Re-evaluate and rearrange spots on the route based on emotional state.

[0440] Output: Sends optimized new route data to the terminal.

[0441] Step 8: Gathering feedback and incorporating it into future proposals.

[0442] The server collects user feedback and incorporates it into future route suggestions.

[0443] Input: Ratings and comments made by users within the app after moving.

[0444] Specific action: The device sends user feedback to the server.

[0445] Data processing: Analyze feedback data to extract preferences and areas of dissatisfaction.

[0446] Output: Saves feedback data to the database to help generate routes in the future.

[0447] In this way, by appropriately processing and analyzing the data collected at each step, it is possible to provide users with the optimal travel experience.

[0448] (Application Example 2)

[0449] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0450] Traditional navigation systems suggest optimal routes based on user preferences and travel history, but they fail to consider the user's real-time emotional state, making it difficult to provide a truly satisfying travel experience. This can lead to user stress and decreased satisfaction.

[0451] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0452] In this invention, the server includes means for inputting user preferences, means for collecting the user's travel history, means for collecting and analyzing local spot data, means for generating a route based on the user's preferences and spot data, means for transmitting the generated route to a terminal, means for the terminal to display and guide the user along the route, means for collecting user feedback and reflecting it in future route suggestions, means for recognizing the user's emotional state, and means for optimizing the route considering the user's emotional state. This makes it possible to suggest the optimal route according to the user's emotional state.

[0453] A "means for inputting user preferences" refers to an interface that allows users to communicate their interests and preferences to the system.

[0454] "Means for collecting user travel history" refers to a function that records data on places and routes that a user has traveled in the past.

[0455] "Methods for collecting and analyzing local spot data" refers to technologies for gathering information on tourist attractions, restaurants, and other locations within a specific region and then analyzing that information.

[0456] "Means for generating routes based on user preferences and spot data" refers to a mechanism for creating optimal travel routes based on user preferences and local spot data.

[0457] "Means for sending the generated route to the terminal" refers to communication means for transferring the created travel route to the terminal used by the user.

[0458] "Means for a terminal to display and guide a route" refers to a system that visually displays the travel route on the user's terminal and provides directions via voice or text.

[0459] "Means for collecting user feedback and reflecting it in future route suggestions" refers to a system that collects evaluations and opinions from users after their trip and uses them to generate future routes.

[0460] "Means of recognizing a user's emotional state" refers to technology that analyzes a user's emotions and psychological state at a given time based on their facial expressions and tone of voice.

[0461] "Methods for optimizing routes while considering the user's emotional state" refers to a mechanism that recalculates the route to include relaxing places and interesting spots according to the user's current emotions.

[0462] This invention is a system that proposes and guides an autonomous vehicle along the optimal route, taking into account the user's preferences and emotional state. To realize this system, the following processes are performed.

[0463] System-wide configuration

[0464] This system primarily consists of a server, terminals, and users. The server collects and analyzes data and generates routes, while the terminals are responsible for route guidance and recognizing the user's emotional state. Users provide the system with their preferences and emotional state and follow the suggested route.

[0465] User data collection and analysis

[0466] The server collects data from the user's device during initial setup and periodic visits, allowing the user to input their preferences. The server also collects the user's travel history and matches it with their preferences. This data is used to analyze what kinds of places the user prefers to visit.

[0467] Collection and analysis of spot data

[0468] The server collects local spot data from the internet and various databases. This process utilizes natural language processing and machine learning techniques. The collected data is used to analyze the characteristics and evaluation of spots, identifying spots that match the user's preferences.

[0469] Route generation and sending

[0470] The server generates the optimal route, taking into account the user's current location, destination, preferences, and emotional state. This route is designed to include, for example, scenic spots or highly-rated restaurants. The generated route is sent to the device, which provides the user with visual and audio directions.

[0471] Recognition and optimization of emotional states

[0472] The device is equipped with an emotion engine that recognizes the user's emotional state in real time. It analyzes the user's facial expressions and voice tone through the camera and microphone to determine their current emotional state. For example, if the user is feeling stressed, the server will suggest a scenic route that will help them relax. This emotional data will also be used to generate routes for future trips.

[0473] Gathering and incorporating feedback

[0474] After a user's journey, they provide feedback on the route through their device, offering ratings and opinions. This feedback is sent to the server and used to improve future route generation. This allows for a greater understanding of user preferences and satisfaction.

[0475] Hardware and software used

[0476] Server: Performs data collection, analysis, and route generation.

[0477] Device: Recognizes the user's emotional state and provides route guidance.

[0478] Emotion Engine API: An API that analyzes a user's emotional state.

[0479] Navigation API: An API that generates routes based on the user's preferences and emotional state.

[0480] Feedback API: An API for collecting user feedback.

[0481] Specific example

[0482] For users who want to enjoy nature:

[0483] 1. The user's preference is "I like nature."

[0484] 2. The autonomous vehicle will suggest routes that include relaxing parks and scenic spots.

[0485] 3. If the user experiences stress while in the vehicle, the system will re-suggest a route that includes relaxation spots.

[0486] For foodies:

[0487] 1. The user's preference is to "discover gourmet spots."

[0488] 2. The autonomous vehicle will suggest a route that includes highly-rated restaurants.

[0489] 3. If the user is satisfied, additional highly-rated restaurants will also be suggested.

[0490] Examples of prompts for generative AI models

[0491] Create a specification for an emotion-driven autonomous driving navigation system that suggests the optimal route in real time based on the user's preferences and emotional state. The navigation system must include the following elements:

[0492] A way to input user preferences

[0493] Means of collecting user movement history

[0494] A means of collecting and analyzing local spot data.

[0495] A means of generating routes based on user preferences and spot data.

[0496] A means of sending the generated route to the terminal.

[0497] Means by which the terminal displays and guides the route

[0498] A means of collecting user feedback and incorporating it into future route suggestions.

[0499] A method for collecting emotional data using an emotion engine that recognizes user emotions.

[0500] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0501] Step 1:

[0502] The server collects user preference data through a means of inputting user preferences. It receives information such as "I like nature" or "I want to discover new restaurants" as input. This input data is stored in a database and analyzed for use in subsequent processing.

[0503] Step 2:

[0504] The server collects data on the places and routes the user has visited, using means to collect the user's travel history. Based on this collected data, the server analyzes the user's behavior patterns and identifies individual preferences in more detail.

[0505] Step 3:

[0506] The server collects and analyzes local spot data, gathering information on tourist attractions, restaurants, and other locations from online databases and the internet. This data is then analyzed using natural language processing and machine learning techniques to classify the ratings and characteristics of each spot. Input data consists of text data about the spots, while output data is a list of spots with ratings.

[0507] Step 4:

[0508] The server calculates the optimal travel route for the user using means to generate routes based on the user's preferences and spot data. It uses the user's current location, destination, preferences, and collected spot data as input, and generates the optimal route as output.

[0509] Step 5:

[0510] The server transmits the calculated optimal route to the user's terminal (such as the display of an autonomous vehicle) through a means of sending the generated route to the terminal. The input data is the generated route information, and the output data is the route information sent to the terminal.

[0511] Step 6:

[0512] The terminal notifies the user of the route visually and audibly using means to display and guide them along the route. The input is route information sent from the server, and the output is visual and audible guidance to the user.

[0513] Step 7:

[0514] The device uses means to recognize the user's emotional state to acquire the user's facial expressions and voice tone through the camera and microphone. An emotion engine API analyzes this data to determine the user's current emotional state. The input data is the user's emotional data, and the output data is the analyzed emotional state.

[0515] Step 8:

[0516] The server uses methods to optimize routes by considering the user's emotional state, and recalculates a new route if the user is experiencing stress. The input is the user's current emotional state and existing route information, and the output is the recalculated new route information.

[0517] Step 9:

[0518] After completing their journey, users provide feedback on the route via their device. This feedback is sent to the server as ratings and comments. The input data is user feedback, and the output data is the feedback data stored on the server.

[0519] Step 10:

[0520] The server analyzes the collected feedback to improve future route suggestions, thereby enhancing user preferences and satisfaction. Input data consists of user feedback, while output data provides information for improving the route generation algorithm for future routes.

[0521] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0522] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0523] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0524] [Second Embodiment]

[0525] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0526] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0527] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0528] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0529] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0530] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0531] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0532] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0533] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0534] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0535] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0536] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0537] This invention relates to a navigation system that suggests routes with scenic spots and good restaurants based on the user's preferences and travel history, in order to make everyday travel a more fulfilling experience. The form of this system is described below.

[0538] 1. Overview of the entire system

[0539] This navigation system mainly includes the following components:

[0540] 1. A means for users to input their preferences.

[0541] 2. Means for collecting user movement history

[0542] 3. Means for collecting and analyzing local spot data

[0543] 4. Means for generating routes based on user preferences and spot data

[0544] 5. Means for sending the generated route to the terminal

[0545] 6. Means by which the terminal displays and guides the route.

[0546] 7. A means of collecting user feedback and incorporating it into future route suggestions.

[0547] 2. Collection and analysis of user data

[0548] The server inputs user preferences (e.g., loves nature, wants to discover new restaurants) through initial user setup and periodic interactions, and stores this information in a database. This data is frequently updated to reflect the user's latest preferences. The server also periodically collects and stores the user's travel history to analyze past travel patterns.

[0549] 3. Collection and analysis of local spot data

[0550] The server collects local spot data from the internet and various databases. During this process, natural language processing technology is used to analyze the collected text data (reviews, ratings, etc.) and classify the characteristics and ratings of each spot. Furthermore, machine learning algorithms are used to identify spots that match the user's preferences.

[0551] 4. Route generation and sending

[0552] Based on the collected and analyzed data, the server generates an optimal route tailored to the user's preferences, taking into account the user's current location and destination. The generated route is designed to pass through scenic spots and highly-rated restaurants. This route information is then transmitted to the user's device.

[0553] 5. Route display and guidance

[0554] The device displays the route received from the server on a map application and provides visual and audio guidance to the user. For example, it can provide specific guidance such as, "There is a beautiful lake on your right 100 meters ahead, so please be sure to stop by." It can also suggest a new route in real time if the user needs to change their route while traveling.

[0555] 6. Gathering feedback and incorporating it into future proposals.

[0556] After completing a trip, users provide feedback within the app, including ratings and comments on the provided route. The device sends this feedback data to a server, which then updates the user's preference database based on this data. This updated data is then reflected in future route suggestions, providing a more personalized experience.

[0557] Specific example

[0558] Use Case 1: For users who want to enjoy nature

[0559] 1. The user enters their preference for nature into the app.

[0560] 2. Based on these preferences and past travel history, the server generates a route that includes parks and natural tourist attractions.

[0561] 3. The server sends the route it generated to the terminal.

[0562] 4. The device provides route information to the user via map and voice guidance.

[0563] 5. After the user has traveled, they provide feedback on the route through the app.

[0564] 6. The server collects feedback and incorporates it into the next route suggestion.

[0565] Use Case 2: For users who love food

[0566] 1. The user enters their preference for "discovering new restaurants" into the app.

[0567] 2. Based on these preferences, the server generates a route that includes new, highly-rated restaurants.

[0568] 3. The server sends the route it generated to the terminal.

[0569] 4. The device provides route information to the user via map and voice guidance.

[0570] 5. After the user has traveled to the restaurant, they provide feedback on the restaurant's rating through the app.

[0571] 6. The server collects feedback and incorporates it into the next route suggestion.

[0572] Thus, the navigation system of the present invention is designed to transform the user's daily travel into an opportunity for new discoveries and enjoyment. By providing a route optimized based on the user's preferences, the journey itself becomes an enjoyable experience.

[0573] The following describes the processing flow.

[0574] Step 1:

[0575] The user installs the app and completes the initial setup. Here, they enter their preferences (e.g., they like nature, they want to discover new restaurants, etc.).

[0576] Step 2:

[0577] The device collects user preference information and sends it to the server. Examples of data sent include the user ID and preferences (e.g., "User ID: 12345, Preferences: Nature, Gourmet").

[0578] Step 3:

[0579] The server stores user preference data in a database. This data forms the basis for future route suggestions.

[0580] Step 4:

[0581] The device periodically sends the user's current location and movement history to the server. Examples of data sent include location information and time information (e.g., "User ID: 12345, Location: (35.6895, 139.6917), Time: 2023-10-01 08:00").

[0582] Step 5:

[0583] The server collects local spot data from the internet and various databases. This data includes information on parks, restaurants, tourist attractions, and more.

[0584] Step 6:

[0585] The server analyzes the collected regional spot data using natural language processing and machine learning techniques. This allows for the classification of the characteristics and evaluation of each spot.

[0586] Step 7:

[0587] The server matches user preference data with local spot data to generate the optimal route for the user. For example, a user who likes nature will be given a route that passes through nearby parks and natural tourist spots.

[0588] Step 8:

[0589] The server sends the generated route to the terminal. An example of the data sent is the route details (e.g., "Route: Starting point (35.6895, 139.6917) → Park (35.6850, 139.6920) → Destination (35.7000, 139.7000)").

[0590] Step 9:

[0591] The device displays route information on a map and provides visual and audio guidance to the user. For example, it might say, "There's a beautiful lake 100 meters ahead, so please stop by."

[0592] Step 10:

[0593] The user confirms the displayed route and begins moving according to the instructions. During the journey, the device will notify the user in real time if any changes or corrections to the route are necessary.

[0594] Step 11:

[0595] After users have traveled, they can rate and provide feedback on the route within the app. For example, they can enter comments such as, "This route was great," or "I loved the new park."

[0596] Step 12:

[0597] The device sends user feedback data to the server. Examples of data sent include user ID, rating, and comment (e.g., "User ID: 12345, Rating: 5, Comment: The new park was great").

[0598] Step 13:

[0599] The server analyzes feedback data, updates user preference data, and incorporates it into future route suggestions. This improves the accuracy of the algorithm, enabling more personalized route suggestions.

[0600] Through these steps, this system can transform users' daily commutes into opportunities for new discoveries and enjoyment. Based on user preferences, it always provides the optimal route, changing travel from a mere means to an end in itself.

[0601] (Example 1)

[0602] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0603] Existing navigation systems often simply guide users to the shortest or fastest routes without adequately considering their diverse preferences or travel history. As a result, the journey itself may not be a satisfying experience for users, and may lack a sense of fulfillment. Furthermore, real-time route changes are difficult, making it difficult to respond to unexpected situations that occur during travel. In addition, there is a lack of mechanisms to incorporate user feedback into future route suggestions, making it difficult to provide routes optimized for individual users.

[0604] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0605] In this invention, the server includes means for inputting user preferences, means for collecting user travel history, means for collecting and analyzing local spot information, means for generating a route based on user preferences and spot information, means for transmitting the generated route to the terminal, and means for suggesting a new route in real time if a route change is necessary during travel. This enables individually optimized route guidance that takes into account the diverse preferences and travel history of the user. Furthermore, it can respond quickly to unexpected situations during travel, providing users with a more fulfilling travel experience.

[0606] "User preferences" refer to themes and activities that users are particularly interested in, such as appreciating the natural environment or discovering new restaurants.

[0607] "Travel history" refers to a record of routes a user has traveled and places they have visited in the past.

[0608] "Local spot information" refers to data on the evaluation and characteristics of tourist attractions, restaurants, parks, and other places located within a specific region.

[0609] "Natural language processing" is a technology that enables computers to understand and analyze human language.

[0610] "Machine learning" is an algorithm that allows computer systems to learn from data and automatically discover patterns and rules.

[0611] "Route generation" is the process of calculating and setting the optimal route based on the user's current location, destination, preferences, and point-of-interest information.

[0612] "Real-time route suggestions" refers to providing users with instant access to updated route guidance based on new information and circumstances while they are on the move.

[0613] "Feedback" refers to users providing opinions and evaluations regarding the services and routes they have experienced.

[0614] "Terminal" refers to an electronic device used to display navigation information and provide voice guidance, such as a smartphone or tablet.

[0615] This invention relates to a navigation system that suggests routes with scenic spots and good restaurants based on the user's preferences and travel history, in order to make everyday travel a more fulfilling experience. The form of this system is described below.

[0616] 1. System Configuration

[0617] This navigation system includes the following hardware and software configuration:

[0618] Servers: Cloud servers are used for data processing and storage. Specifically, Amazon Web Services (AWS) and Google Cloud Platform can be used.

[0619] Device: A mobile device such as a smartphone or tablet will be used as the device for user operation. These devices must have GPS functionality and an internet connection.

[0620] Software: This involves using application software for inputting user preferences and travel history, as well as for navigation. Specifically, this includes mobile apps for iOS and Android.

[0621] 2. Collection and Analysis of User Data

[0622] The server inputs user preferences (e.g., loves nature, wants to discover new restaurants) through initial user setup and regular interactions, and stores this information in a database. This data is frequently updated to reflect the user's latest preferences. The server also periodically collects and stores the user's travel history to analyze past travel patterns.

[0623] 3. Collection and analysis of local spot data

[0624] The server collects local spot data from the internet and various databases. During this process, natural language processing technology is used to analyze the collected text data (reviews, ratings, etc.) and classify the characteristics of each spot. Furthermore, machine learning algorithms are used to identify spots that match the user's preferences.

[0625] 4. Route generation

[0626] Based on the collected and analyzed data, the server considers the user's current location and destination to generate an optimal route tailored to the user's preferences. The generated route is designed to pass through scenic spots, highly-rated restaurants, and other points of interest.

[0627] 5. Send the route

[0628] The server sends the generated route information to the user's terminal. HTTP or WebSocket are used as the communication protocol.

[0629] 6. Route display and guidance

[0630] The device displays the route received from the server on a map application and provides visual and audio guidance to the user. For example, it can provide specific guidance such as, "There is a beautiful lake on your right 100 meters ahead, so please be sure to stop by." Furthermore, if a route change is necessary during travel, it can suggest a new route in real time.

[0631] 7. Gathering feedback and incorporating it into future proposals.

[0632] After completing a trip, users provide feedback within the app, including ratings and comments on the provided route. The device sends this feedback data to a server, which then updates the user's preference database based on this data. This updated data is then reflected in future route suggestions, providing a more personalized experience.

[0633] Specific example

[0634] Use Case 1: For users who want to enjoy nature

[0635] The user enters their preference for nature into the app.

[0636] Based on these preferences and past travel history, the server generates a route that includes parks and natural tourist attractions.

[0637] The server sends the generated route to the terminal.

[0638] The device provides route information to the user via map and voice guidance.

[0639] After the user has traveled, they provide feedback on the route through the app.

[0640] The server collects feedback and incorporates it into the next route suggestion.

[0641] Use Case 2: For users who love food

[0642] Users input their preference for "discovering new restaurants" into the app.

[0643] Based on these preferences, the server generates a route that includes new, highly-rated restaurants.

[0644] The server sends the generated route to the terminal.

[0645] The device provides route information to the user via map and voice guidance.

[0646] After the user has visited the restaurant, they provide feedback on their experience through the app.

[0647] The server collects feedback and incorporates it into the next route suggestion.

[0648] Examples of input prompts for a generative AI model

[0649] "Please explain the specifications of the program that suggests the best routes for users who want to enjoy nature."

[0650] "Please explain how the navigation system works for users who want to discover new restaurants."

[0651] This allows the generating AI model to dynamically generate optimal routes based on the user's diverse preferences and travel history, and to generate a detailed description of the system that provides a rich travel experience.

[0652] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0653] Step 1:

[0654] The user launches the application and enters their preferences. When the user selects preferences such as "I like nature" or "I want to discover new restaurants," this input data is generated. Specifically, the user accesses the app's preference settings screen and taps on the options to set their preferences. This preference information is sent to the server in JSON format. The server parses this data and stores it in a database.

[0655] Input: User preference data (e.g., "I like nature," "I want to discover new restaurants")

[0656] Output: User preference information stored in the database

[0657] Step 2:

[0658] The device collects the user's movement history in real time. It uses GPS to acquire location information at regular intervals and sends it to the server. The server analyzes the received location information and stores the user's movement patterns in a database. Specifically, the device acquires location information every 10 seconds and sends it to the server using an HTTP request.

[0659] Input: User's real-time location information (e.g., latitude and longitude data)

[0660] Output: Movement history information stored in the database

[0661] Step 3:

[0662] The server collects local spot information from the internet and various databases. It analyzes the collected text data (such as reviews and ratings) using natural language processing technology and classifies the characteristics of each spot. For example, the server collects reviews from review sites via an API and calculates a spot rating score using a natural language processing algorithm. Furthermore, it uses machine learning algorithms to identify spots that match the user's preferences.

[0663] Input: Spot information collected from the internet and databases.

[0664] Output: Characteristic data and evaluation scores of the analyzed spots

[0665] Step 4:

[0666] The server generates the optimal route based on the user's current location, destination, preferences, and analyzed spot data. It uses algorithms to calculate paths between points and design routes that include spots tailored to the user's preferences. For example, the server might obtain the user's current location and destination and generate a route that includes parks and highly-rated restaurants. The generated route is sent to the terminal in JSON format.

[0667] Input: User's current location, destination, preference data, and location data.

[0668] Output: Optimal route data sent to the terminal

[0669] Step 5:

[0670] The device analyzes route information received from the server and displays it on the map application. Specifically, it analyzes the JSON data received by the device and renders the route information on the map screen. Furthermore, it initiates voice guidance and provides guidance to the user at designated points. For example, it might announce, "There is a highly-rated cafe at the next right turn."

[0671] Input: Optimal route data received from the server

[0672] Output: Route display and voice guidance on the map app.

[0673] Step 6:

[0674] After completing a trip, users provide feedback within the app, including ratings and comments on the provided route. The device sends this feedback to the server in JSON format. The server analyzes the feedback data and updates the user preference database. For example, a user might rate a route they experienced as "excellent" and enter specific comments.

[0675] Input: User feedback data (ratings and comments)

[0676] Output: Updated preference database

[0677] Step 7:

[0678] The server suggests new routes in real time if new conditions arise during travel. For example, when traffic congestion or road construction occurs, it receives new information and sends a recalculated route to the terminal, providing the user with the latest route information.

[0679] Input: Real-time traffic information, user's current location, destination

[0680] Output: Updated optimal route data

[0681] (Application Example 1)

[0682] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0683] Conventional navigation systems can suggest routes based on user preferences and tastes, but they are limited to real-world travel environments, making it difficult to provide access to remote locations or virtual sightseeing experiences. Furthermore, optimization based on travel history and preferences is insufficient, limiting their ability to provide a truly satisfying experience. Therefore, a new system is needed to offer a more enriching sightseeing experience.

[0684] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0685] In this invention, the server includes means for inputting user preferences, means for collecting the user's travel history, and means for collecting and analyzing local spot data. This makes it possible to generate an optimal virtual sightseeing route based on the user's preferences.

[0686] "User preferences" refer to elements that users are interested in or concerned with, and include categories such as historical buildings, nature, and shopping.

[0687] "User travel history" refers to information about places a user has visited and routes they have used in the past.

[0688] "Local spot data" refers to data that includes detailed information and evaluations of tourist destinations, facilities, restaurants, natural landscapes, and other similar locations.

[0689] "Methods for generating routes" refer to systems that design the optimal route based on user preferences and location data.

[0690] "Terminal" refers to a device used by a user, and includes smartphones, smart glasses, and head-mounted displays.

[0691] "Means of visual display" refers to a system that presents information visually through a screen or display.

[0692] "Virtual movement history" refers to a record of the movements a user has made in a virtual environment.

[0693] A "virtual sightseeing route" refers to a route that includes paths and spots for sightseeing in a virtual space.

[0694] "Means of collecting feedback" refers to a system for obtaining ratings and comments from users.

[0695] "Natural language processing" refers to computational techniques for understanding and analyzing human language.

[0696] "Machine learning" refers to the technology of learning patterns from data to perform predictions and classifications.

[0697] A "navigation terminal" refers to a device that provides route guidance, including voice guidance functionality.

[0698] This invention is a navigation system that provides an optimal virtual sightseeing experience based on the user's preferences and travel history. This system consists of the following various means.

[0699] 1. User's preferred input method

[0700] Users can input their travel preferences via devices such as smartphones or head-mounted displays. For example, this can be done by selecting categories such as historical buildings, natural landscapes, and shopping.

[0701] 2. Means for collecting user movement history

[0702] The device collects a history of places visited and routes traveled by the user during virtual sightseeing and sends this information to a server. This allows past virtual travel history to be stored in a database.

[0703] 3. Means for collecting and analyzing local spot data

[0704] The server retrieves tourist spot information, reviews, and image data from the internet and various databases, and analyzes this data using natural language processing and machine learning algorithms. As a result, the characteristics and ratings of each spot are classified.

[0705] 4. Means for generating routes

[0706] The server generates an optimal virtual sightseeing route based on the user's preferences and collected spot data. The generated route is optimized to include spots that the user is interested in.

[0707] 5. Means for transmitting route information to the terminal

[0708] The server sends the generated tourist route information to the user's device. This allows the user to receive route information in real time.

[0709] 6. Means by which the terminal visually displays and guides the route.

[0710] The device (smart glasses or head-mounted display) visually displays the received route information in a VR environment and provides specific guidance to the user. For example, it might say, "There's a famous historical site 300 meters ahead, so please stop by."

[0711] 7. Means of collecting user feedback

[0712] After users complete their virtual tour, they provide ratings and comments on the provided routes and spots. This feedback is sent from the device to the server and used to improve future route suggestions.

[0713] Specific example

[0714] Example 1: Virtual sightseeing to enjoy nature

[0715] If a user prefers "natural scenery," the server generates a sightseeing route based on the user's preferences, including natural spots such as mountains, lakes, and forests. The device visually displays this route in a VR environment and provides audio guidance. After completing the tour, the user sends feedback such as, "The scenery was very beautiful."

[0716] Example 2: Virtual sightseeing enjoying shopping in the city

[0717] If a user enjoys shopping, the server generates a sightseeing route based on the user's preferences, including shopping areas and highly-rated stores. The device visually displays this route in a VR environment, showcasing store information and reviews. After completing the tour, the user sends feedback such as "I discovered a new brand."

[0718] Example of a prompt

[0719] "Create an application that generates optimal virtual sightseeing routes based on user preferences. Use user preference data, tourist spot information, and rating data to suggest routes tailored to the user's preferences and visually display them in a VR environment. Implement a function to collect user feedback and incorporate it into future suggestions."

[0720] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0721] Step 1: Enter user preferences

[0722] Users input their travel preferences (e.g., historical buildings, natural landscapes, shopping) using devices such as smartphones or head-mounted displays. This input data is stored on the device and sent to the server in the next step.

[0723] Input: User preference data (historical buildings, natural landscapes, shopping, etc.)

[0724] Output: User preference data stored on the device

[0725] Step 2: Collect the user's movement history.

[0726] The device automatically collects the user's history of virtual tourist destinations and travel routes. This data is sent to the server as a list of the spots and routes the user has visited.

[0727] Input: Data on virtual tourist destinations and travel routes previously visited by the user.

[0728] Output: User movement history data sent to the server

[0729] Step 3: Collect and analyze local spot data.

[0730] The server collects spot information, reviews, and image data related to tourist destinations from the internet and various databases, and analyzes them using natural language processing technology and machine learning algorithms. As a result of the analysis, the characteristics and evaluations of each spot are classified.

[0731] Input: Information and reviews of tourist attractions collected from the internet and databases.

[0732] Output: Spot data categorized by features and evaluations.

[0733] Step 4: Route Generation

[0734] The server generates the optimal virtual sightseeing route based on the user's preferences and collected and analyzed spot data. This process prioritizes the types of spots the user wants to visit and optimizes the route accordingly.

[0735] Input: User preference data and categorized spot data

[0736] Output: Optimal virtual sightseeing route

[0737] Step 5: Send route information to the device.

[0738] The server sends the generated tourist route information to the user's device. This allows the user to receive route information in real time.

[0739] Input: Optimal virtual sightseeing route

[0740] Output: Route information sent to the terminal

[0741] Step 6: Visual representation and guidance of the route

[0742] The device (smart glasses or head-mounted display) visually displays the received route information in a VR environment and provides specific guidance to the user. For example, it might provide voice guidance such as, "There's a famous historical site 300 meters ahead, so please stop by."

[0743] Input: Received route information

[0744] Output: Visually displayed route and voice guidance in a VR environment

[0745] Step 7: Gathering Feedback

[0746] After the user completes their virtual tour, they enter their evaluation and comments on the provided route and spots. This feedback data is sent from the device to the server and used to improve future route suggestions.

[0747] Input: User feedback (ratings and comments on routes and spots)

[0748] Output: Feedback data sent to the server

[0749] Through these steps, the system can provide an optimal virtual sightseeing experience based on the user's preferences and travel history, thereby increasing user satisfaction.

[0750] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0751] This invention combines an emotion engine with a navigation system that suggests routes based on the user's preferences and travel history, in order to enhance the user's daily travel experience. This system can recognize the user's emotional state in real time and suggest routes that take this into consideration. The embodiments for carrying out this invention will be described in detail below.

[0752] 1. Overview of the entire system

[0753] This navigation system has the following main components:

[0754] 1. A means for users to input their preferences.

[0755] 2. Means for collecting user movement history

[0756] 3. Means for collecting and analyzing local spot data

[0757] 4. Means for generating routes based on user preferences and spot data

[0758] 5. Means for sending the generated route to the terminal

[0759] 6. Means by which the terminal displays and guides the route.

[0760] 7. A means of collecting user feedback and incorporating it into future route suggestions.

[0761] 8. A means of collecting emotional data using an emotion engine that recognizes user emotions.

[0762] 2. Collection and Analysis of User Data

[0763] The server inputs and stores the user's preferences (e.g., loves nature, wants to discover new restaurants) during the user's initial setup and through periodic visits. Furthermore, the server periodically collects and stores the user's travel history.

[0764] 3. Collection and analysis of local spot data

[0765] The server collects local spot data from the internet and various databases. During this process, it analyzes text data (such as reviews and ratings) using natural language processing and machine learning techniques to classify the characteristics and ratings of each spot.

[0766] 4. Route generation and sending

[0767] The server, based on collected and analyzed data, considers the user's current location and destination to generate an optimal route tailored to the user's preferences. The generated route includes scenic spots and highly-rated restaurants. This route information is then transmitted to the user's device.

[0768] 5. Route display and guidance

[0769] The device displays the route received from the server on a map application and provides visual and audio guidance to the user. For example, it provides specific guidance such as, "There is a beautiful lake on your right 100 meters ahead, so please be sure to stop by."

[0770] 6. Collection and Analysis of Emotional Data

[0771] When users are on the move or providing feedback on route guidance, the device uses an emotion engine to analyze the user's facial expressions and voice tone, collecting emotional data. Examples include reading the user's facial expressions through the camera and analyzing their voice tone through the microphone.

[0772] 7. Optimizing route suggestions based on emotional data

[0773] Based on emotional data collected by the server, the system analyzes the user's current emotional state and regenerates a route accordingly. For example, if the user is feeling stressed, it might suggest a route that takes them through a scenic park to help them relax.

[0774] 8. Gathering feedback and incorporating it into future proposals.

[0775] After a user completes a journey, they provide feedback within the app, including their evaluation and comments on the route. This feedback also includes the user's emotional state. The device sends this feedback data to a server, which analyzes the data and uses it to improve future route suggestions.

[0776] Specific example

[0777] Use Case 1: For users who want to enjoy nature

[0778] 1. The user enters their preference for nature into the app.

[0779] 2. Based on these preferences and past travel history, the server generates a route that includes parks and natural tourist attractions.

[0780] 3. The server sends the route it generated to the terminal.

[0781] 4. The device provides route information to the user via map and voice guidance.

[0782] 5. The emotion engine recognizes the user's emotions while they are on the move, and if it determines that they are feeling stressed, it suggests a new route that includes relaxation spots.

[0783] 6. After the user has traveled, they will rate the route and provide emotional feedback through the app.

[0784] 7. The server collects feedback and incorporates it into the next route suggestion.

[0785] Use Case 2: For users who love food

[0786] 1. The user enters their preference for "discovering new restaurants" into the app.

[0787] 2. Based on these preferences, the server generates a route that includes new, highly-rated restaurants.

[0788] 3. The server sends the route it generated to the terminal.

[0789] 4. The device provides route information to the user via map and voice guidance.

[0790] 5. The emotion engine recognizes the user's emotions while they are on the move, and if they express excitement or satisfaction, it will continue to suggest similar new locations.

[0791] 6. After visiting the restaurant, users rate it and provide sentimental feedback through the app.

[0792] 7. The server collects feedback and incorporates it into the next route suggestion.

[0793] Thus, the navigation system of the present invention can provide the optimal route based on the user's preferences and emotional state, transforming travel from a mere means of transportation into entertainment.

[0794] The following describes the processing flow.

[0795] Step 1:

[0796] The user installs the app and completes the initial setup. Here, they enter their preferences (e.g., they like nature, want to discover new restaurants, etc.) and permission settings.

[0797] Step 2:

[0798] The device collects user preference information and permission settings and sends them to the server. Examples of data sent include the user ID and preferences (e.g., "User ID: 12345, Preferences: Nature, Gourmet").

[0799] Step 3:

[0800] The server stores user preference data in a database. This data forms the basis for future route suggestions.

[0801] Step 4:

[0802] The device periodically sends the user's current location and movement history to the server. Examples of data sent include location information and time information (e.g., "User ID: 12345, Location: (35.6895, 139.6917), Time: 2023-10-01 08:00").

[0803] Step 5:

[0804] The server collects local spot data from the internet and various databases. This data includes information on parks, restaurants, tourist attractions, and more.

[0805] Step 6:

[0806] The server analyzes the collected regional spot data using natural language processing and machine learning techniques. This allows for the classification of the characteristics and evaluation of each spot.

[0807] Step 7:

[0808] The server matches user preference data with local spot data to generate the optimal route for the user. For example, a user who likes nature will be given a route that passes through nearby parks and natural tourist spots.

[0809] Step 8:

[0810] The server sends the generated route to the terminal. An example of the data sent is the route details (e.g., "Route: Starting point (35.6895, 139.6917) → Park (35.6850, 139.6920) → Destination (35.7000, 139.7000)").

[0811] Step 9:

[0812] The device displays route information on a map and provides visual and audio guidance to the user. For example, it might say, "There's a beautiful lake 100 meters ahead, so please stop by."

[0813] Step 10:

[0814] The user confirms the displayed route and begins moving according to the instructions. During the journey, the device will notify the user in real time if any changes or corrections to the route are necessary.

[0815] Step 11:

[0816] The device activates an emotion engine while in motion, analyzing the user's facial expressions and voice tone to collect emotional data. For example, it uses a camera for facial recognition and a microphone to analyze voice tone.

[0817] Step 12:

[0818] The device sends collected emotional data to the server. Examples of data sent include the user ID and emotional state (e.g., "User ID: 12345, Emotion: Stress").

[0819] Step 13:

[0820] The server analyzes emotional data and optimizes route suggestions based on the user's current emotional state. For example, if the user is feeling stressed, it will regenerate a route that includes relaxation spots.

[0821] Step 14:

[0822] The server sends the newly generated route to the terminal, and the terminal guides the user along it. For example, it might say, "There's a relaxation spot nearby; we recommend you stop by."

[0823] Step 15:

[0824] After users have traveled, they can rate and provide feedback on the route within the app. For example, they can enter comments such as, "This route was great," or "I loved the new park."

[0825] Step 16:

[0826] The device sends user feedback data to the server. Examples of data sent include user ID, rating, and comment (e.g., "User ID: 12345, Rating: 5, Comment: The new park was great").

[0827] Step 17:

[0828] The server analyzes feedback data, updates user preference and sentiment data, and incorporates these into future route suggestions. This improves the accuracy of the algorithm, enabling more personalized route suggestions.

[0829] Through these steps, this system can transform users' daily commutes into opportunities for new discoveries and enjoyment. By considering users' preferences and emotions and always providing the optimal route, the journey itself becomes an enjoyable experience.

[0830] (Example 2)

[0831] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0832] Traditional navigation systems viewed user travel merely as a means to an end, failing to consider individual user preferences and emotional states when suggesting routes. As a result, users often found it difficult to experience satisfaction, enjoyment, or relaxation during their journeys. Furthermore, current systems have limited ability to incorporate user feedback into future suggestions. Against this backdrop, there is a need for a navigation system that enhances the user's travel experience and provides optimal routes for each individual user.

[0833] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting user preferences, means for collecting the user's travel history, means for collecting and analyzing local spot data, means for generating a route based on the user's preferences and spot data, means for transmitting the generated route to a terminal, means for the terminal to display and guide the route, means for collecting user feedback and reflecting it in subsequent route suggestions, means for recognizing the user's emotional state and regenerating the route taking it into consideration, means for collecting user emotional data, and means for making route suggestions based on the user's emotional data. This makes it possible to reflect the user's preferences and emotional state in real time and enhance the travel experience.

[0834] "Means for inputting user preferences" refers to an interface and related functions for users to input their interests and preferences (e.g., liking nature, liking historical places, liking food, etc.).

[0835] "Means for collecting user movement history" refers to a function that periodically acquires historical data of routes and locations that a user has traveled in the past and transmits it to the system.

[0836] "Means for collecting and analyzing local spot data" refers to a function that collects spot data such as tourist destinations, restaurants, and parks related to a region from the internet and various databases, and then classifies and analyzes that data using natural language processing and machine learning techniques.

[0837] "Means for generating routes based on user preferences and spot data" refers to a function that combines user-inputted preferences with collected spot data to calculate and generate the optimal travel route.

[0838] "Means for sending generated routes to the terminal" refers to a function for sending travel route information generated on the server to the user's terminal.

[0839] "Means for a terminal to display and guide a route" refers to a function on the user's terminal that displays the received travel route on a map and provides visual and audio guidance.

[0840] "Means for collecting user feedback and reflecting it in future route suggestions" refers to a function that stores feedback data such as ratings and comments collected from users after their trip in the system and reflects it in subsequent route suggestions.

[0841] "Means for recognizing the user's emotional state and regenerating the route taking it into consideration" refers to a function that analyzes the user's emotions in real time and generates a more appropriate route based on that analysis.

[0842] "Means for collecting user emotional data" refers to functions that use cameras and microphones to analyze the user's facial expressions and voice tone in order to collect user emotional data.

[0843] "Methods for providing route suggestions based on user emotional data" refers to a function that uses collected emotional data to inform future route suggestions and optimizes routes according to the user's emotional state.

[0844] This invention is a navigation system that enhances the user's daily travel experience by suggesting routes based on the user's preferences and travel history, and further recognizing the user's emotional state in real time to suggest routes accordingly. Specific embodiments are described in detail below.

[0845] 1. Components of the entire system

[0846] This navigation system includes the following components:

[0847] A way to input user preferences

[0848] Means of collecting user movement history

[0849] A means of collecting and analyzing local spot data.

[0850] A means of generating routes based on user preferences and spot data.

[0851] A means of sending the generated route to the terminal.

[0852] Means by which the terminal displays and guides the route

[0853] A means of collecting user feedback and incorporating it into future route suggestions.

[0854] Means of recognizing a user's emotional state

[0855] Means of collecting user sentiment data

[0856] A method for providing route suggestions based on user sentiment data.

[0857] 2. Program processing and the hardware and software used

[0858] 2.1 Collection of User Data

[0859] The server inputs and stores user preferences in a database during initial setup and through periodic user interactions. It also periodically sends the user's movement history to the server. Specifically, if a user enters "I like nature" on the app's initial setup screen, the device sends this information to the server, which then stores it in the database. Movement history is obtained using the device's GPS function.

[0860] 2.2 Collection and Analysis of Local Spot Data

[0861] The server retrieves spot data from internet APIs and databases (e.g., Google Places API, Yelp API) and analyzes it using natural language processing (NLP) and machine learning techniques. This allows it to classify the characteristics and ratings of each spot. For example, it analyzes text reviews to quantify the spot's rating and stores it in the database.

[0862] 2.3 Route generation and transmission

[0863] The server obtains the user's current location and destination, and calculates and generates the optimal route based on the user's preferences and spot data. The route includes spots the user can enjoy (e.g., parks, cafes, tourist attractions, etc.). This information is sent to the device.

[0864] 2.4 Route display and guidance

[0865] The device displays route information received from the server on a map application (e.g., Google Maps, Apple Maps) and provides visual and audio guidance to the user. For example, it might say, "There's a beautiful lake 100 meters ahead on your right, so be sure to stop by."

[0866] 2.5 Collection and Analysis of Emotional Data

[0867] The device uses an emotion engine to analyze the user's facial expressions and voice tone, collecting emotional data. For example, it uses a camera to read the user's facial expressions and a microphone to analyze their voice tone. This data is transmitted to a server in real time.

[0868] 2.6 Route optimization based on emotional data

[0869] The server receives emotional data and regenerates the route to match the user's current emotional state. For example, if the user is feeling stressed, it will suggest a route that includes many relaxation spots. The new route information is then sent back to the device.

[0870] 2.7 Gathering Feedback and Incorporating it into Future Proposals

[0871] After traveling, users rate and comment on their route within the app. The device sends this feedback data to a server, which analyzes the data and uses it to improve future route suggestions. This continuously improves the user's travel experience.

[0872] 3. Specific Examples

[0873] Use Case 1: For users who want to enjoy nature

[0874] 1. The user enters "I like nature" into the app.

[0875] 2. The device sends this data to the server.

[0876] 3. The server generates the optimal route, including parks and natural tourist spots, and sends it to the terminal.

[0877] 4. The device displays the route on a map and provides voice guidance.

[0878] 5. If the server detects that the user is experiencing stress through the camera or microphone while traveling, it will generate a new route that includes relaxation spots and send it to the device.

[0879] 6. After the user has traveled, they can leave a rating in the app, such as "This route was very good."

[0880] 7. The device sends this feedback to the server, which will then be used to improve the next route suggestion.

[0881] Use Case 2: For users who love food

[0882] 1. The user enters "I want to discover a new restaurant" into the app.

[0883] 2. The device sends this data to the server.

[0884] 3. The server generates a route that includes new restaurants with high ratings and sends it to the terminal.

[0885] 4. The device displays the route on a map and provides voice guidance.

[0886] 5. If the server detects that the user is expressing satisfaction through the camera or microphone while on the move, it will suggest similar new locations.

[0887] 6. After the user has visited the restaurant, they can leave a review on the app, such as "This restaurant was very good."

[0888] 7. The device sends this feedback to the server, which will then be used to improve the next route suggestion.

[0889] Examples of prompts for generative AI models

[0890] "I love nature. Please suggest some recommended routes that include natural spots."

[0891] This system reflects the user's preferences and emotional state in real time, enriching the travel experience. This allows users to enjoy an experience that goes beyond mere transportation and becomes a form of entertainment.

[0892] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0893] Specific processing steps

[0894] Step 1: Collecting User Data

[0895] The server collects the initial user data.

[0896] Input: The user enters their preferences, such as "I like nature," on the initial setup screen of the app.

[0897] Specific action: The device sends this preference data to the server.

[0898] Data processing: The server receives this data and saves it to the database along with the user's ID.

[0899] Output: User preference data stored in the database.

[0900] Step 2: Collecting movement history

[0901] The device collects the user's movement history.

[0902] Input: The user launches the app and begins moving. The GPS function is enabled.

[0903] Specific operation: The device periodically obtains the user's current location via GPS and generates movement history data.

[0904] Data processing: Acquired current location data is saved in chronological order to construct a movement history.

[0905] Output: Send the constructed movement history data to the server.

[0906] Step 3: Collect and analyze local spot data.

[0907] The server collects and analyzes local spot data from the internet and databases.

[0908] Input: Spot data obtained from APIs or online databases.

[0909] Specific operation: The server uses the Google Places API and Yelp API to collect information about local spots (e.g., reviews, ratings, categories).

[0910] Data Processing: Using natural language processing (NLP) and machine learning, we analyze text data to extract the characteristics and evaluations of the locations.

[0911] Output: Store spot data classified by characteristics and evaluations in a database.

[0912] Step 4: Route Generation

[0913] The server generates the optimal route based on user preferences and spot data.

[0914] Input: User's current location, destination, user preference data, and location data.

[0915] Specific operation: The server uses an algorithm to calculate the optimal route between the user's current location and destination.

[0916] Data processing: Optimize the route to include spots that match your preferences.

[0917] Output: Sends optimized route data to the terminal.

[0918] Step 5: Route display and guidance

[0919] The terminal displays and guides the user through the generated route.

[0920] Input: Route data received from the server.

[0921] Specific actions: The device draws the route on the map app and provides voice guidance such as, "There is a beautiful lake on the right 100 meters ahead."

[0922] Data calculation: Calculate the timing of map display updates and voice guidance.

[0923] Output: A user interface that provides visual and auditory guidance.

[0924] Step 6: Collect and analyze emotional data

[0925] The device collects and analyzes user emotional data.

[0926] Input: User's facial expressions and voice.

[0927] Specific operation: The device uses its camera and microphone to collect the user's facial expressions and voice in real time.

[0928] Data processing: The emotion engine analyzes facial expressions and voice tone to determine the emotional state (e.g., happy, relaxed, stressed).

[0929] Output: Send the determined emotion data to the server.

[0930] Step 7: Route optimization based on emotional data

[0931] The server regenerates the route based on sentiment data.

[0932] Input: Emotional data sent from the device.

[0933] Specific operation: The server analyzes the user's emotional state and optimizes the route.

[0934] Data calculation: Re-evaluate and rearrange spots on the route based on emotional state.

[0935] Output: Sends optimized new route data to the terminal.

[0936] Step 8: Gathering feedback and incorporating it into future proposals.

[0937] The server collects user feedback and incorporates it into future route suggestions.

[0938] Input: Ratings and comments made by users within the app after moving.

[0939] Specific action: The device sends user feedback to the server.

[0940] Data processing: Analyze feedback data to extract preferences and areas of dissatisfaction.

[0941] Output: Saves feedback data to the database to help generate routes in the future.

[0942] In this way, by appropriately processing and analyzing the data collected at each step, it is possible to provide users with the optimal travel experience.

[0943] (Application Example 2)

[0944] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0945] Traditional navigation systems suggest optimal routes based on user preferences and travel history, but they fail to consider the user's real-time emotional state, making it difficult to provide a truly satisfying travel experience. This can lead to user stress and decreased satisfaction.

[0946] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0947] In this invention, the server includes means for inputting user preferences, means for collecting the user's travel history, means for collecting and analyzing local spot data, means for generating a route based on the user's preferences and spot data, means for transmitting the generated route to a terminal, means for the terminal to display and guide the user along the route, means for collecting user feedback and reflecting it in future route suggestions, means for recognizing the user's emotional state, and means for optimizing the route considering the user's emotional state. This makes it possible to suggest the optimal route according to the user's emotional state.

[0948] A "means for inputting user preferences" refers to an interface that allows users to communicate their interests and preferences to the system.

[0949] "Means for collecting user travel history" refers to a function that records data on places and routes that a user has traveled in the past.

[0950] "Methods for collecting and analyzing local spot data" refers to technologies for gathering information on tourist attractions, restaurants, and other locations within a specific region and then analyzing that information.

[0951] "Means for generating routes based on user preferences and spot data" refers to a mechanism for creating optimal travel routes based on user preferences and local spot data.

[0952] "Means for sending the generated route to the terminal" refers to communication means for transferring the created travel route to the terminal used by the user.

[0953] "Means for a terminal to display and guide a route" refers to a system that visually displays the travel route on the user's terminal and provides directions via voice or text.

[0954] "Means for collecting user feedback and reflecting it in future route suggestions" refers to a system that collects evaluations and opinions from users after their trip and uses them to generate future routes.

[0955] "Means of recognizing a user's emotional state" refers to technology that analyzes a user's emotions and psychological state at a given time based on their facial expressions and tone of voice.

[0956] "Methods for optimizing routes while considering the user's emotional state" refers to a mechanism that recalculates the route to include relaxing places and interesting spots according to the user's current emotions.

[0957] This invention is a system that proposes and guides an autonomous vehicle along the optimal route, taking into account the user's preferences and emotional state. To realize this system, the following processes are performed.

[0958] System-wide configuration

[0959] This system primarily consists of a server, terminals, and users. The server collects and analyzes data and generates routes, while the terminals are responsible for route guidance and recognizing the user's emotional state. Users provide the system with their preferences and emotional state and follow the suggested route.

[0960] User data collection and analysis

[0961] The server collects data from the user's device during initial setup and periodic visits, allowing the user to input their preferences. The server also collects the user's travel history and matches it with their preferences. This data is used to analyze what kinds of places the user prefers to visit.

[0962] Collection and analysis of spot data

[0963] The server collects local spot data from the internet and various databases. This process utilizes natural language processing and machine learning techniques. The collected data is used to analyze the characteristics and evaluation of spots, identifying spots that match the user's preferences.

[0964] Route generation and sending

[0965] The server generates the optimal route, taking into account the user's current location, destination, preferences, and emotional state. This route is designed to include, for example, scenic spots or highly-rated restaurants. The generated route is sent to the device, which provides the user with visual and audio directions.

[0966] Recognition and optimization of emotional states

[0967] The device is equipped with an emotion engine that recognizes the user's emotional state in real time. It analyzes the user's facial expressions and voice tone through the camera and microphone to determine their current emotional state. For example, if the user is feeling stressed, the server will suggest a scenic route that will help them relax. This emotional data will also be used to generate routes for future trips.

[0968] Gathering and incorporating feedback

[0969] After a user's journey, they provide feedback on the route through their device, offering ratings and opinions. This feedback is sent to the server and used to improve future route generation. This allows for a greater understanding of user preferences and satisfaction.

[0970] Hardware and software used

[0971] Server: Performs data collection, analysis, and route generation.

[0972] Device: Recognizes the user's emotional state and provides route guidance.

[0973] Emotion Engine API: An API that analyzes a user's emotional state.

[0974] Navigation API: An API that generates routes based on the user's preferences and emotional state.

[0975] Feedback API: An API for collecting user feedback.

[0976] Specific example

[0977] For users who want to enjoy nature:

[0978] 1. The user's preference is "I like nature."

[0979] 2. The autonomous vehicle will suggest routes that include relaxing parks and scenic spots.

[0980] 3. If the user experiences stress while in the vehicle, the system will re-suggest a route that includes relaxation spots.

[0981] For foodies:

[0982] 1. The user's preference is to "discover gourmet spots."

[0983] 2. The autonomous vehicle will suggest a route that includes highly-rated restaurants.

[0984] 3. If the user is satisfied, additional highly-rated restaurants will also be suggested.

[0985] Examples of prompts for generative AI models

[0986] Create a specification for an emotion-driven autonomous driving navigation system that suggests the optimal route in real time based on the user's preferences and emotional state. The navigation system must include the following elements:

[0987] A way to input user preferences

[0988] Means of collecting user movement history

[0989] A means of collecting and analyzing local spot data.

[0990] A means of generating routes based on user preferences and spot data.

[0991] A means of sending the generated route to the terminal.

[0992] Means by which the terminal displays and guides the route

[0993] A means of collecting user feedback and incorporating it into future route suggestions.

[0994] A method for collecting emotional data using an emotion engine that recognizes user emotions.

[0995] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0996] Step 1:

[0997] The server collects user preference data through a means of inputting user preferences. It receives information such as "I like nature" or "I want to discover new restaurants" as input. This input data is stored in a database and analyzed for use in subsequent processing.

[0998] Step 2:

[0999] The server collects data on the places and routes the user has visited, using means to collect the user's travel history. Based on this collected data, the server analyzes the user's behavior patterns and identifies individual preferences in more detail.

[1000] Step 3:

[1001] The server collects and analyzes local spot data, gathering information on tourist attractions, restaurants, and other locations from online databases and the internet. This data is then analyzed using natural language processing and machine learning techniques to classify the ratings and characteristics of each spot. Input data consists of text data about the spots, while output data is a list of spots with ratings.

[1002] Step 4:

[1003] The server calculates the optimal travel route for the user using means to generate routes based on the user's preferences and spot data. It uses the user's current location, destination, preferences, and collected spot data as input, and generates the optimal route as output.

[1004] Step 5:

[1005] The server transmits the calculated optimal route to the user's terminal (such as the display of an autonomous vehicle) through a means of sending the generated route to the terminal. The input data is the generated route information, and the output data is the route information sent to the terminal.

[1006] Step 6:

[1007] The terminal notifies the user of the route visually and audibly using means to display and guide them along the route. The input is route information sent from the server, and the output is visual and audible guidance to the user.

[1008] Step 7:

[1009] The device uses means to recognize the user's emotional state to acquire the user's facial expressions and voice tone through the camera and microphone. An emotion engine API analyzes this data to determine the user's current emotional state. The input data is the user's emotional data, and the output data is the analyzed emotional state.

[1010] Step 8:

[1011] The server uses methods to optimize routes by considering the user's emotional state, and recalculates a new route if the user is experiencing stress. The input is the user's current emotional state and existing route information, and the output is the recalculated new route information.

[1012] Step 9:

[1013] After completing their journey, users provide feedback on the route via their device. This feedback is sent to the server as ratings and comments. The input data is user feedback, and the output data is the feedback data stored on the server.

[1014] Step 10:

[1015] The server analyzes the collected feedback to improve future route suggestions, thereby enhancing user preferences and satisfaction. Input data consists of user feedback, while output data provides information for improving the route generation algorithm for future routes.

[1016] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1017] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1018] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1019] [Third Embodiment]

[1020] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1021] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1023] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1024] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1025] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1027] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1028] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1030] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1032] This invention relates to a navigation system that suggests routes with scenic spots and good restaurants based on the user's preferences and travel history, in order to make everyday travel a more fulfilling experience. The form of this system is described below.

[1033] 1. Overview of the entire system

[1034] This navigation system mainly includes the following components:

[1035] 1. A means for users to input their preferences.

[1036] 2. Means for collecting user movement history

[1037] 3. Means for collecting and analyzing local spot data

[1038] 4. Means for generating routes based on user preferences and spot data

[1039] 5. Means for sending the generated route to the terminal

[1040] 6. Means by which the terminal displays and guides the route.

[1041] 7. A means of collecting user feedback and incorporating it into future route suggestions.

[1042] 2. Collection and analysis of user data

[1043] The server inputs user preferences (e.g., loves nature, wants to discover new restaurants) through initial user setup and periodic interactions, and stores this information in a database. This data is frequently updated to reflect the user's latest preferences. The server also periodically collects and stores the user's travel history to analyze past travel patterns.

[1044] 3. Collection and analysis of local spot data

[1045] The server collects local spot data from the internet and various databases. During this process, natural language processing technology is used to analyze the collected text data (reviews, ratings, etc.) and classify the characteristics and ratings of each spot. Furthermore, machine learning algorithms are used to identify spots that match the user's preferences.

[1046] 4. Route generation and sending

[1047] Based on the collected and analyzed data, the server generates an optimal route tailored to the user's preferences, taking into account the user's current location and destination. The generated route is designed to pass through scenic spots and highly-rated restaurants. This route information is then transmitted to the user's device.

[1048] 5. Route display and guidance

[1049] The device displays the route received from the server on a map application and provides visual and audio guidance to the user. For example, it can provide specific guidance such as, "There is a beautiful lake on your right 100 meters ahead, so please be sure to stop by." It can also suggest a new route in real time if the user needs to change their route while traveling.

[1050] 6. Gathering feedback and incorporating it into future proposals.

[1051] After completing a trip, users provide feedback within the app, including ratings and comments on the provided route. The device sends this feedback data to a server, which then updates the user's preference database based on this data. This updated data is then reflected in future route suggestions, providing a more personalized experience.

[1052] Specific example

[1053] Use Case 1: For users who want to enjoy nature

[1054] 1. The user enters their preference for nature into the app.

[1055] 2. Based on these preferences and past travel history, the server generates a route that includes parks and natural tourist attractions.

[1056] 3. The server sends the route it generated to the terminal.

[1057] 4. The device provides route information to the user via map and voice guidance.

[1058] 5. After the user has traveled, they provide feedback on the route through the app.

[1059] 6. The server collects feedback and incorporates it into the next route suggestion.

[1060] Use Case 2: For users who love food

[1061] 1. The user enters their preference for "discovering new restaurants" into the app.

[1062] 2. Based on these preferences, the server generates a route that includes new, highly-rated restaurants.

[1063] 3. The server sends the route it generated to the terminal.

[1064] 4. The device provides route information to the user via map and voice guidance.

[1065] 5. After the user has traveled to the restaurant, they provide feedback on the restaurant's rating through the app.

[1066] 6. The server collects feedback and incorporates it into the next route suggestion.

[1067] Thus, the navigation system of the present invention is designed to transform the user's daily travel into an opportunity for new discoveries and enjoyment. By providing a route optimized based on the user's preferences, the journey itself becomes an enjoyable experience.

[1068] The following describes the processing flow.

[1069] Step 1:

[1070] The user installs the app and completes the initial setup. Here, they enter their preferences (e.g., they like nature, they want to discover new restaurants, etc.).

[1071] Step 2:

[1072] The device collects user preference information and sends it to the server. Examples of data sent include the user ID and preferences (e.g., "User ID: 12345, Preferences: Nature, Gourmet").

[1073] Step 3:

[1074] The server stores user preference data in a database. This data forms the basis for future route suggestions.

[1075] Step 4:

[1076] The device periodically sends the user's current location and movement history to the server. Examples of data sent include location information and time information (e.g., "User ID: 12345, Location: (35.6895, 139.6917), Time: 2023-10-01 08:00").

[1077] Step 5:

[1078] The server collects local spot data from the internet and various databases. This data includes information on parks, restaurants, tourist attractions, and more.

[1079] Step 6:

[1080] The server analyzes the collected regional spot data using natural language processing and machine learning techniques. This allows for the classification of the characteristics and evaluation of each spot.

[1081] Step 7:

[1082] The server matches user preference data with local spot data to generate the optimal route for the user. For example, a user who likes nature will be given a route that passes through nearby parks and natural tourist spots.

[1083] Step 8:

[1084] The server sends the generated route to the terminal. An example of the data sent is the route details (e.g., "Route: Starting point (35.6895, 139.6917) → Park (35.6850, 139.6920) → Destination (35.7000, 139.7000)").

[1085] Step 9:

[1086] The device displays route information on a map and provides visual and audio guidance to the user. For example, it might say, "There's a beautiful lake 100 meters ahead, so please stop by."

[1087] Step 10:

[1088] The user confirms the displayed route and begins moving according to the instructions. During the journey, the device will notify the user in real time if any changes or corrections to the route are necessary.

[1089] Step 11:

[1090] After users have traveled, they can rate and provide feedback on the route within the app. For example, they can enter comments such as, "This route was great," or "I loved the new park."

[1091] Step 12:

[1092] The device sends user feedback data to the server. Examples of data sent include user ID, rating, and comment (e.g., "User ID: 12345, Rating: 5, Comment: The new park was great").

[1093] Step 13:

[1094] The server analyzes feedback data, updates user preference data, and incorporates it into future route suggestions. This improves the accuracy of the algorithm, enabling more personalized route suggestions.

[1095] Through these steps, this system can transform users' daily commutes into opportunities for new discoveries and enjoyment. Based on user preferences, it always provides the optimal route, changing travel from a mere means to an end in itself.

[1096] (Example 1)

[1097] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1098] Existing navigation systems often simply guide users to the shortest or fastest routes without adequately considering their diverse preferences or travel history. As a result, the journey itself may not be a satisfying experience for users, and may lack a sense of fulfillment. Furthermore, real-time route changes are difficult, making it difficult to respond to unexpected situations that occur during travel. In addition, there is a lack of mechanisms to incorporate user feedback into future route suggestions, making it difficult to provide routes optimized for individual users.

[1099] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1100] In this invention, the server includes means for inputting user preferences, means for collecting user travel history, means for collecting and analyzing local spot information, means for generating a route based on user preferences and spot information, means for transmitting the generated route to the terminal, and means for suggesting a new route in real time if a route change is necessary during travel. This enables individually optimized route guidance that takes into account the diverse preferences and travel history of the user. Furthermore, it can respond quickly to unexpected situations during travel, providing users with a more fulfilling travel experience.

[1101] "User preferences" refer to themes and activities that users are particularly interested in, such as appreciating the natural environment or discovering new restaurants.

[1102] "Travel history" refers to a record of routes a user has traveled and places they have visited in the past.

[1103] "Local spot information" refers to data on the evaluation and characteristics of tourist attractions, restaurants, parks, and other places located within a specific region.

[1104] "Natural language processing" is a technology that enables computers to understand and analyze human language.

[1105] "Machine learning" is an algorithm that allows computer systems to learn from data and automatically discover patterns and rules.

[1106] "Route generation" is the process of calculating and setting the optimal route based on the user's current location, destination, preferences, and point-of-interest information.

[1107] "Real-time route suggestions" refers to providing users with instant access to updated route guidance based on new information and circumstances while they are on the move.

[1108] "Feedback" refers to users providing opinions and evaluations regarding the services and routes they have experienced.

[1109] "Terminal" refers to an electronic device used to display navigation information and provide voice guidance, such as a smartphone or tablet.

[1110] This invention relates to a navigation system that suggests routes with scenic spots and good restaurants based on the user's preferences and travel history, in order to make everyday travel a more fulfilling experience. The form of this system is described below.

[1111] 1. System Configuration

[1112] This navigation system includes the following hardware and software configuration:

[1113] Servers: Cloud servers are used for data processing and storage. Specifically, Amazon Web Services (AWS) and Google Cloud Platform can be used.

[1114] Device: A mobile device such as a smartphone or tablet will be used as the device for user operation. These devices must have GPS functionality and an internet connection.

[1115] Software: This involves using application software for inputting user preferences and travel history, as well as for navigation. Specifically, this includes mobile apps for iOS and Android.

[1116] 2. Collection and Analysis of User Data

[1117] The server inputs user preferences (e.g., loves nature, wants to discover new restaurants) through initial user setup and regular interactions, and stores this information in a database. This data is frequently updated to reflect the user's latest preferences. The server also periodically collects and stores the user's travel history to analyze past travel patterns.

[1118] 3. Collection and analysis of local spot data

[1119] The server collects local spot data from the internet and various databases. During this process, natural language processing technology is used to analyze the collected text data (reviews, ratings, etc.) and classify the characteristics of each spot. Furthermore, machine learning algorithms are used to identify spots that match the user's preferences.

[1120] 4. Route generation

[1121] Based on the collected and analyzed data, the server considers the user's current location and destination to generate an optimal route tailored to the user's preferences. The generated route is designed to pass through scenic spots, highly-rated restaurants, and other points of interest.

[1122] 5. Send the route

[1123] The server sends the generated route information to the user's terminal. HTTP or WebSocket are used as the communication protocol.

[1124] 6. Route display and guidance

[1125] The device displays the route received from the server on a map application and provides visual and audio guidance to the user. For example, it can provide specific guidance such as, "There is a beautiful lake on your right 100 meters ahead, so please be sure to stop by." Furthermore, if a route change is necessary during travel, it can suggest a new route in real time.

[1126] 7. Gathering feedback and incorporating it into future proposals.

[1127] After completing a trip, users provide feedback within the app, including ratings and comments on the provided route. The device sends this feedback data to a server, which then updates the user's preference database based on this data. This updated data is then reflected in future route suggestions, providing a more personalized experience.

[1128] Specific example

[1129] Use Case 1: For users who want to enjoy nature

[1130] The user enters their preference for nature into the app.

[1131] Based on these preferences and past travel history, the server generates a route that includes parks and natural tourist attractions.

[1132] The server sends the generated route to the terminal.

[1133] The device provides route information to the user via map and voice guidance.

[1134] After the user has traveled, they provide feedback on the route through the app.

[1135] The server collects feedback and incorporates it into the next route suggestion.

[1136] Use Case 2: For users who love food

[1137] Users input their preference for "discovering new restaurants" into the app.

[1138] Based on these preferences, the server generates a route that includes new, highly-rated restaurants.

[1139] The server sends the generated route to the terminal.

[1140] The device provides route information to the user via map and voice guidance.

[1141] After the user has visited the restaurant, they provide feedback on their experience through the app.

[1142] The server collects feedback and incorporates it into the next route suggestion.

[1143] Examples of input prompts for a generative AI model

[1144] "Please explain the specifications of the program that suggests the best routes for users who want to enjoy nature."

[1145] "Please explain how the navigation system works for users who want to discover new restaurants."

[1146] This allows the generating AI model to dynamically generate optimal routes based on the user's diverse preferences and travel history, and to generate a detailed description of the system that provides a rich travel experience.

[1147] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1148] Step 1:

[1149] The user launches the application and enters their preferences. When the user selects preferences such as "I like nature" or "I want to discover new restaurants," this input data is generated. Specifically, the user accesses the app's preference settings screen and taps on the options to set their preferences. This preference information is sent to the server in JSON format. The server parses this data and stores it in a database.

[1150] Input: User preference data (e.g., "I like nature," "I want to discover new restaurants")

[1151] Output: User preference information stored in the database

[1152] Step 2:

[1153] The device collects the user's movement history in real time. It uses GPS to acquire location information at regular intervals and sends it to the server. The server analyzes the received location information and stores the user's movement patterns in a database. Specifically, the device acquires location information every 10 seconds and sends it to the server using an HTTP request.

[1154] Input: User's real-time location information (e.g., latitude and longitude data)

[1155] Output: Movement history information stored in the database

[1156] Step 3:

[1157] The server collects local spot information from the internet and various databases. It analyzes the collected text data (such as reviews and ratings) using natural language processing technology and classifies the characteristics of each spot. For example, the server collects reviews from review sites via an API and calculates a spot rating score using a natural language processing algorithm. Furthermore, it uses machine learning algorithms to identify spots that match the user's preferences.

[1158] Input: Spot information collected from the internet and databases.

[1159] Output: Characteristic data and evaluation scores of the analyzed spots

[1160] Step 4:

[1161] The server generates the optimal route based on the user's current location, destination, preferences, and analyzed spot data. It uses algorithms to calculate paths between points and design routes that include spots tailored to the user's preferences. For example, the server might obtain the user's current location and destination and generate a route that includes parks and highly-rated restaurants. The generated route is sent to the terminal in JSON format.

[1162] Input: User's current location, destination, preference data, and location data.

[1163] Output: Optimal route data sent to the terminal

[1164] Step 5:

[1165] The device analyzes route information received from the server and displays it on the map application. Specifically, it analyzes the JSON data received by the device and renders the route information on the map screen. Furthermore, it initiates voice guidance and provides guidance to the user at designated points. For example, it might announce, "There is a highly-rated cafe at the next right turn."

[1166] Input: Optimal route data received from the server

[1167] Output: Route display and voice guidance on the map app.

[1168] Step 6:

[1169] After completing a trip, users provide feedback within the app, including ratings and comments on the provided route. The device sends this feedback to the server in JSON format. The server analyzes the feedback data and updates the user preference database. For example, a user might rate a route they experienced as "excellent" and enter specific comments.

[1170] Input: User feedback data (ratings and comments)

[1171] Output: Updated preference database

[1172] Step 7:

[1173] The server suggests new routes in real time if new conditions arise during travel. For example, when traffic congestion or road construction occurs, it receives new information and sends a recalculated route to the terminal, providing the user with the latest route information.

[1174] Input: Real-time traffic information, user's current location, destination

[1175] Output: Updated optimal route data

[1176] (Application Example 1)

[1177] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1178] Conventional navigation systems can suggest routes based on user preferences and tastes, but they are limited to real-world travel environments, making it difficult to provide access to remote locations or virtual sightseeing experiences. Furthermore, optimization based on travel history and preferences is insufficient, limiting their ability to provide a truly satisfying experience. Therefore, a new system is needed to offer a more enriching sightseeing experience.

[1179] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1180] In this invention, the server includes means for inputting user preferences, means for collecting the user's travel history, and means for collecting and analyzing local spot data. This makes it possible to generate an optimal virtual sightseeing route based on the user's preferences.

[1181] "User preferences" refer to elements that users are interested in or concerned with, and include categories such as historical buildings, nature, and shopping.

[1182] "User travel history" refers to information about places a user has visited and routes they have used in the past.

[1183] "Local spot data" refers to data that includes detailed information and evaluations of tourist destinations, facilities, restaurants, natural landscapes, and other similar locations.

[1184] "Methods for generating routes" refer to systems that design the optimal route based on user preferences and location data.

[1185] "Terminal" refers to a device used by a user, and includes smartphones, smart glasses, and head-mounted displays.

[1186] "Means of visual display" refers to a system that presents information visually through a screen or display.

[1187] "Virtual movement history" refers to a record of the movements a user has made in a virtual environment.

[1188] A "virtual sightseeing route" refers to a route that includes paths and spots for sightseeing in a virtual space.

[1189] "Means of collecting feedback" refers to a system for obtaining ratings and comments from users.

[1190] "Natural language processing" refers to computational techniques for understanding and analyzing human language.

[1191] "Machine learning" refers to the technology of learning patterns from data to perform predictions and classifications.

[1192] A "navigation terminal" refers to a device that provides route guidance, including voice guidance functionality.

[1193] This invention is a navigation system that provides an optimal virtual sightseeing experience based on the user's preferences and travel history. This system consists of the following various means.

[1194] 1. User's preferred input method

[1195] Users can input their travel preferences via devices such as smartphones or head-mounted displays. For example, this can be done by selecting categories such as historical buildings, natural landscapes, and shopping.

[1196] 2. Means for collecting user movement history

[1197] The device collects a history of places visited and routes traveled by the user during virtual sightseeing and sends this information to a server. This allows past virtual travel history to be stored in a database.

[1198] 3. Means for collecting and analyzing local spot data

[1199] The server retrieves tourist spot information, reviews, and image data from the internet and various databases, and analyzes this data using natural language processing and machine learning algorithms. As a result, the characteristics and ratings of each spot are classified.

[1200] 4. Means for generating routes

[1201] The server generates an optimal virtual sightseeing route based on the user's preferences and collected spot data. The generated route is optimized to include spots that the user is interested in.

[1202] 5. Means for transmitting route information to the terminal

[1203] The server sends the generated tourist route information to the user's device. This allows the user to receive route information in real time.

[1204] 6. Means by which the terminal visually displays and guides the route.

[1205] The device (smart glasses or head-mounted display) visually displays the received route information in a VR environment and provides specific guidance to the user. For example, it might say, "There's a famous historical site 300 meters ahead, so please stop by."

[1206] 7. Means of collecting user feedback

[1207] After users complete their virtual tour, they provide ratings and comments on the provided routes and spots. This feedback is sent from the device to the server and used to improve future route suggestions.

[1208] Specific example

[1209] Example 1: Virtual sightseeing to enjoy nature

[1210] If a user prefers "natural scenery," the server generates a sightseeing route based on the user's preferences, including natural spots such as mountains, lakes, and forests. The device visually displays this route in a VR environment and provides audio guidance. After completing the tour, the user sends feedback such as, "The scenery was very beautiful."

[1211] Example 2: Virtual sightseeing enjoying shopping in the city

[1212] If a user enjoys shopping, the server generates a sightseeing route based on the user's preferences, including shopping areas and highly-rated stores. The device visually displays this route in a VR environment, showcasing store information and reviews. After completing the tour, the user sends feedback such as "I discovered a new brand."

[1213] Example of a prompt

[1214] "Create an application that generates optimal virtual sightseeing routes based on user preferences. Use user preference data, tourist spot information, and rating data to suggest routes tailored to the user's preferences and visually display them in a VR environment. Implement a function to collect user feedback and incorporate it into future suggestions."

[1215] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1216] Step 1: Enter user preferences

[1217] Users input their travel preferences (e.g., historical buildings, natural landscapes, shopping) using devices such as smartphones or head-mounted displays. This input data is stored on the device and sent to the server in the next step.

[1218] Input: User preference data (historical buildings, natural landscapes, shopping, etc.)

[1219] Output: User preference data stored on the device

[1220] Step 2: Collect the user's movement history.

[1221] The device automatically collects the user's history of virtual tourist destinations and travel routes. This data is sent to the server as a list of the spots and routes the user has visited.

[1222] Input: Data on virtual tourist destinations and travel routes previously visited by the user.

[1223] Output: User movement history data sent to the server

[1224] Step 3: Collect and analyze local spot data.

[1225] The server collects spot information, reviews, and image data related to tourist destinations from the internet and various databases, and analyzes them using natural language processing technology and machine learning algorithms. As a result of the analysis, the characteristics and evaluations of each spot are classified.

[1226] Input: Information and reviews of tourist attractions collected from the internet and databases.

[1227] Output: Spot data categorized by features and evaluations.

[1228] Step 4: Route Generation

[1229] The server generates the optimal virtual sightseeing route based on the user's preferences and collected and analyzed spot data. This process prioritizes the types of spots the user wants to visit and optimizes the route accordingly.

[1230] Input: User preference data and categorized spot data

[1231] Output: Optimal virtual sightseeing route

[1232] Step 5: Send route information to the device.

[1233] The server sends the generated tourist route information to the user's device. This allows the user to receive route information in real time.

[1234] Input: Optimal virtual sightseeing route

[1235] Output: Route information sent to the terminal

[1236] Step 6: Visual representation and guidance of the route

[1237] The device (smart glasses or head-mounted display) visually displays the received route information in a VR environment and provides specific guidance to the user. For example, it might provide voice guidance such as, "There's a famous historical site 300 meters ahead, so please stop by."

[1238] Input: Received route information

[1239] Output: Visually displayed route and voice guidance in a VR environment

[1240] Step 7: Gathering Feedback

[1241] After the user completes their virtual tour, they enter their evaluation and comments on the provided route and spots. This feedback data is sent from the device to the server and used to improve future route suggestions.

[1242] Input: User feedback (ratings and comments on routes and spots)

[1243] Output: Feedback data sent to the server

[1244] Through these steps, the system can provide an optimal virtual sightseeing experience based on the user's preferences and travel history, thereby increasing user satisfaction.

[1245] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1246] This invention combines an emotion engine with a navigation system that suggests routes based on the user's preferences and travel history, in order to enhance the user's daily travel experience. This system can recognize the user's emotional state in real time and suggest routes that take this into consideration. The embodiments for carrying out this invention will be described in detail below.

[1247] 1. Overview of the entire system

[1248] This navigation system has the following main components:

[1249] 1. A means for users to input their preferences.

[1250] 2. Means for collecting user movement history

[1251] 3. Means for collecting and analyzing local spot data

[1252] 4. Means for generating routes based on user preferences and spot data

[1253] 5. Means for sending the generated route to the terminal

[1254] 6. Means by which the terminal displays and guides the route.

[1255] 7. A means of collecting user feedback and incorporating it into future route suggestions.

[1256] 8. A means of collecting emotional data using an emotion engine that recognizes user emotions.

[1257] 2. Collection and Analysis of User Data

[1258] The server inputs and stores the user's preferences (e.g., loves nature, wants to discover new restaurants) during the user's initial setup and through periodic visits. Furthermore, the server periodically collects and stores the user's travel history.

[1259] 3. Collection and analysis of local spot data

[1260] The server collects local spot data from the internet and various databases. During this process, it analyzes text data (such as reviews and ratings) using natural language processing and machine learning techniques to classify the characteristics and ratings of each spot.

[1261] 4. Route generation and sending

[1262] The server, based on collected and analyzed data, considers the user's current location and destination to generate an optimal route tailored to the user's preferences. The generated route includes scenic spots and highly-rated restaurants. This route information is then transmitted to the user's device.

[1263] 5. Route display and guidance

[1264] The device displays the route received from the server on a map application and provides visual and audio guidance to the user. For example, it provides specific guidance such as, "There is a beautiful lake on your right 100 meters ahead, so please be sure to stop by."

[1265] 6. Collection and Analysis of Emotional Data

[1266] When users are on the move or providing feedback on route guidance, the device uses an emotion engine to analyze the user's facial expressions and voice tone, collecting emotional data. Examples include reading the user's facial expressions through the camera and analyzing their voice tone through the microphone.

[1267] 7. Optimizing route suggestions based on emotional data

[1268] Based on emotional data collected by the server, the system analyzes the user's current emotional state and regenerates a route accordingly. For example, if the user is feeling stressed, it might suggest a route that takes them through a scenic park to help them relax.

[1269] 8. Gathering feedback and incorporating it into future proposals.

[1270] After a user completes a journey, they provide feedback within the app, including their evaluation and comments on the route. This feedback also includes the user's emotional state. The device sends this feedback data to a server, which analyzes the data and uses it to improve future route suggestions.

[1271] Specific example

[1272] Use Case 1: For users who want to enjoy nature

[1273] 1. The user enters their preference for nature into the app.

[1274] 2. Based on these preferences and past travel history, the server generates a route that includes parks and natural tourist attractions.

[1275] 3. The server sends the route it generated to the terminal.

[1276] 4. The device provides route information to the user via map and voice guidance.

[1277] 5. The emotion engine recognizes the user's emotions while they are on the move, and if it determines that they are feeling stressed, it suggests a new route that includes relaxation spots.

[1278] 6. After the user has traveled, they will rate the route and provide emotional feedback through the app.

[1279] 7. The server collects feedback and incorporates it into the next route suggestion.

[1280] Use Case 2: For users who love food

[1281] 1. The user enters their preference for "discovering new restaurants" into the app.

[1282] 2. Based on these preferences, the server generates a route that includes new, highly-rated restaurants.

[1283] 3. The server sends the route it generated to the terminal.

[1284] 4. The device provides route information to the user via map and voice guidance.

[1285] 5. The emotion engine recognizes the user's emotions while they are on the move, and if they express excitement or satisfaction, it will continue to suggest similar new locations.

[1286] 6. After visiting the restaurant, users rate it and provide sentimental feedback through the app.

[1287] 7. The server collects feedback and incorporates it into the next route suggestion.

[1288] Thus, the navigation system of the present invention can provide the optimal route based on the user's preferences and emotional state, transforming travel from a mere means of transportation into entertainment.

[1289] The following describes the processing flow.

[1290] Step 1:

[1291] The user installs the app and completes the initial setup. Here, they enter their preferences (e.g., they like nature, want to discover new restaurants, etc.) and permission settings.

[1292] Step 2:

[1293] The device collects user preference information and permission settings and sends them to the server. Examples of data sent include the user ID and preferences (e.g., "User ID: 12345, Preferences: Nature, Gourmet").

[1294] Step 3:

[1295] The server stores user preference data in a database. This data forms the basis for future route suggestions.

[1296] Step 4:

[1297] The device periodically sends the user's current location and movement history to the server. Examples of data sent include location information and time information (e.g., "User ID: 12345, Location: (35.6895, 139.6917), Time: 2023-10-01 08:00").

[1298] Step 5:

[1299] The server collects local spot data from the internet and various databases. This data includes information on parks, restaurants, tourist attractions, and more.

[1300] Step 6:

[1301] The server analyzes the collected regional spot data using natural language processing and machine learning techniques. This allows for the classification of the characteristics and evaluation of each spot.

[1302] Step 7:

[1303] The server matches user preference data with local spot data to generate the optimal route for the user. For example, a user who likes nature will be given a route that passes through nearby parks and natural tourist spots.

[1304] Step 8:

[1305] The server sends the generated route to the terminal. An example of the data sent is the route details (e.g., "Route: Starting point (35.6895, 139.6917) → Park (35.6850, 139.6920) → Destination (35.7000, 139.7000)").

[1306] Step 9:

[1307] The device displays route information on a map and provides visual and audio guidance to the user. For example, it might say, "There's a beautiful lake 100 meters ahead, so please stop by."

[1308] Step 10:

[1309] The user confirms the displayed route and begins moving according to the instructions. During the journey, the device will notify the user in real time if any changes or corrections to the route are necessary.

[1310] Step 11:

[1311] The device activates an emotion engine while in motion, analyzing the user's facial expressions and voice tone to collect emotional data. For example, it uses a camera for facial recognition and a microphone to analyze voice tone.

[1312] Step 12:

[1313] The device sends collected emotional data to the server. Examples of data sent include the user ID and emotional state (e.g., "User ID: 12345, Emotion: Stress").

[1314] Step 13:

[1315] The server analyzes emotional data and optimizes route suggestions based on the user's current emotional state. For example, if the user is feeling stressed, it will regenerate a route that includes relaxation spots.

[1316] Step 14:

[1317] The server sends the newly generated route to the terminal, and the terminal guides the user along it. For example, it might say, "There's a relaxation spot nearby; we recommend you stop by."

[1318] Step 15:

[1319] After users have traveled, they can rate and provide feedback on the route within the app. For example, they can enter comments such as, "This route was great," or "I loved the new park."

[1320] Step 16:

[1321] The device sends user feedback data to the server. Examples of data sent include user ID, rating, and comment (e.g., "User ID: 12345, Rating: 5, Comment: The new park was great").

[1322] Step 17:

[1323] The server analyzes feedback data, updates user preference and sentiment data, and incorporates these into future route suggestions. This improves the accuracy of the algorithm, enabling more personalized route suggestions.

[1324] Through these steps, this system can transform users' daily commutes into opportunities for new discoveries and enjoyment. By considering users' preferences and emotions and always providing the optimal route, the journey itself becomes an enjoyable experience.

[1325] (Example 2)

[1326] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1327] Traditional navigation systems viewed user travel merely as a means to an end, failing to consider individual user preferences and emotional states when suggesting routes. As a result, users often found it difficult to experience satisfaction, enjoyment, or relaxation during their journeys. Furthermore, current systems have limited ability to incorporate user feedback into future suggestions. Against this backdrop, there is a need for a navigation system that enhances the user's travel experience and provides optimal routes for each individual user.

[1328] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting user preferences, means for collecting the user's travel history, means for collecting and analyzing local spot data, means for generating a route based on the user's preferences and spot data, means for transmitting the generated route to a terminal, means for the terminal to display and guide the route, means for collecting user feedback and reflecting it in subsequent route suggestions, means for recognizing the user's emotional state and regenerating the route taking it into consideration, means for collecting user emotional data, and means for making route suggestions based on the user's emotional data. This makes it possible to reflect the user's preferences and emotional state in real time and enhance the travel experience.

[1329] "Means for inputting user preferences" refers to an interface and related functions for users to input their interests and preferences (e.g., liking nature, liking historical places, liking food, etc.).

[1330] "Means for collecting user movement history" refers to a function that periodically acquires historical data of routes and locations that a user has traveled in the past and transmits it to the system.

[1331] "Means for collecting and analyzing local spot data" refers to a function that collects spot data such as tourist destinations, restaurants, and parks related to a region from the internet and various databases, and then classifies and analyzes that data using natural language processing and machine learning techniques.

[1332] "Means for generating routes based on user preferences and spot data" refers to a function that combines user-inputted preferences with collected spot data to calculate and generate the optimal travel route.

[1333] "Means for sending generated routes to the terminal" refers to a function for sending travel route information generated on the server to the user's terminal.

[1334] "Means for a terminal to display and guide a route" refers to a function on the user's terminal that displays the received travel route on a map and provides visual and audio guidance.

[1335] "Means for collecting user feedback and reflecting it in future route suggestions" refers to a function that stores feedback data such as ratings and comments collected from users after their trip in the system and reflects it in subsequent route suggestions.

[1336] "Means for recognizing the user's emotional state and regenerating the route taking it into consideration" refers to a function that analyzes the user's emotions in real time and generates a more appropriate route based on that analysis.

[1337] "Means for collecting user emotional data" refers to functions that use cameras and microphones to analyze the user's facial expressions and voice tone in order to collect user emotional data.

[1338] "Methods for providing route suggestions based on user emotional data" refers to a function that uses collected emotional data to inform future route suggestions and optimizes routes according to the user's emotional state.

[1339] This invention is a navigation system that enhances the user's daily travel experience by suggesting routes based on the user's preferences and travel history, and further recognizing the user's emotional state in real time to suggest routes accordingly. Specific embodiments are described in detail below.

[1340] 1. Components of the entire system

[1341] This navigation system includes the following components:

[1342] A way to input user preferences

[1343] Means of collecting user movement history

[1344] A means of collecting and analyzing local spot data.

[1345] A means of generating routes based on user preferences and spot data.

[1346] A means of sending the generated route to the terminal.

[1347] Means by which the terminal displays and guides the route

[1348] A means of collecting user feedback and incorporating it into future route suggestions.

[1349] Means of recognizing a user's emotional state

[1350] Means of collecting user sentiment data

[1351] A method for providing route suggestions based on user sentiment data.

[1352] 2. Program processing and the hardware and software used

[1353] 2.1 Collection of User Data

[1354] The server inputs and stores user preferences in a database during initial setup and through periodic user interactions. It also periodically sends the user's movement history to the server. Specifically, if a user enters "I like nature" on the app's initial setup screen, the device sends this information to the server, which then stores it in the database. Movement history is obtained using the device's GPS function.

[1355] 2.2 Collection and Analysis of Local Spot Data

[1356] The server retrieves spot data from internet APIs and databases (e.g., Google Places API, Yelp API) and analyzes it using natural language processing (NLP) and machine learning techniques. This allows it to classify the characteristics and ratings of each spot. For example, it analyzes text reviews to quantify the spot's rating and stores it in the database.

[1357] 2.3 Route generation and transmission

[1358] The server obtains the user's current location and destination, and calculates and generates the optimal route based on the user's preferences and spot data. The route includes spots the user can enjoy (e.g., parks, cafes, tourist attractions, etc.). This information is sent to the device.

[1359] 2.4 Route display and guidance

[1360] The device displays route information received from the server on a map application (e.g., Google Maps, Apple Maps) and provides visual and audio guidance to the user. For example, it might say, "There's a beautiful lake 100 meters ahead on your right, so be sure to stop by."

[1361] 2.5 Collection and Analysis of Emotional Data

[1362] The device uses an emotion engine to analyze the user's facial expressions and voice tone, collecting emotional data. For example, it uses a camera to read the user's facial expressions and a microphone to analyze their voice tone. This data is transmitted to a server in real time.

[1363] 2.6 Route optimization based on emotional data

[1364] The server receives emotional data and regenerates the route to match the user's current emotional state. For example, if the user is feeling stressed, it will suggest a route that includes many relaxation spots. The new route information is then sent back to the device.

[1365] 2.7 Gathering Feedback and Incorporating it into Future Proposals

[1366] After traveling, users rate and comment on their route within the app. The device sends this feedback data to a server, which analyzes the data and uses it to improve future route suggestions. This continuously improves the user's travel experience.

[1367] 3. Specific Examples

[1368] Use Case 1: For users who want to enjoy nature

[1369] 1. The user enters "I like nature" into the app.

[1370] 2. The device sends this data to the server.

[1371] 3. The server generates the optimal route, including parks and natural tourist spots, and sends it to the terminal.

[1372] 4. The device displays the route on a map and provides voice guidance.

[1373] 5. If the server detects that the user is experiencing stress through the camera or microphone while traveling, it will generate a new route that includes relaxation spots and send it to the device.

[1374] 6. After the user has traveled, they can leave a rating in the app, such as "This route was very good."

[1375] 7. The device sends this feedback to the server, which will then be used to improve the next route suggestion.

[1376] Use Case 2: For users who love food

[1377] 1. The user enters "I want to discover a new restaurant" into the app.

[1378] 2. The device sends this data to the server.

[1379] 3. The server generates a route that includes new restaurants with high ratings and sends it to the terminal.

[1380] 4. The device displays the route on a map and provides voice guidance.

[1381] 5. If the server detects that the user is expressing satisfaction through the camera or microphone while on the move, it will suggest similar new locations.

[1382] 6. After the user has visited the restaurant, they can leave a review on the app, such as "This restaurant was very good."

[1383] 7. The device sends this feedback to the server, which will then be used to improve the next route suggestion.

[1384] Examples of prompts for generative AI models

[1385] "I love nature. Please suggest some recommended routes that include natural spots."

[1386] This system reflects the user's preferences and emotional state in real time, enriching the travel experience. This allows users to enjoy an experience that goes beyond mere transportation and becomes a form of entertainment.

[1387] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1388] Specific processing steps

[1389] Step 1: Collecting User Data

[1390] The server collects the initial user data.

[1391] Input: The user enters their preferences, such as "I like nature," on the initial setup screen of the app.

[1392] Specific action: The device sends this preference data to the server.

[1393] Data processing: The server receives this data and saves it to the database along with the user's ID.

[1394] Output: User preference data stored in the database.

[1395] Step 2: Collecting movement history

[1396] The device collects the user's movement history.

[1397] Input: The user launches the app and begins moving. The GPS function is enabled.

[1398] Specific operation: The device periodically obtains the user's current location via GPS and generates movement history data.

[1399] Data processing: Acquired current location data is saved in chronological order to construct a movement history.

[1400] Output: Send the constructed movement history data to the server.

[1401] Step 3: Collect and analyze local spot data.

[1402] The server collects and analyzes local spot data from the internet and databases.

[1403] Input: Spot data obtained from APIs or online databases.

[1404] Specific operation: The server uses the Google Places API and Yelp API to collect information about local spots (e.g., reviews, ratings, categories).

[1405] Data Processing: Using natural language processing (NLP) and machine learning, we analyze text data to extract the characteristics and evaluations of the locations.

[1406] Output: Store spot data classified by characteristics and evaluations in a database.

[1407] Step 4: Route Generation

[1408] The server generates the optimal route based on user preferences and spot data.

[1409] Input: User's current location, destination, user preference data, and location data.

[1410] Specific operation: The server uses an algorithm to calculate the optimal route between the user's current location and destination.

[1411] Data processing: Optimize the route to include spots that match your preferences.

[1412] Output: Sends optimized route data to the terminal.

[1413] Step 5: Route display and guidance

[1414] The terminal displays and guides the user through the generated route.

[1415] Input: Route data received from the server.

[1416] Specific actions: The device draws the route on the map app and provides voice guidance such as, "There is a beautiful lake on the right 100 meters ahead."

[1417] Data calculation: Calculate the timing of map display updates and voice guidance.

[1418] Output: A user interface that provides visual and auditory guidance.

[1419] Step 6: Collect and analyze emotional data

[1420] The device collects and analyzes user emotional data.

[1421] Input: User's facial expressions and voice.

[1422] Specific operation: The device uses its camera and microphone to collect the user's facial expressions and voice in real time.

[1423] Data processing: The emotion engine analyzes facial expressions and voice tone to determine the emotional state (e.g., happy, relaxed, stressed).

[1424] Output: Send the determined emotion data to the server.

[1425] Step 7: Route optimization based on emotional data

[1426] The server regenerates the route based on sentiment data.

[1427] Input: Emotional data sent from the device.

[1428] Specific operation: The server analyzes the user's emotional state and optimizes the route.

[1429] Data calculation: Re-evaluate and rearrange spots on the route based on emotional state.

[1430] Output: Sends optimized new route data to the terminal.

[1431] Step 8: Gathering feedback and incorporating it into future proposals.

[1432] The server collects user feedback and incorporates it into future route suggestions.

[1433] Input: Ratings and comments made by users within the app after moving.

[1434] Specific action: The device sends user feedback to the server.

[1435] Data processing: Analyze feedback data to extract preferences and areas of dissatisfaction.

[1436] Output: Saves feedback data to the database to help generate routes in the future.

[1437] In this way, by appropriately processing and analyzing the data collected at each step, it is possible to provide users with the optimal travel experience.

[1438] (Application Example 2)

[1439] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1440] Traditional navigation systems suggest optimal routes based on user preferences and travel history, but they fail to consider the user's real-time emotional state, making it difficult to provide a truly satisfying travel experience. This can lead to user stress and decreased satisfaction.

[1441] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1442] In this invention, the server includes means for inputting user preferences, means for collecting the user's travel history, means for collecting and analyzing local spot data, means for generating a route based on the user's preferences and spot data, means for transmitting the generated route to a terminal, means for the terminal to display and guide the user along the route, means for collecting user feedback and reflecting it in future route suggestions, means for recognizing the user's emotional state, and means for optimizing the route considering the user's emotional state. This makes it possible to suggest the optimal route according to the user's emotional state.

[1443] A "means for inputting user preferences" refers to an interface that allows users to communicate their interests and preferences to the system.

[1444] "Means for collecting user travel history" refers to a function that records data on places and routes that a user has traveled in the past.

[1445] "Methods for collecting and analyzing local spot data" refers to technologies for gathering information on tourist attractions, restaurants, and other locations within a specific region and then analyzing that information.

[1446] "Means for generating routes based on user preferences and spot data" refers to a mechanism for creating optimal travel routes based on user preferences and local spot data.

[1447] "Means for sending the generated route to the terminal" refers to communication means for transferring the created travel route to the terminal used by the user.

[1448] "Means for a terminal to display and guide a route" refers to a system that visually displays the travel route on the user's terminal and provides directions via voice or text.

[1449] "Means for collecting user feedback and reflecting it in future route suggestions" refers to a system that collects evaluations and opinions from users after their trip and uses them to generate future routes.

[1450] "Means of recognizing a user's emotional state" refers to technology that analyzes a user's emotions and psychological state at a given time based on their facial expressions and tone of voice.

[1451] "Methods for optimizing routes while considering the user's emotional state" refers to a mechanism that recalculates the route to include relaxing places and interesting spots according to the user's current emotions.

[1452] This invention is a system that proposes and guides an autonomous vehicle along the optimal route, taking into account the user's preferences and emotional state. To realize this system, the following processes are performed.

[1453] System-wide configuration

[1454] This system primarily consists of a server, terminals, and users. The server collects and analyzes data and generates routes, while the terminals are responsible for route guidance and recognizing the user's emotional state. Users provide the system with their preferences and emotional state and follow the suggested route.

[1455] User data collection and analysis

[1456] The server collects data from the user's device during initial setup and periodic visits, allowing the user to input their preferences. The server also collects the user's travel history and matches it with their preferences. This data is used to analyze what kinds of places the user prefers to visit.

[1457] Collection and analysis of spot data

[1458] The server collects local spot data from the internet and various databases. This process utilizes natural language processing and machine learning techniques. The collected data is used to analyze the characteristics and evaluation of spots, identifying spots that match the user's preferences.

[1459] Route generation and sending

[1460] The server generates the optimal route, taking into account the user's current location, destination, preferences, and emotional state. This route is designed to include, for example, scenic spots or highly-rated restaurants. The generated route is sent to the device, which provides the user with visual and audio directions.

[1461] Recognition and optimization of emotional states

[1462] The device is equipped with an emotion engine that recognizes the user's emotional state in real time. It analyzes the user's facial expressions and voice tone through the camera and microphone to determine their current emotional state. For example, if the user is feeling stressed, the server will suggest a scenic route that will help them relax. This emotional data will also be used to generate routes for future trips.

[1463] Gathering and incorporating feedback

[1464] After a user's journey, they provide feedback on the route through their device, offering ratings and opinions. This feedback is sent to the server and used to improve future route generation. This allows for a greater understanding of user preferences and satisfaction.

[1465] Hardware and software used

[1466] Server: Performs data collection, analysis, and route generation.

[1467] Device: Recognizes the user's emotional state and provides route guidance.

[1468] Emotion Engine API: An API that analyzes a user's emotional state.

[1469] Navigation API: An API that generates routes based on the user's preferences and emotional state.

[1470] Feedback API: An API for collecting user feedback.

[1471] Specific example

[1472] For users who want to enjoy nature:

[1473] 1. The user's preference is "I like nature."

[1474] 2. The autonomous vehicle will suggest routes that include relaxing parks and scenic spots.

[1475] 3. If the user experiences stress while in the vehicle, the system will re-suggest a route that includes relaxation spots.

[1476] For foodies:

[1477] 1. The user's preference is to "discover gourmet spots."

[1478] 2. The autonomous vehicle will suggest a route that includes highly-rated restaurants.

[1479] 3. If the user is satisfied, additional highly-rated restaurants will also be suggested.

[1480] Examples of prompts for generative AI models

[1481] Create a specification for an emotion-driven autonomous driving navigation system that suggests the optimal route in real time based on the user's preferences and emotional state. The navigation system must include the following elements:

[1482] A way to input user preferences

[1483] Means of collecting user movement history

[1484] A means of collecting and analyzing local spot data.

[1485] A means of generating routes based on user preferences and spot data.

[1486] A means of sending the generated route to the terminal.

[1487] Means by which the terminal displays and guides the route

[1488] A means of collecting user feedback and incorporating it into future route suggestions.

[1489] A method for collecting emotional data using an emotion engine that recognizes user emotions.

[1490] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1491] Step 1:

[1492] The server collects user preference data through a means of inputting user preferences. It receives information such as "I like nature" or "I want to discover new restaurants" as input. This input data is stored in a database and analyzed for use in subsequent processing.

[1493] Step 2:

[1494] The server collects data on the places and routes the user has visited, using means to collect the user's travel history. Based on this collected data, the server analyzes the user's behavior patterns and identifies individual preferences in more detail.

[1495] Step 3:

[1496] The server collects and analyzes local spot data, gathering information on tourist attractions, restaurants, and other locations from online databases and the internet. This data is then analyzed using natural language processing and machine learning techniques to classify the ratings and characteristics of each spot. Input data consists of text data about the spots, while output data is a list of spots with ratings.

[1497] Step 4:

[1498] The server calculates the optimal travel route for the user using means to generate routes based on the user's preferences and spot data. It uses the user's current location, destination, preferences, and collected spot data as input, and generates the optimal route as output.

[1499] Step 5:

[1500] The server transmits the calculated optimal route to the user's terminal (such as the display of an autonomous vehicle) through a means of sending the generated route to the terminal. The input data is the generated route information, and the output data is the route information sent to the terminal.

[1501] Step 6:

[1502] The terminal notifies the user of the route visually and audibly using means to display and guide them along the route. The input is route information sent from the server, and the output is visual and audible guidance to the user.

[1503] Step 7:

[1504] The device uses means to recognize the user's emotional state to acquire the user's facial expressions and voice tone through the camera and microphone. An emotion engine API analyzes this data to determine the user's current emotional state. The input data is the user's emotional data, and the output data is the analyzed emotional state.

[1505] Step 8:

[1506] The server uses methods to optimize routes by considering the user's emotional state, and recalculates a new route if the user is experiencing stress. The input is the user's current emotional state and existing route information, and the output is the recalculated new route information.

[1507] Step 9:

[1508] After completing their journey, users provide feedback on the route via their device. This feedback is sent to the server as ratings and comments. The input data is user feedback, and the output data is the feedback data stored on the server.

[1509] Step 10:

[1510] The server analyzes the collected feedback to improve future route suggestions, thereby enhancing user preferences and satisfaction. Input data consists of user feedback, while output data provides information for improving the route generation algorithm for future routes.

[1511] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1512] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1513] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1514] [Fourth Embodiment]

[1515] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1516] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1517] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1518] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1519] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1520] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1521] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1522] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1523] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1524] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1525] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1526] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1527] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1528] This invention relates to a navigation system that suggests routes with scenic spots and good restaurants based on the user's preferences and travel history, in order to make everyday travel a more fulfilling experience. The form of this system is described below.

[1529] 1. Overview of the entire system

[1530] This navigation system mainly includes the following components:

[1531] 1. A means for users to input their preferences.

[1532] 2. Means for collecting user movement history

[1533] 3. Means for collecting and analyzing local spot data

[1534] 4. Means for generating routes based on user preferences and spot data

[1535] 5. Means for sending the generated route to the terminal

[1536] 6. Means by which the terminal displays and guides the route.

[1537] 7. A means of collecting user feedback and incorporating it into future route suggestions.

[1538] 2. Collection and analysis of user data

[1539] The server inputs user preferences (e.g., loves nature, wants to discover new restaurants) through initial user setup and periodic interactions, and stores this information in a database. This data is frequently updated to reflect the user's latest preferences. The server also periodically collects and stores the user's travel history to analyze past travel patterns.

[1540] 3. Collection and analysis of local spot data

[1541] The server collects local spot data from the internet and various databases. During this process, natural language processing technology is used to analyze the collected text data (reviews, ratings, etc.) and classify the characteristics and ratings of each spot. Furthermore, machine learning algorithms are used to identify spots that match the user's preferences.

[1542] 4. Route generation and sending

[1543] Based on the collected and analyzed data, the server generates an optimal route tailored to the user's preferences, taking into account the user's current location and destination. The generated route is designed to pass through scenic spots and highly-rated restaurants. This route information is then transmitted to the user's device.

[1544] 5. Route display and guidance

[1545] The device displays the route received from the server on a map application and provides visual and audio guidance to the user. For example, it can provide specific guidance such as, "There is a beautiful lake on your right 100 meters ahead, so please be sure to stop by." It can also suggest a new route in real time if the user needs to change their route while traveling.

[1546] 6. Gathering feedback and incorporating it into future proposals.

[1547] After completing a trip, users provide feedback within the app, including ratings and comments on the provided route. The device sends this feedback data to a server, which then updates the user's preference database based on this data. This updated data is then reflected in future route suggestions, providing a more personalized experience.

[1548] Specific example

[1549] Use Case 1: For users who want to enjoy nature

[1550] 1. The user enters their preference for nature into the app.

[1551] 2. Based on these preferences and past travel history, the server generates a route that includes parks and natural tourist attractions.

[1552] 3. The server sends the route it generated to the terminal.

[1553] 4. The device provides route information to the user via map and voice guidance.

[1554] 5. After the user has traveled, they provide feedback on the route through the app.

[1555] 6. The server collects feedback and incorporates it into the next route suggestion.

[1556] Use Case 2: For users who love food

[1557] 1. The user enters their preference for "discovering new restaurants" into the app.

[1558] 2. Based on these preferences, the server generates a route that includes new, highly-rated restaurants.

[1559] 3. The server sends the route it generated to the terminal.

[1560] 4. The device provides route information to the user via map and voice guidance.

[1561] 5. After the user has traveled to the restaurant, they provide feedback on the restaurant's rating through the app.

[1562] 6. The server collects feedback and incorporates it into the next route suggestion.

[1563] Thus, the navigation system of the present invention is designed to transform the user's daily travel into an opportunity for new discoveries and enjoyment. By providing a route optimized based on the user's preferences, the journey itself becomes an enjoyable experience.

[1564] The following describes the processing flow.

[1565] Step 1:

[1566] The user installs the app and completes the initial setup. Here, they enter their preferences (e.g., they like nature, they want to discover new restaurants, etc.).

[1567] Step 2:

[1568] The device collects user preference information and sends it to the server. Examples of data sent include the user ID and preferences (e.g., "User ID: 12345, Preferences: Nature, Gourmet").

[1569] Step 3:

[1570] The server stores user preference data in a database. This data forms the basis for future route suggestions.

[1571] Step 4:

[1572] The device periodically sends the user's current location and movement history to the server. Examples of data sent include location information and time information (e.g., "User ID: 12345, Location: (35.6895, 139.6917), Time: 2023-10-01 08:00").

[1573] Step 5:

[1574] The server collects local spot data from the internet and various databases. This data includes information on parks, restaurants, tourist attractions, and more.

[1575] Step 6:

[1576] The server analyzes the collected regional spot data using natural language processing and machine learning techniques. This allows for the classification of the characteristics and evaluation of each spot.

[1577] Step 7:

[1578] The server matches user preference data with local spot data to generate the optimal route for the user. For example, a user who likes nature will be given a route that passes through nearby parks and natural tourist spots.

[1579] Step 8:

[1580] The server sends the generated route to the terminal. An example of the data sent is the route details (e.g., "Route: Starting point (35.6895, 139.6917) → Park (35.6850, 139.6920) → Destination (35.7000, 139.7000)").

[1581] Step 9:

[1582] The device displays route information on a map and provides visual and audio guidance to the user. For example, it might say, "There's a beautiful lake 100 meters ahead, so please stop by."

[1583] Step 10:

[1584] The user confirms the displayed route and begins moving according to the instructions. During the journey, the device will notify the user in real time if any changes or corrections to the route are necessary.

[1585] Step 11:

[1586] After users have traveled, they can rate and provide feedback on the route within the app. For example, they can enter comments such as, "This route was great," or "I loved the new park."

[1587] Step 12:

[1588] The device sends user feedback data to the server. Examples of data sent include user ID, rating, and comment (e.g., "User ID: 12345, Rating: 5, Comment: The new park was great").

[1589] Step 13:

[1590] The server analyzes feedback data, updates user preference data, and incorporates it into future route suggestions. This improves the accuracy of the algorithm, enabling more personalized route suggestions.

[1591] Through these steps, this system can transform users' daily commutes into opportunities for new discoveries and enjoyment. Based on user preferences, it always provides the optimal route, changing travel from a mere means to an end in itself.

[1592] (Example 1)

[1593] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1594] Existing navigation systems often simply guide users to the shortest or fastest routes without adequately considering their diverse preferences or travel history. As a result, the journey itself may not be a satisfying experience for users, and may lack a sense of fulfillment. Furthermore, real-time route changes are difficult, making it difficult to respond to unexpected situations that occur during travel. In addition, there is a lack of mechanisms to incorporate user feedback into future route suggestions, making it difficult to provide routes optimized for individual users.

[1595] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1596] In this invention, the server includes means for inputting user preferences, means for collecting user travel history, means for collecting and analyzing local spot information, means for generating a route based on user preferences and spot information, means for transmitting the generated route to the terminal, and means for suggesting a new route in real time if a route change is necessary during travel. This enables individually optimized route guidance that takes into account the diverse preferences and travel history of the user. Furthermore, it can respond quickly to unexpected situations during travel, providing users with a more fulfilling travel experience.

[1597] "User preferences" refer to themes and activities that users are particularly interested in, such as appreciating the natural environment or discovering new restaurants.

[1598] "Travel history" refers to a record of routes a user has traveled and places they have visited in the past.

[1599] "Local spot information" refers to data on the evaluation and characteristics of tourist attractions, restaurants, parks, and other places located within a specific region.

[1600] "Natural language processing" is a technology that enables computers to understand and analyze human language.

[1601] "Machine learning" is an algorithm that allows computer systems to learn from data and automatically discover patterns and rules.

[1602] "Route generation" is the process of calculating and setting the optimal route based on the user's current location, destination, preferences, and point-of-interest information.

[1603] "Real-time route suggestions" refers to providing users with instant access to updated route guidance based on new information and circumstances while they are on the move.

[1604] "Feedback" refers to users providing opinions and evaluations regarding the services and routes they have experienced.

[1605] "Terminal" refers to an electronic device used to display navigation information and provide voice guidance, such as a smartphone or tablet.

[1606] This invention relates to a navigation system that suggests routes with scenic spots and good restaurants based on the user's preferences and travel history, in order to make everyday travel a more fulfilling experience. The form of this system is described below.

[1607] 1. System Configuration

[1608] This navigation system includes the following hardware and software configuration:

[1609] Servers: Cloud servers are used for data processing and storage. Specifically, Amazon Web Services (AWS) and Google Cloud Platform can be used.

[1610] Device: A mobile device such as a smartphone or tablet will be used as the device for user operation. These devices must have GPS functionality and an internet connection.

[1611] Software: This involves using application software for inputting user preferences and travel history, as well as for navigation. Specifically, this includes mobile apps for iOS and Android.

[1612] 2. Collection and Analysis of User Data

[1613] The server inputs user preferences (e.g., loves nature, wants to discover new restaurants) through initial user setup and regular interactions, and stores this information in a database. This data is frequently updated to reflect the user's latest preferences. The server also periodically collects and stores the user's travel history to analyze past travel patterns.

[1614] 3. Collection and analysis of local spot data

[1615] The server collects local spot data from the internet and various databases. During this process, natural language processing technology is used to analyze the collected text data (reviews, ratings, etc.) and classify the characteristics of each spot. Furthermore, machine learning algorithms are used to identify spots that match the user's preferences.

[1616] 4. Route generation

[1617] Based on the collected and analyzed data, the server considers the user's current location and destination to generate an optimal route tailored to the user's preferences. The generated route is designed to pass through scenic spots, highly-rated restaurants, and other points of interest.

[1618] 5. Send the route

[1619] The server sends the generated route information to the user's terminal. HTTP or WebSocket are used as the communication protocol.

[1620] 6. Route display and guidance

[1621] The device displays the route received from the server on a map application and provides visual and audio guidance to the user. For example, it can provide specific guidance such as, "There is a beautiful lake on your right 100 meters ahead, so please be sure to stop by." Furthermore, if a route change is necessary during travel, it can suggest a new route in real time.

[1622] 7. Gathering feedback and incorporating it into future proposals.

[1623] After completing a trip, users provide feedback within the app, including ratings and comments on the provided route. The device sends this feedback data to a server, which then updates the user's preference database based on this data. This updated data is then reflected in future route suggestions, providing a more personalized experience.

[1624] Specific example

[1625] Use Case 1: For users who want to enjoy nature

[1626] The user enters their preference for nature into the app.

[1627] Based on these preferences and past travel history, the server generates a route that includes parks and natural tourist attractions.

[1628] The server sends the generated route to the terminal.

[1629] The device provides route information to the user via map and voice guidance.

[1630] After the user has traveled, they provide feedback on the route through the app.

[1631] The server collects feedback and incorporates it into the next route suggestion.

[1632] Use Case 2: For users who love food

[1633] Users input their preference for "discovering new restaurants" into the app.

[1634] Based on these preferences, the server generates a route that includes new, highly-rated restaurants.

[1635] The server sends the generated route to the terminal.

[1636] The device provides route information to the user via map and voice guidance.

[1637] After the user has visited the restaurant, they provide feedback on their experience through the app.

[1638] The server collects feedback and incorporates it into the next route suggestion.

[1639] Examples of input prompts for a generative AI model

[1640] "Please explain the specifications of the program that suggests the best routes for users who want to enjoy nature."

[1641] "Please explain how the navigation system works for users who want to discover new restaurants."

[1642] This allows the generating AI model to dynamically generate optimal routes based on the user's diverse preferences and travel history, and to generate a detailed description of the system that provides a rich travel experience.

[1643] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1644] Step 1:

[1645] The user launches the application and enters their preferences. When the user selects preferences such as "I like nature" or "I want to discover new restaurants," this input data is generated. Specifically, the user accesses the app's preference settings screen and taps on the options to set their preferences. This preference information is sent to the server in JSON format. The server parses this data and stores it in a database.

[1646] Input: User preference data (e.g., "I like nature," "I want to discover new restaurants")

[1647] Output: User preference information stored in the database

[1648] Step 2:

[1649] The device collects the user's movement history in real time. It uses GPS to acquire location information at regular intervals and sends it to the server. The server analyzes the received location information and stores the user's movement patterns in a database. Specifically, the device acquires location information every 10 seconds and sends it to the server using an HTTP request.

[1650] Input: User's real-time location information (e.g., latitude and longitude data)

[1651] Output: Movement history information stored in the database

[1652] Step 3:

[1653] The server collects local spot information from the internet and various databases. It analyzes the collected text data (such as reviews and ratings) using natural language processing technology and classifies the characteristics of each spot. For example, the server collects reviews from review sites via an API and calculates a spot rating score using a natural language processing algorithm. Furthermore, it uses machine learning algorithms to identify spots that match the user's preferences.

[1654] Input: Spot information collected from the internet and databases.

[1655] Output: Characteristic data and evaluation scores of the analyzed spots

[1656] Step 4:

[1657] The server generates the optimal route based on the user's current location, destination, preferences, and analyzed spot data. It uses algorithms to calculate paths between points and design routes that include spots tailored to the user's preferences. For example, the server might obtain the user's current location and destination and generate a route that includes parks and highly-rated restaurants. The generated route is sent to the terminal in JSON format.

[1658] Input: User's current location, destination, preference data, and location data.

[1659] Output: Optimal route data sent to the terminal

[1660] Step 5:

[1661] The device analyzes route information received from the server and displays it on the map application. Specifically, it analyzes the JSON data received by the device and renders the route information on the map screen. Furthermore, it initiates voice guidance and provides guidance to the user at designated points. For example, it might announce, "There is a highly-rated cafe at the next right turn."

[1662] Input: Optimal route data received from the server

[1663] Output: Route display and voice guidance on the map app.

[1664] Step 6:

[1665] After completing a trip, users provide feedback within the app, including ratings and comments on the provided route. The device sends this feedback to the server in JSON format. The server analyzes the feedback data and updates the user preference database. For example, a user might rate a route they experienced as "excellent" and enter specific comments.

[1666] Input: User feedback data (ratings and comments)

[1667] Output: Updated preference database

[1668] Step 7:

[1669] The server suggests new routes in real time if new conditions arise during travel. For example, when traffic congestion or road construction occurs, it receives new information and sends a recalculated route to the terminal, providing the user with the latest route information.

[1670] Input: Real-time traffic information, user's current location, destination

[1671] Output: Updated optimal route data

[1672] (Application Example 1)

[1673] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1674] Conventional navigation systems can suggest routes based on user preferences and tastes, but they are limited to real-world travel environments, making it difficult to provide access to remote locations or virtual sightseeing experiences. Furthermore, optimization based on travel history and preferences is insufficient, limiting their ability to provide a truly satisfying experience. Therefore, a new system is needed to offer a more enriching sightseeing experience.

[1675] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1676] In this invention, the server includes means for inputting user preferences, means for collecting the user's travel history, and means for collecting and analyzing local spot data. This makes it possible to generate an optimal virtual sightseeing route based on the user's preferences.

[1677] "User preferences" refer to elements that users are interested in or concerned with, and include categories such as historical buildings, nature, and shopping.

[1678] "User travel history" refers to information about places a user has visited and routes they have used in the past.

[1679] "Local spot data" refers to data that includes detailed information and evaluations of tourist destinations, facilities, restaurants, natural landscapes, and other similar locations.

[1680] "Methods for generating routes" refer to systems that design the optimal route based on user preferences and location data.

[1681] "Terminal" refers to a device used by a user, and includes smartphones, smart glasses, and head-mounted displays.

[1682] "Means of visual display" refers to a system that presents information visually through a screen or display.

[1683] "Virtual movement history" refers to a record of the movements a user has made in a virtual environment.

[1684] A "virtual sightseeing route" refers to a route that includes paths and spots for sightseeing in a virtual space.

[1685] "Means of collecting feedback" refers to a system for obtaining ratings and comments from users.

[1686] "Natural language processing" refers to computational techniques for understanding and analyzing human language.

[1687] "Machine learning" refers to the technology of learning patterns from data to perform predictions and classifications.

[1688] A "navigation terminal" refers to a device that provides route guidance, including voice guidance functionality.

[1689] This invention is a navigation system that provides an optimal virtual sightseeing experience based on the user's preferences and travel history. This system consists of the following various means.

[1690] 1. User's preferred input method

[1691] Users can input their travel preferences via devices such as smartphones or head-mounted displays. For example, this can be done by selecting categories such as historical buildings, natural landscapes, and shopping.

[1692] 2. Means for collecting user movement history

[1693] The device collects a history of places visited and routes traveled by the user during virtual sightseeing and sends this information to a server. This allows past virtual travel history to be stored in a database.

[1694] 3. Means for collecting and analyzing local spot data

[1695] The server retrieves tourist spot information, reviews, and image data from the internet and various databases, and analyzes this data using natural language processing and machine learning algorithms. As a result, the characteristics and ratings of each spot are classified.

[1696] 4. Means for generating routes

[1697] The server generates an optimal virtual sightseeing route based on the user's preferences and collected spot data. The generated route is optimized to include spots that the user is interested in.

[1698] 5. Means for transmitting route information to the terminal

[1699] The server sends the generated tourist route information to the user's device. This allows the user to receive route information in real time.

[1700] 6. Means by which the terminal visually displays and guides the route.

[1701] The device (smart glasses or head-mounted display) visually displays the received route information in a VR environment and provides specific guidance to the user. For example, it might say, "There's a famous historical site 300 meters ahead, so please stop by."

[1702] 7. Means of collecting user feedback

[1703] After users complete their virtual tour, they provide ratings and comments on the provided routes and spots. This feedback is sent from the device to the server and used to improve future route suggestions.

[1704] Specific example

[1705] Example 1: Virtual sightseeing to enjoy nature

[1706] If a user prefers "natural scenery," the server generates a sightseeing route based on the user's preferences, including natural spots such as mountains, lakes, and forests. The device visually displays this route in a VR environment and provides audio guidance. After completing the tour, the user sends feedback such as, "The scenery was very beautiful."

[1707] Example 2: Virtual sightseeing enjoying shopping in the city

[1708] If a user enjoys shopping, the server generates a sightseeing route based on the user's preferences, including shopping areas and highly-rated stores. The device visually displays this route in a VR environment, showcasing store information and reviews. After completing the tour, the user sends feedback such as "I discovered a new brand."

[1709] Example of a prompt

[1710] "Create an application that generates optimal virtual sightseeing routes based on user preferences. Use user preference data, tourist spot information, and rating data to suggest routes tailored to the user's preferences and visually display them in a VR environment. Implement a function to collect user feedback and incorporate it into future suggestions."

[1711] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1712] Step 1: Enter user preferences

[1713] Users input their travel preferences (e.g., historical buildings, natural landscapes, shopping) using devices such as smartphones or head-mounted displays. This input data is stored on the device and sent to the server in the next step.

[1714] Input: User preference data (historical buildings, natural landscapes, shopping, etc.)

[1715] Output: User preference data stored on the device

[1716] Step 2: Collect the user's movement history.

[1717] The device automatically collects the user's history of virtual tourist destinations and travel routes. This data is sent to the server as a list of the spots and routes the user has visited.

[1718] Input: Data on virtual tourist destinations and travel routes previously visited by the user.

[1719] Output: User movement history data sent to the server

[1720] Step 3: Collect and analyze local spot data.

[1721] The server collects spot information, reviews, and image data related to tourist destinations from the internet and various databases, and analyzes them using natural language processing technology and machine learning algorithms. As a result of the analysis, the characteristics and evaluations of each spot are classified.

[1722] Input: Information and reviews of tourist attractions collected from the internet and databases.

[1723] Output: Spot data categorized by features and evaluations.

[1724] Step 4: Route Generation

[1725] The server generates the optimal virtual sightseeing route based on the user's preferences and collected and analyzed spot data. This process prioritizes the types of spots the user wants to visit and optimizes the route accordingly.

[1726] Input: User preference data and categorized spot data

[1727] Output: Optimal virtual sightseeing route

[1728] Step 5: Send route information to the device.

[1729] The server sends the generated tourist route information to the user's device. This allows the user to receive route information in real time.

[1730] Input: Optimal virtual sightseeing route

[1731] Output: Route information sent to the terminal

[1732] Step 6: Visual representation and guidance of the route

[1733] The device (smart glasses or head-mounted display) visually displays the received route information in a VR environment and provides specific guidance to the user. For example, it might provide voice guidance such as, "There's a famous historical site 300 meters ahead, so please stop by."

[1734] Input: Received route information

[1735] Output: Visually displayed route and voice guidance in a VR environment

[1736] Step 7: Gathering Feedback

[1737] After the user completes their virtual tour, they enter their evaluation and comments on the provided route and spots. This feedback data is sent from the device to the server and used to improve future route suggestions.

[1738] Input: User feedback (ratings and comments on routes and spots)

[1739] Output: Feedback data sent to the server

[1740] Through these steps, the system can provide an optimal virtual sightseeing experience based on the user's preferences and travel history, thereby increasing user satisfaction.

[1741] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1742] This invention combines an emotion engine with a navigation system that suggests routes based on the user's preferences and travel history, in order to enhance the user's daily travel experience. This system can recognize the user's emotional state in real time and suggest routes that take this into consideration. The embodiments for carrying out this invention will be described in detail below.

[1743] 1. Overview of the entire system

[1744] This navigation system has the following main components:

[1745] 1. A means for users to input their preferences.

[1746] 2. Means for collecting user movement history

[1747] 3. Means for collecting and analyzing local spot data

[1748] 4. Means for generating routes based on user preferences and spot data

[1749] 5. Means for sending the generated route to the terminal

[1750] 6. Means by which the terminal displays and guides the route.

[1751] 7. A means of collecting user feedback and incorporating it into future route suggestions.

[1752] 8. A means of collecting emotional data using an emotion engine that recognizes user emotions.

[1753] 2. Collection and Analysis of User Data

[1754] The server inputs and stores the user's preferences (e.g., loves nature, wants to discover new restaurants) during the user's initial setup and through periodic visits. Furthermore, the server periodically collects and stores the user's travel history.

[1755] 3. Collection and analysis of local spot data

[1756] The server collects local spot data from the internet and various databases. During this process, it analyzes text data (such as reviews and ratings) using natural language processing and machine learning techniques to classify the characteristics and ratings of each spot.

[1757] 4. Route generation and sending

[1758] The server, based on collected and analyzed data, considers the user's current location and destination to generate an optimal route tailored to the user's preferences. The generated route includes scenic spots and highly-rated restaurants. This route information is then transmitted to the user's device.

[1759] 5. Route display and guidance

[1760] The device displays the route received from the server on a map application and provides visual and audio guidance to the user. For example, it provides specific guidance such as, "There is a beautiful lake on your right 100 meters ahead, so please be sure to stop by."

[1761] 6. Collection and Analysis of Emotional Data

[1762] When users are on the move or providing feedback on route guidance, the device uses an emotion engine to analyze the user's facial expressions and voice tone, collecting emotional data. Examples include reading the user's facial expressions through the camera and analyzing their voice tone through the microphone.

[1763] 7. Optimizing route suggestions based on emotional data

[1764] Based on emotional data collected by the server, the system analyzes the user's current emotional state and regenerates a route accordingly. For example, if the user is feeling stressed, it might suggest a route that takes them through a scenic park to help them relax.

[1765] 8. Gathering feedback and incorporating it into future proposals.

[1766] After a user completes a journey, they provide feedback within the app, including their evaluation and comments on the route. This feedback also includes the user's emotional state. The device sends this feedback data to a server, which analyzes the data and uses it to improve future route suggestions.

[1767] Specific example

[1768] Use Case 1: For users who want to enjoy nature

[1769] 1. The user enters their preference for nature into the app.

[1770] 2. Based on these preferences and past travel history, the server generates a route that includes parks and natural tourist attractions.

[1771] 3. The server sends the route it generated to the terminal.

[1772] 4. The device provides route information to the user via map and voice guidance.

[1773] 5. The emotion engine recognizes the user's emotions while they are on the move, and if it determines that they are feeling stressed, it suggests a new route that includes relaxation spots.

[1774] 6. After the user has traveled, they will rate the route and provide emotional feedback through the app.

[1775] 7. The server collects feedback and incorporates it into the next route suggestion.

[1776] Use Case 2: For users who love food

[1777] 1. The user enters their preference for "discovering new restaurants" into the app.

[1778] 2. Based on these preferences, the server generates a route that includes new, highly-rated restaurants.

[1779] 3. The server sends the route it generated to the terminal.

[1780] 4. The device provides route information to the user via map and voice guidance.

[1781] 5. The emotion engine recognizes the user's emotions while they are on the move, and if they express excitement or satisfaction, it will continue to suggest similar new locations.

[1782] 6. After visiting the restaurant, users rate it and provide sentimental feedback through the app.

[1783] 7. The server collects feedback and incorporates it into the next route suggestion.

[1784] Thus, the navigation system of the present invention can provide the optimal route based on the user's preferences and emotional state, transforming travel from a mere means of transportation into entertainment.

[1785] The following describes the processing flow.

[1786] Step 1:

[1787] The user installs the app and completes the initial setup. Here, they enter their preferences (e.g., they like nature, want to discover new restaurants, etc.) and permission settings.

[1788] Step 2:

[1789] The device collects user preference information and permission settings and sends them to the server. Examples of data sent include the user ID and preferences (e.g., "User ID: 12345, Preferences: Nature, Gourmet").

[1790] Step 3:

[1791] The server stores user preference data in a database. This data forms the basis for future route suggestions.

[1792] Step 4:

[1793] The device periodically sends the user's current location and movement history to the server. Examples of data sent include location information and time information (e.g., "User ID: 12345, Location: (35.6895, 139.6917), Time: 2023-10-01 08:00").

[1794] Step 5:

[1795] The server collects local spot data from the internet and various databases. This data includes information on parks, restaurants, tourist attractions, and more.

[1796] Step 6:

[1797] The server analyzes the collected regional spot data using natural language processing and machine learning techniques. This allows for the classification of the characteristics and evaluation of each spot.

[1798] Step 7:

[1799] The server matches user preference data with local spot data to generate the optimal route for the user. For example, a user who likes nature will be given a route that passes through nearby parks and natural tourist spots.

[1800] Step 8:

[1801] The server sends the generated route to the terminal. An example of the data sent is the route details (e.g., "Route: Starting point (35.6895, 139.6917) → Park (35.6850, 139.6920) → Destination (35.7000, 139.7000)").

[1802] Step 9:

[1803] The device displays route information on a map and provides visual and audio guidance to the user. For example, it might say, "There's a beautiful lake 100 meters ahead, so please stop by."

[1804] Step 10:

[1805] The user confirms the displayed route and begins moving according to the instructions. During the journey, the device will notify the user in real time if any changes or corrections to the route are necessary.

[1806] Step 11:

[1807] The device activates an emotion engine while in motion, analyzing the user's facial expressions and voice tone to collect emotional data. For example, it uses a camera for facial recognition and a microphone to analyze voice tone.

[1808] Step 12:

[1809] The device sends collected emotional data to the server. Examples of data sent include the user ID and emotional state (e.g., "User ID: 12345, Emotion: Stress").

[1810] Step 13:

[1811] The server analyzes emotional data and optimizes route suggestions based on the user's current emotional state. For example, if the user is feeling stressed, it will regenerate a route that includes relaxation spots.

[1812] Step 14:

[1813] The server sends the newly generated route to the terminal, and the terminal guides the user along it. For example, it might say, "There's a relaxation spot nearby; we recommend you stop by."

[1814] Step 15:

[1815] After users have traveled, they can rate and provide feedback on the route within the app. For example, they can enter comments such as, "This route was great," or "I loved the new park."

[1816] Step 16:

[1817] The device sends user feedback data to the server. Examples of data sent include user ID, rating, and comment (e.g., "User ID: 12345, Rating: 5, Comment: The new park was great").

[1818] Step 17:

[1819] The server analyzes feedback data, updates user preference and sentiment data, and incorporates these into future route suggestions. This improves the accuracy of the algorithm, enabling more personalized route suggestions.

[1820] Through these steps, this system can transform users' daily commutes into opportunities for new discoveries and enjoyment. By considering users' preferences and emotions and always providing the optimal route, the journey itself becomes an enjoyable experience.

[1821] (Example 2)

[1822] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1823] Traditional navigation systems viewed user travel merely as a means to an end, failing to consider individual user preferences and emotional states when suggesting routes. As a result, users often found it difficult to experience satisfaction, enjoyment, or relaxation during their journeys. Furthermore, current systems have limited ability to incorporate user feedback into future suggestions. Against this backdrop, there is a need for a navigation system that enhances the user's travel experience and provides optimal routes for each individual user.

[1824] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting user preferences, means for collecting the user's travel history, means for collecting and analyzing local spot data, means for generating a route based on the user's preferences and spot data, means for transmitting the generated route to a terminal, means for the terminal to display and guide the route, means for collecting user feedback and reflecting it in subsequent route suggestions, means for recognizing the user's emotional state and regenerating the route taking it into consideration, means for collecting user emotional data, and means for making route suggestions based on the user's emotional data. This makes it possible to reflect the user's preferences and emotional state in real time and enhance the travel experience.

[1825] "Means for inputting user preferences" refers to an interface and related functions for users to input their interests and preferences (e.g., liking nature, liking historical places, liking food, etc.).

[1826] "Means for collecting user movement history" refers to a function that periodically acquires historical data of routes and locations that a user has traveled in the past and transmits it to the system.

[1827] "Means for collecting and analyzing local spot data" refers to a function that collects spot data such as tourist destinations, restaurants, and parks related to a region from the internet and various databases, and then classifies and analyzes that data using natural language processing and machine learning techniques.

[1828] "Means for generating routes based on user preferences and spot data" refers to a function that combines user-inputted preferences with collected spot data to calculate and generate the optimal travel route.

[1829] "Means for sending generated routes to the terminal" refers to a function for sending travel route information generated on the server to the user's terminal.

[1830] "Means for a terminal to display and guide a route" refers to a function on the user's terminal that displays the received travel route on a map and provides visual and audio guidance.

[1831] "Means for collecting user feedback and reflecting it in future route suggestions" refers to a function that stores feedback data such as ratings and comments collected from users after their trip in the system and reflects it in subsequent route suggestions.

[1832] "Means for recognizing the user's emotional state and regenerating the route taking it into consideration" refers to a function that analyzes the user's emotions in real time and generates a more appropriate route based on that analysis.

[1833] "Means for collecting user emotional data" refers to functions that use cameras and microphones to analyze the user's facial expressions and voice tone in order to collect user emotional data.

[1834] "Methods for providing route suggestions based on user emotional data" refers to a function that uses collected emotional data to inform future route suggestions and optimizes routes according to the user's emotional state.

[1835] This invention is a navigation system that enhances the user's daily travel experience by suggesting routes based on the user's preferences and travel history, and further recognizing the user's emotional state in real time to suggest routes accordingly. Specific embodiments are described in detail below.

[1836] 1. Components of the entire system

[1837] This navigation system includes the following components:

[1838] A way to input user preferences

[1839] Means of collecting user movement history

[1840] A means of collecting and analyzing local spot data.

[1841] A means of generating routes based on user preferences and spot data.

[1842] A means of sending the generated route to the terminal.

[1843] Means by which the terminal displays and guides the route

[1844] A means of collecting user feedback and incorporating it into future route suggestions.

[1845] Means of recognizing a user's emotional state

[1846] Means of collecting user sentiment data

[1847] A method for providing route suggestions based on user sentiment data.

[1848] 2. Program processing and the hardware and software used

[1849] 2.1 Collection of User Data

[1850] The server inputs and stores user preferences in a database during initial setup and through periodic user interactions. It also periodically sends the user's movement history to the server. Specifically, if a user enters "I like nature" on the app's initial setup screen, the device sends this information to the server, which then stores it in the database. Movement history is obtained using the device's GPS function.

[1851] 2.2 Collection and Analysis of Local Spot Data

[1852] The server retrieves spot data from internet APIs and databases (e.g., Google Places API, Yelp API) and analyzes it using natural language processing (NLP) and machine learning techniques. This allows it to classify the characteristics and ratings of each spot. For example, it analyzes text reviews to quantify the spot's rating and stores it in the database.

[1853] 2.3 Route generation and transmission

[1854] The server obtains the user's current location and destination, and calculates and generates the optimal route based on the user's preferences and spot data. The route includes spots the user can enjoy (e.g., parks, cafes, tourist attractions, etc.). This information is sent to the device.

[1855] 2.4 Route display and guidance

[1856] The device displays route information received from the server on a map application (e.g., Google Maps, Apple Maps) and provides visual and audio guidance to the user. For example, it might say, "There's a beautiful lake 100 meters ahead on your right, so be sure to stop by."

[1857] 2.5 Collection and Analysis of Emotional Data

[1858] The device uses an emotion engine to analyze the user's facial expressions and voice tone, collecting emotional data. For example, it uses a camera to read the user's facial expressions and a microphone to analyze their voice tone. This data is transmitted to a server in real time.

[1859] 2.6 Route optimization based on emotional data

[1860] The server receives emotional data and regenerates the route to match the user's current emotional state. For example, if the user is feeling stressed, it will suggest a route that includes many relaxation spots. The new route information is then sent back to the device.

[1861] 2.7 Gathering Feedback and Incorporating it into Future Proposals

[1862] After traveling, users rate and comment on their route within the app. The device sends this feedback data to a server, which analyzes the data and uses it to improve future route suggestions. This continuously improves the user's travel experience.

[1863] 3. Specific Examples

[1864] Use Case 1: For users who want to enjoy nature

[1865] 1. The user enters "I like nature" into the app.

[1866] 2. The device sends this data to the server.

[1867] 3. The server generates the optimal route, including parks and natural tourist spots, and sends it to the terminal.

[1868] 4. The device displays the route on a map and provides voice guidance.

[1869] 5. If the server detects that the user is experiencing stress through the camera or microphone while traveling, it will generate a new route that includes relaxation spots and send it to the device.

[1870] 6. After the user has traveled, they can leave a rating in the app, such as "This route was very good."

[1871] 7. The device sends this feedback to the server, which will then be used to improve the next route suggestion.

[1872] Use Case 2: For users who love food

[1873] 1. The user enters "I want to discover a new restaurant" into the app.

[1874] 2. The device sends this data to the server.

[1875] 3. The server generates a route that includes new restaurants with high ratings and sends it to the terminal.

[1876] 4. The device displays the route on a map and provides voice guidance.

[1877] 5. If the server detects that the user is expressing satisfaction through the camera or microphone while on the move, it will suggest similar new locations.

[1878] 6. After the user has visited the restaurant, they can leave a review on the app, such as "This restaurant was very good."

[1879] 7. The device sends this feedback to the server, which will then be used to improve the next route suggestion.

[1880] Examples of prompts for generative AI models

[1881] "I love nature. Please suggest some recommended routes that include natural spots."

[1882] This system reflects the user's preferences and emotional state in real time, enriching the travel experience. This allows users to enjoy an experience that goes beyond mere transportation and becomes a form of entertainment.

[1883] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1884] Specific processing steps

[1885] Step 1: Collecting User Data

[1886] The server collects the initial user data.

[1887] Input: The user enters their preferences, such as "I like nature," on the initial setup screen of the app.

[1888] Specific action: The device sends this preference data to the server.

[1889] Data processing: The server receives this data and saves it to the database along with the user's ID.

[1890] Output: User preference data stored in the database.

[1891] Step 2: Collecting movement history

[1892] The device collects the user's movement history.

[1893] Input: The user launches the app and begins moving. The GPS function is enabled.

[1894] Specific operation: The device periodically obtains the user's current location via GPS and generates movement history data.

[1895] Data processing: Acquired current location data is saved in chronological order to construct a movement history.

[1896] Output: Send the constructed movement history data to the server.

[1897] Step 3: Collect and analyze local spot data.

[1898] The server collects and analyzes local spot data from the internet and databases.

[1899] Input: Spot data obtained from APIs or online databases.

[1900] Specific operation: The server uses the Google Places API and Yelp API to collect information about local spots (e.g., reviews, ratings, categories).

[1901] Data Processing: Using natural language processing (NLP) and machine learning, we analyze text data to extract the characteristics and evaluations of the locations.

[1902] Output: Store spot data classified by characteristics and evaluations in a database.

[1903] Step 4: Route Generation

[1904] The server generates the optimal route based on user preferences and spot data.

[1905] Input: User's current location, destination, user preference data, and location data.

[1906] Specific operation: The server uses an algorithm to calculate the optimal route between the user's current location and destination.

[1907] Data processing: Optimize the route to include spots that match your preferences.

[1908] Output: Sends optimized route data to the terminal.

[1909] Step 5: Route display and guidance

[1910] The terminal displays and guides the user through the generated route.

[1911] Input: Route data received from the server.

[1912] Specific actions: The device draws the route on the map app and provides voice guidance such as, "There is a beautiful lake on the right 100 meters ahead."

[1913] Data calculation: Calculate the timing of map display updates and voice guidance.

[1914] Output: A user interface that provides visual and auditory guidance.

[1915] Step 6: Collect and analyze emotional data

[1916] The device collects and analyzes user emotional data.

[1917] Input: User's facial expressions and voice.

[1918] Specific operation: The device uses its camera and microphone to collect the user's facial expressions and voice in real time.

[1919] Data processing: The emotion engine analyzes facial expressions and voice tone to determine the emotional state (e.g., happy, relaxed, stressed).

[1920] Output: Send the determined emotion data to the server.

[1921] Step 7: Route optimization based on emotional data

[1922] The server regenerates the route based on sentiment data.

[1923] Input: Emotional data sent from the device.

[1924] Specific operation: The server analyzes the user's emotional state and optimizes the route.

[1925] Data calculation: Re-evaluate and rearrange spots on the route based on emotional state.

[1926] Output: Sends optimized new route data to the terminal.

[1927] Step 8: Gathering feedback and incorporating it into future proposals.

[1928] The server collects user feedback and incorporates it into future route suggestions.

[1929] Input: Ratings and comments made by users within the app after moving.

[1930] Specific action: The device sends user feedback to the server.

[1931] Data processing: Analyze feedback data to extract preferences and areas of dissatisfaction.

[1932] Output: Saves feedback data to the database to help generate routes in the future.

[1933] In this way, by appropriately processing and analyzing the data collected at each step, it is possible to provide users with the optimal travel experience.

[1934] (Application Example 2)

[1935] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1936] Traditional navigation systems suggest optimal routes based on user preferences and travel history, but they fail to consider the user's real-time emotional state, making it difficult to provide a truly satisfying travel experience. This can lead to user stress and decreased satisfaction.

[1937] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1938] In this invention, the server includes means for inputting user preferences, means for collecting the user's travel history, means for collecting and analyzing local spot data, means for generating a route based on the user's preferences and spot data, means for transmitting the generated route to a terminal, means for the terminal to display and guide the user along the route, means for collecting user feedback and reflecting it in future route suggestions, means for recognizing the user's emotional state, and means for optimizing the route considering the user's emotional state. This makes it possible to suggest the optimal route according to the user's emotional state.

[1939] A "means for inputting user preferences" refers to an interface that allows users to communicate their interests and preferences to the system.

[1940] "Means for collecting user travel history" refers to a function that records data on places and routes that a user has traveled in the past.

[1941] "Methods for collecting and analyzing local spot data" refers to technologies for gathering information on tourist attractions, restaurants, and other locations within a specific region and then analyzing that information.

[1942] "Means for generating routes based on user preferences and spot data" refers to a mechanism for creating optimal travel routes based on user preferences and local spot data.

[1943] "Means for sending the generated route to the terminal" refers to communication means for transferring the created travel route to the terminal used by the user.

[1944] "Means for a terminal to display and guide a route" refers to a system that visually displays the travel route on the user's terminal and provides directions via voice or text.

[1945] "Means for collecting user feedback and reflecting it in future route suggestions" refers to a system that collects evaluations and opinions from users after their trip and uses them to generate future routes.

[1946] "Means of recognizing a user's emotional state" refers to technology that analyzes a user's emotions and psychological state at a given time based on their facial expressions and tone of voice.

[1947] "Methods for optimizing routes while considering the user's emotional state" refers to a mechanism that recalculates the route to include relaxing places and interesting spots according to the user's current emotions.

[1948] This invention is a system that proposes and guides an autonomous vehicle along the optimal route, taking into account the user's preferences and emotional state. To realize this system, the following processes are performed.

[1949] System-wide configuration

[1950] This system primarily consists of a server, terminals, and users. The server collects and analyzes data and generates routes, while the terminals are responsible for route guidance and recognizing the user's emotional state. Users provide the system with their preferences and emotional state and follow the suggested route.

[1951] User data collection and analysis

[1952] The server collects data from the user's device during initial setup and periodic visits, allowing the user to input their preferences. The server also collects the user's travel history and matches it with their preferences. This data is used to analyze what kinds of places the user prefers to visit.

[1953] Collection and analysis of spot data

[1954] The server collects local spot data from the internet and various databases. This process utilizes natural language processing and machine learning techniques. The collected data is used to analyze the characteristics and evaluation of spots, identifying spots that match the user's preferences.

[1955] Route generation and sending

[1956] The server generates the optimal route, taking into account the user's current location, destination, preferences, and emotional state. This route is designed to include, for example, scenic spots or highly-rated restaurants. The generated route is sent to the device, which provides the user with visual and audio directions.

[1957] Recognition and optimization of emotional states

[1958] The device is equipped with an emotion engine that recognizes the user's emotional state in real time. It analyzes the user's facial expressions and voice tone through the camera and microphone to determine their current emotional state. For example, if the user is feeling stressed, the server will suggest a scenic route that will help them relax. This emotional data will also be used to generate routes for future trips.

[1959] Gathering and incorporating feedback

[1960] After a user's journey, they provide feedback on the route through their device, offering ratings and opinions. This feedback is sent to the server and used to improve future route generation. This allows for a greater understanding of user preferences and satisfaction.

[1961] Hardware and software used

[1962] Server: Performs data collection, analysis, and route generation.

[1963] Device: Recognizes the user's emotional state and provides route guidance.

[1964] Emotion Engine API: An API that analyzes a user's emotional state.

[1965] Navigation API: An API that generates routes based on the user's preferences and emotional state.

[1966] Feedback API: An API for collecting user feedback.

[1967] Specific example

[1968] For users who want to enjoy nature:

[1969] 1. The user's preference is "I like nature."

[1970] 2. The autonomous vehicle will suggest routes that include relaxing parks and scenic spots.

[1971] 3. If the user experiences stress while in the vehicle, the system will re-suggest a route that includes relaxation spots.

[1972] For foodies:

[1973] 1. The user's preference is to "discover gourmet spots."

[1974] 2. The autonomous vehicle will suggest a route that includes highly-rated restaurants.

[1975] 3. If the user is satisfied, additional highly-rated restaurants will also be suggested.

[1976] Examples of prompts for generative AI models

[1977] Create a specification for an emotion-driven autonomous driving navigation system that suggests the optimal route in real time based on the user's preferences and emotional state. The navigation system must include the following elements:

[1978] A way to input user preferences

[1979] Means of collecting user movement history

[1980] A means of collecting and analyzing local spot data.

[1981] A means of generating routes based on user preferences and spot data.

[1982] A means of sending the generated route to the terminal.

[1983] Means by which the terminal displays and guides the route

[1984] A means of collecting user feedback and incorporating it into future route suggestions.

[1985] A method for collecting emotional data using an emotion engine that recognizes user emotions.

[1986] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1987] Step 1:

[1988] The server collects user preference data through a means of inputting user preferences. It receives information such as "I like nature" or "I want to discover new restaurants" as input. This input data is stored in a database and analyzed for use in subsequent processing.

[1989] Step 2:

[1990] The server collects data on the places and routes the user has visited, using means to collect the user's travel history. Based on this collected data, the server analyzes the user's behavior patterns and identifies individual preferences in more detail.

[1991] Step 3:

[1992] The server collects and analyzes local spot data, gathering information on tourist attractions, restaurants, and other locations from online databases and the internet. This data is then analyzed using natural language processing and machine learning techniques to classify the ratings and characteristics of each spot. Input data consists of text data about the spots, while output data is a list of spots with ratings.

[1993] Step 4:

[1994] The server calculates the optimal travel route for the user using means to generate routes based on the user's preferences and spot data. It uses the user's current location, destination, preferences, and collected spot data as input, and generates the optimal route as output.

[1995] Step 5:

[1996] The server transmits the calculated optimal route to the user's terminal (such as the display of an autonomous vehicle) through a means of sending the generated route to the terminal. The input data is the generated route information, and the output data is the route information sent to the terminal.

[1997] Step 6:

[1998] The terminal notifies the user of the route visually and audibly using means to display and guide them along the route. The input is route information sent from the server, and the output is visual and audible guidance to the user.

[1999] Step 7:

[2000] The device uses means to recognize the user's emotional state to acquire the user's facial expressions and voice tone through the camera and microphone. An emotion engine API analyzes this data to determine the user's current emotional state. The input data is the user's emotional data, and the output data is the analyzed emotional state.

[2001] Step 8:

[2002] The server uses methods to optimize routes by considering the user's emotional state, and recalculates a new route if the user is experiencing stress. The input is the user's current emotional state and existing route information, and the output is the recalculated new route information.

[2003] Step 9:

[2004] After completing their journey, users provide feedback on the route via their device. This feedback is sent to the server as ratings and comments. The input data is user feedback, and the output data is the feedback data stored on the server.

[2005] Step 10:

[2006] The server analyzes the collected feedback to improve future route suggestions, thereby enhancing user preferences and satisfaction. Input data consists of user feedback, while output data provides information for improving the route generation algorithm for future routes.

[2007] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2008] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2009] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2010] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using t...

Claims

1. A means for users to input their preferences, Means for collecting users' movement history, A means of collecting and analyzing local spot data, A means of generating routes based on user preferences and spot data, A means of sending the generated route to the terminal, A means by which the terminal displays and guides the route, A system that includes means for collecting user feedback and incorporating it into future route suggestions.

2. The system according to claim 1, comprising means for collecting and classifying local spot data using natural language processing and machine learning.

3. The system according to claim 1, further comprising means for providing voice guidance to a user via a terminal that performs navigation based on a generated route.

Images