system

Abstract

We provide the system. [Solution] A means for receiving travel plan information entered by the user, Means for generating an initial travel plan based on the aforementioned travel plan information, A means of monitoring weather information, traffic information, and tourist information in real time while traveling, A generative artificial intelligence means for generating an alternative travel plan based on the changes in circumstances detected by the monitoring means, Means for notifying the user of the aforementioned alternative travel plan, A means for allowing the user to choose whether or not to accept an alternative travel plan, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In travel and tourism, it is often difficult to visit the destination as planned. In particular, a user's travel plan may be greatly affected by unexpected weather changes, traffic troubles, temporary closures of tourist attractions, etc. When the plan collapses due to these unpredictable situations, a precious opportunity for the user becomes a regrettable memory, and the satisfaction of the travel also decreases. Current travel planning applications and systems lack the function of flexibly responding to real - time situation changes and providing appropriate alternatives.

Means for Solving the Problems

[0005] This invention provides means for receiving travel plan information entered by a user and generating an initial travel plan. Furthermore, it provides means for monitoring weather, traffic, and tourist information in real time during the trip and detecting changes in circumstances based on this information. Using generative artificial intelligence (AI) means, it generates an alternative travel plan based on the detected changes in circumstances and notifies the user. The user can review and select the notified alternative travel plan. By generating a new schedule and notifying the user again, the system maximizes support for the user's travel experience. This system makes it possible to flexibly respond to unexpected situations during travel and provide a better travel experience.

[0006] "Travel plan information" refers to information entered by the user, such as the travel destination, planned dates of visit, desired experiences, and mode of transportation.

[0007] The "initial travel plan" is the initial schedule generated by the server based on the user's travel plan information.

[0008] "Real-time" means acquiring, processing, and monitoring information instantly in the present moment.

[0009] "Weather information" refers to data on the current and predicted weather conditions at your travel destination.

[0010] "Traffic information" refers to data on current and predicted traffic conditions related to your destination and route during your trip.

[0011] "Tourism information" refers to data such as opening and closing times and congestion levels of tourist attractions at the destination.

[0012] "Monitoring means" refers to functions for collecting and analyzing weather information, traffic information, and tourism information in real time.

[0013] "Changes in circumstances" refer to changes that could affect the plan, such as weather fluctuations, traffic congestion, or temporary closures of tourist attractions.

[0014] The "generative AI means" is a function by AI technology for generating a new travel plan based on changes in the situation.

[0015] The "alternative travel plan" is a new schedule or a proposal for destinations generated to cope with changes in the situation.

[0016] The "notification means" is a function for notifying the user of the generated alternative travel plan.

[0017] The "selection means" is an interface for the user to choose whether to accept the alternative travel plan.

[0018] The "schedule" is a timetable showing the user's schedule during the trip.

[0019] The "travel experience" is a series of experiences and satisfactions that the user obtains during the trip.

Brief Explanation of Drawings

[0020] [Figure 1] It is a conceptual diagram showing an example of the configuration of the data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of the data processing device and the smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of the data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of the data processing device and the smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of the data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of the data processing device and the headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of the data processing system according to the fourth embodiment. [Figure 8]It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0021] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0022] First, the language used in the following description will be explained.

[0023] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Further, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0024] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0025] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0026] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0028] [First Embodiment]

[0029] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0030] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0033] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0036] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0040] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0041] This invention is a system that provides maximum support for the user's travel experience, and is characterized in particular by real-time monitoring and the provision of alternative plans to respond to unexpected situations during travel.

[0042] Overall system configuration

[0043] The system mainly consists of the following components:

[0044] User device: A device such as a smartphone or computer where the user enters their travel plans and receives notifications and alternative suggestions.

[0045] Server: A central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed.

[0046] Generative AI: This is a generative artificial intelligence (AI) embedded within a server that provides optimal alternative solutions based on real-time changes in the situation.

[0047] Operation on the user terminal

[0048] The user enters the following information into the terminal to plan a specific trip:

[0049] Travel destination

[0050] Scheduled visit date

[0051] Desired experiences and tourist destinations

[0052] means of transportation

[0053] The entered plan information is sent from the user terminal to the server.

[0054] Processing on the server

[0055] The server performs the following processes based on the received travel plan information:

[0056] To generate an initial travel plan, the system cross-references databases such as weather forecasts, traffic information, and tourist information to determine the optimal schedule and order of visits.

[0057] After the initial plan is generated, weather information, traffic information, and opening / closing status of tourist attractions are monitored in real time. This allows for the timely detection of unexpected weather changes, traffic problems, closures of tourist attractions, and other changes in circumstances.

[0058] Generating alternative plans using generative AI

[0059] The generative AI performs the following processes based on data from the server monitoring system:

[0060] If a change in circumstances is detected, an alternative travel plan will be generated accordingly. This alternative travel plan will include the options best suited to the user's preferences and current situation.

[0061] Example: If the Louvre Museum is temporarily closed, we would suggest a shopping plan at the Musée d'Orsay or along the Champs-Élysées.

[0062] Notification and selection of alternative plans

[0063] The server notifies the user terminal of an alternative plan created by the generative AI. The user can choose whether or not to accept the notified alternative plan.

[0064] The device will display the new schedule as needed if the user accepts an alternative plan. Furthermore, it will support the user's travel experience by providing maps and navigation.

[0065] Specific Scenario

[0066] As an example, consider a user planning a trip to Paris:

[0067] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[0068] 2. The terminal sends the entered information to the server.

[0069] 3. The server generates an initial travel plan based on the received information and notifies the user.

[0070] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[0071] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[0072] 6. The server notifies the user of the generated alternative plan.

[0073] 7. The user selects to visit the Musée d'Orsay, and the device displays a new schedule.

[0074] This system allows users to make the most of their trip, even in the face of unexpected events. Real-time information monitoring and flexible responses powered by generative AI enrich the user's travel experience.

[0075] The following describes the processing flow.

[0076] Step 1:

[0077] The user enters travel plan information into the terminal, such as the travel destination, planned visit dates, desired experiences, and mode of transportation.

[0078] Step 2:

[0079] The terminal receives the entered travel plan information and sends it to the server.

[0080] Step 3:

[0081] The server analyzes the received travel plan information and compares it with databases containing weather forecasts, traffic information, and tourist information to generate an initial travel plan.

[0082] Step 4:

[0083] The server sends the initial travel plan to the user's terminal and notifies the user.

[0084] Step 5:

[0085] The user reviews their initial travel plan.

[0086] Step 6:

[0087] The server monitors weather, traffic, and tourist information for the travel destination in real time during the trip.

[0088] Step 7:

[0089] The server periodically retrieves the user's location information to understand their current status.

[0090] Step 8:

[0091] When the server detects a change in circumstances (for example, a sudden change in weather, traffic congestion, or temporary closure of a tourist attraction), it uses generative artificial intelligence (AI) to generate an alternative travel plan.

[0092] Step 9:

[0093] The server sends the generated alternative travel plan to the user's terminal and notifies the user.

[0094] Step 10:

[0095] The device displays alternative travel plans to the user, allowing them to choose whether or not to accept them.

[0096] Step 11:

[0097] The user reviews the alternative travel plan and chooses whether to accept it.

[0098] Step 12:

[0099] The device will display the new schedule if the user accepts an alternative travel plan.

[0100] Step 13:

[0101] The server provides the necessary information (maps, navigation, details of destinations, etc.) based on the new schedule.

[0102] Step 14:

[0103] The user continues their journey according to the new schedule. During this time, they can check real-time information (traffic conditions, weather forecast, etc.) on their device as needed.

[0104] This series of processes allows users to respond flexibly to unexpected situations and have a better travel experience.

[0105] (Example 1)

[0106] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0107] Current travel support systems struggle to respond quickly and appropriately to unexpected situations that may arise during travel, such as weather changes, transportation problems, or the closure of tourist attractions. As a result, this increases the stress and difficulty users experience when changing their travel plans, degrading the quality of their travel experience. Furthermore, the limited availability of alternative plans means that they cannot fully meet the diverse needs of users.

[0108] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0109] In this invention, the server includes means for receiving travel plan information entered by a user, means for generating an initial travel plan based on the travel plan information, means for monitoring weather information, traffic information, and tourist information in real time during the trip, generative artificial intelligence (AI) means for generating an alternative travel plan based on changes in circumstances detected by the monitoring means, means for notifying the user of the alternative travel plan, means for allowing the user to choose whether to accept the alternative travel plan, and if the user accepts the alternative travel plan, means for displaying the new schedule and providing maps and navigation information as necessary. This enables the user to respond quickly and appropriately to unexpected situations that occur during the trip, thereby improving the quality of the travel experience.

[0110] A "user" refers to an individual who uses the system to plan a trip and receives information and services provided by the system during their trip.

[0111] A "terminal" refers to a device such as a smartphone or personal computer used by the user, through which travel planning information is entered and notifications are received.

[0112] A "server" refers to a central system that receives travel plan information sent from user terminals, performs analysis, real-time monitoring, and generates and notifies users of alternative plans.

[0113] "Travel planning information" refers to information that users enter to plan their trip, such as the travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation.

[0114] "Initial travel plan" refers to the travel schedule generated by the server based on travel plan information entered by the user.

[0115] "Weather information" refers to information about the weather at the travel destination, which may affect your travel plans.

[0116] "Traffic information" refers to information about public transportation and road conditions between travel destinations.

[0117] "Tourism information" refers to all information related to users' tourism activities, such as opening and closing times of tourist attractions and event information.

[0118] "Generative artificial intelligence (AI)" refers to an AI system that has the function of generating optimal alternative travel plans in real time based on changing circumstances.

[0119] An "alternative travel plan" refers to a new travel plan generated by generative artificial intelligence (AI) to adapt to unexpected changes in circumstances, in addition to the initial travel plan.

[0120] "Notifications" refer to messages and alerts sent from the server to the user's terminal, including information about travel plans and alternative plans.

[0121] "Selection" refers to the action a user takes to decide whether or not to accept the alternative travel plan they have been notified about.

[0122] "New schedule" refers to the revised travel schedule that is generated when the user accepts an alternative travel plan.

[0123] "Maps" and "navigation information" refer to the directions and information necessary for users to travel to their next destination based on the new schedule.

[0124] This invention is a system designed to maximize the user's travel experience, particularly characterized by real-time monitoring and the provision of alternative plans to address unexpected situations during travel. The system consists of user terminals, a server, and generative artificial intelligence (AI) components.

[0125] Operation on the user terminal

[0126] Users input travel plan information using their smartphones or computers (user terminals). Input fields include travel destination, planned visit dates, desired experiences and sightseeing spots, and mode of transportation. Once the user enters this information into their terminal and presses the submit button, the information is transferred to the server.

[0127] Processing on the server

[0128] The server receives travel plan information sent from the user's terminal and begins analysis. The server generates an optimal initial travel plan by cross-referencing various databases, including weather forecasts, traffic information, and opening / closing information for tourist attractions. It also monitors weather, traffic, and tourist information in real time during the trip, detecting changes in conditions in a timely manner. The databases used include, for example, weather forecast APIs for weather information and public transport APIs for traffic information.

[0129] Generating alternative plans using generative AI

[0130] Generative AI generates alternative travel plans based on changes in circumstances detected by the server. The AI ​​provides the optimal alternative plan, taking into account the user's preferences and current situation. Generative AI uses machine learning models, enabling it to respond quickly to unexpected situations.

[0131] Notification and selection of alternative plans

[0132] The server notifies the user of alternative plans created by generative AI. These notifications are sent via push notifications, email, or other methods. The user can choose whether or not to accept the alternative plan. If the user accepts the alternative plan, the device displays the new schedule as needed and also provides maps and navigation information, thereby supporting the user's travel experience.

[0133] Specific Scenario

[0134] As an example, consider a user planning a trip to Paris:

[0135] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[0136] 2. The terminal sends the entered information to the server.

[0137] 3. The server generates an initial travel plan based on the received information and notifies the user.

[0138] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[0139] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[0140] 6. The server notifies the user of the generated alternative plan.

[0141] 7. The user selects to visit the Musée d'Orsay, and the device displays a new schedule.

[0142] Example of a prompt

[0143] "Please plan and provide a trip to Paris."

[0144] "Please propose alternative plans in case the Louvre Museum closes."

[0145] "Please generate the optimal travel schedule based on current traffic and weather information."

[0146] This invention enables users to respond quickly and appropriately to unexpected situations that may arise during their travels, allowing them to enjoy a comfortable and fulfilling travel experience. Real-time information monitoring and flexible responses powered by generative AI enrich the user's travel experience.

[0147] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0148] Step 1: Collecting user input information

[0149] The user enters travel plan information into the terminal, including the travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation. The entered data is saved on the user's terminal as "travel plan information," and when the send button is pressed, it is transferred to the server.

[0150] Step 2: Transferring information to the server

[0151] The device sends the travel plan information entered by the user to the server via the internet. Specifically, it sends data to the server's API endpoint using an HTTP request. The input is "travel plan information," and the output is "data transmission to the server."

[0152] Step 3: Generating an initial travel plan

[0153] The server uses the received travel plan information to compare it with databases of weather forecasts, traffic information, and tourist information. Specifically, it queries the "Weather Forecast API," "Traffic Information API," and "Tourist Attraction Opening / Closing Information Database," among others. As a result, with "travel plan information" as input, an "initial travel plan" is generated as output, ready to be notified to the user.

[0154] Step 4: Real-time monitoring

[0155] The server monitors weather, traffic, and tourist information in real time while the generated initial travel plan is being implemented. Specifically, it periodically queries various APIs (e.g., weather forecast API, traffic information API) to obtain updated information. The input is the "initial travel plan," and the output is the update of "real-time monitoring data."

[0156] Step 5: Detecting changes in the situation

[0157] The server analyzes data acquired through real-time monitoring to detect changes in conditions such as weather fluctuations, traffic problems, and temporary closures of tourist attractions. Specifically, it runs an algorithm that compares the monitoring data with the initial travel plan and detects any discrepancies. The inputs are "real-time monitoring data" and the "initial travel plan," and the output is "detected changes in conditions."

[0158] Step 6: Generate an alternative plan

[0159] The generative AI generates alternative travel plans based on detected situational changes. Specifically, it uses a machine learning model to consider the user's preferences and current situation, and proposes the most suitable alternative. The input consists of "detected situational changes" and "prompts from the generative AI model," and the output is an "alternative travel plan."

[0160] Step 7: Notify the user

[0161] The server notifies the user's device of the alternative plan generated by the generative AI. Specifically, the user is provided with the new plan via push notification or email. The input is an "alternative travel plan," and the output is a "notification to the user."

[0162] Step 8: User selection and presentation of the final plan

[0163] The user reviews the notification and chooses whether to accept the alternative travel plan. The input is "Notification of Alternative Plan," and the new schedule is confirmed based on the user's selection.

[0164] The device displays a new schedule and provides map and navigation information once the user accepts an alternative plan. Specifically, it displays route information in conjunction with map and navigation apps. The input is "user selection," and the output provides "new schedule" and "navigation information."

[0165] The above outlines the specific processing steps of this system's program.

[0166] (Application Example 1)

[0167] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0168] In food delivery services, delivery status and traffic conditions can change in real time, potentially leading to delivery delays and a decline in service quality. Furthermore, users may become dissatisfied with the quality and speed of alternatives offered. To address this, a system is needed that monitors the situation in real time and quickly generates and presents appropriate alternatives.

[0169] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0170] In this invention, the server includes means for receiving plan information entered by the user, means for generating an initial plan based on the plan information, means for monitoring situation information in real time, generative artificial intelligence (AI) means for generating alternative plans, means for notifying the user of the alternative plans, and means for allowing the user to choose whether or not to accept the alternative plans. This makes it possible to respond to real-time changes in the situation and provide the user with a quick and optimal alternative.

[0171] A "user" refers to a person who uses the system.

[0172] "Planning information" refers to order and delivery information entered by the user.

[0173] "Means" refers to the methods or equipment used to achieve a particular objective.

[0174] "Initial plan" refers to the delivery schedule and route initially generated based on the information entered by the user.

[0175] "Real-time" refers to information and situations occurring at a given moment.

[0176] "Status information" refers to real-time information that affects deliveries, such as traffic conditions during delivery or the state of the kitchen.

[0177] "To monitor" refers to continuously observing a specific object and detecting changes in it.

[0178] "Changes in circumstances" refer to events that affect the delivery schedule, such as unexpected traffic congestion or changes in weather.

[0179] An "alternative plan" refers to a new delivery plan generated when circumstances arise that make the initial plan difficult to implement.

[0180] "Generative artificial intelligence (AI)" refers to artificial intelligence technology that generates optimal alternative solutions based on situational information.

[0181] "To notify" refers to conveying information or messages to a user.

[0182] "To allow a user to choose" means enabling the user to select one option from several choices.

[0183] The embodiments for carrying out this invention will be described in detail.

[0184] System Configuration

[0185] This system is primarily composed of three components: a server, a terminal, and generative artificial intelligence (AI).

[0186] 1. Server

[0187] The server plays a central role in receiving user-entered planning information and generating an initial plan. The server includes the following functions:

[0188] Function to receive and analyze planning information

[0189] A feature that monitors the situation in real time (traffic, weather, kitchen status).

[0190] A function that generates alternative plans in conjunction with generative AI.

[0191] A feature to notify users of alternative plans.

[0192] 2. Terminal

[0193] The terminal is a device used by the user to input planning information and receive notifications and alternatives. Generally, smartphones or tablets are used. Specific operations are as follows:

[0194] The user places an order and enters the delivery address and desired delivery time.

[0195] Receive notifications of initial and alternative plans from the server.

[0196] Choose whether to accept the alternative.

[0197] 3. Generative artificial intelligence (AI)

[0198] Generative AI is an AI model embedded within a server that generates optimal alternatives in response to real-time changes in the situation. Common AI models used include GPT-4® and BERT. The AI ​​performs the following processes:

[0199] Perform data analysis based on situational information.

[0200] Generate the optimal alternative in response to changing circumstances.

[0201] Create a prompt message and notify the user.

[0202] Hardware and software

[0203] hardware

[0204] Smartphones and tablets (user devices)

[0205] Servers (cloud-based is common)

[0206] software

[0207] Server software (e.g., Flask, Django)

[0208] Real-time monitoring systems (e.g., Geolocation API)

[0209] Generative AI (e.g. GPT-4, BERT)

[0210] Specific example

[0211] The following is a scenario in which a user orders a pizza:

[0212] 1. The user orders a pizza using a terminal and enters the delivery address and desired delivery time.

[0213] 2. The server receives this, generates an initial delivery plan, and notifies the user.

[0214] 3. If traffic congestion occurs during delivery, the server's real-time monitoring system will detect it.

[0215] 4. Generative AI generates optimal alternative routes and new delivery time suggestions based on changes in the situation.

[0216] 5. This alternative will be communicated to the user via the server, and the user will choose whether to accept it on their device.

[0217] Example of a prompt

[0218] "User ID: 123, pizza ordered. Traffic congestion occurred during delivery. Please offer the user an alternative: a restaurant discount coupon or another food item."

[0219] This embodiment of the invention makes it possible to respond immediately to real-time changes in circumstances and quickly provide users with the most suitable alternative. This, in turn, can improve the quality of food delivery services.

[0220] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0221] Step 1:

[0222] The user uses a terminal to enter order information and submits the product (e.g., pizza), delivery address, and desired delivery time.

[0223] Input: Product information, delivery address, desired delivery time

[0224] Output: Initial order data

[0225] Operation: Form input on the terminal

[0226] Step 2:

[0227] The terminal sends the order information entered by the user to the server.

[0228] Input: Initial order data

[0229] Output: Order data sent to the server

[0230] Operation: Sends data using HTTP requests

[0231] Step 3:

[0232] The server receives the initial order data and generates an initial delivery plan.

[0233] Input: Initial order data

[0234] Output: Initial delivery plan

[0235] Operation: A planning generation algorithm is executed to calculate the optimal delivery route and time.

[0236] Step 4:

[0237] The server notifies the terminal of the initial delivery plan and displays it to the user.

[0238] Input: Initial delivery plan

[0239] Output: Delivery plan notification displayed to the user

[0240] Operation: Sends data to the terminal via HTTP response.

[0241] Step 5:

[0242] It monitors situational information (traffic, weather, etc.) in real time and sends the data to the server.

[0243] Input: Real-time status information

[0244] Output: Updated status data

[0245] Operation: The monitoring system periodically collects data and sends it to the server.

[0246] Step 6:

[0247] The server analyzes real-time status information, and if a change in the situation is detected, it uses generative artificial intelligence (AI) to generate an alternative plan.

[0248] Input: Updated status data

[0249] Output: Alternative plan

[0250] Operation: An AI model (e.g., GPT-4) analyzes the input data and generates the optimal alternative.

[0251] Step 7:

[0252] The server notifies the terminal of the generated alternative plan along with a prompt message.

[0253] Input: Alternative plan, prompt statement

[0254] Output: Alternatives notified to the user

[0255] Operation: Sends notification data to the device via HTTP response.

[0256] Step 8:

[0257] The device displays an alternative plan notification to the user, who then chooses whether to accept the alternative.

[0258] Input: Alternative proposal notification

[0259] Output: User Selection

[0260] Operation: Selection behavior in the user interface

[0261] Step 9:

[0262] The user's selection is sent to the server, which then confirms and notifies the user of the final delivery plan based on the selection.

[0263] Input: User Selection

[0264] Output: Final delivery plan

[0265] Operation: Receive selected data, generate a new schedule, and send another notification.

[0266] This allows for immediate response to real-time changes in the food delivery process and enables the rapid provision of the most suitable alternative to the user.

[0267] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0268] This invention is a system that provides maximum support for the user's travel experience, and is characterized by its ability to flexibly respond to real-time changes in circumstances, as well as recognizing the user's emotions and adjusting alternatives and notification content accordingly.

[0269] Overall system configuration

[0270] The system consists of the following main components:

[0271] User device: A device such as a smartphone or computer where the user inputs travel plans and receives notifications and alternative suggestions. This includes cameras and microphones for emotion recognition.

[0272] Server: A central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed. It also incorporates an emotion engine.

[0273] Generative AI: This is a generative artificial intelligence (AI) embedded within a server that provides optimal alternative solutions based on real-time changes in the situation.

[0274] Emotion Engine: A system that recognizes a user's emotions through methods such as voice analysis and camera image analysis.

[0275] Operation on the user terminal

[0276] The user enters information into the device to plan their trip. Specifically, they enter the following information:

[0277] Travel destination

[0278] Scheduled visit date

[0279] Desired experiences and tourist destinations

[0280] means of transportation

[0281] The entered plan information is sent from the user terminal to the server.

[0282] Processing on the server

[0283] The server performs the following processes based on the received travel plan information:

[0284] Analyze the input travel plan information and generate an initial travel plan by referring to databases such as weather forecasts, traffic information, and tourism information. Furthermore, monitor real-time weather information, traffic information, opening and closing hours of tourist attractions, etc., and detect changes in the situation based on this information.

[0285] Periodically obtain the user's location information to grasp the current situation.

[0286] Generation of alternative plans by generative AI and emotion engine

[0287] Based on data from the server's monitoring system, the generative AI generates alternative travel plans according to changes in the situation. This generation process includes the following:

[0288] Propose new destinations or timings based on changes in the situation, for example, in case of bad weather or traffic congestion. Specifically, flexible responses such as changing from outdoor to indoor tourist attractions are possible.

[0289] The emotion engine analyzes data obtained from the user's camera and microphone to recognize the user's emotions. Based on this, if the user is feeling stressed, a relaxing plan is proposed, and if the user is excited, an active plan is proposed.

[0290] Example: If it is recognized from the user's voice or facial expression that they are feeling stressed, a break time at a café is proposed.

[0291] Notification and selection of alternative plans

[0292] The server notifies the user terminal of the alternative plan created by the generative AI. The notification content is also adjusted according to the user's emotions. For example, if the user is feeling stressed, the notification is made in a supportive tone.

[0293] The terminal displays the alternative plan to the user and provides an interface for the user to select whether to accept it.

[0294] The user reviews the notified alternative plan and chooses whether to accept it. If the user accepts the alternative plan, a new schedule is generated and displayed on the device.

[0295] Specific Scenario

[0296] As an example, consider a user planning a trip to Paris:

[0297] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[0298] 2. The terminal sends the entered information to the server.

[0299] 3. The server generates an initial travel plan based on the received information and notifies the user.

[0300] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[0301] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[0302] 6. The emotion engine recognizes the user's emotions from their voice and facial expressions, and adjusts the suggestions to match the user's mood. For example, if the user is tired, it will suggest an indoor cafe where they can rest.

[0303] 7. The server notifies the user of the adjusted alternative plan.

[0304] 8. The user reviews the alternative plan and chooses whether to accept it.

[0305] 9. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[0306] With this system, even in the face of unexpected situations, users can obtain a better travel experience by receiving optimal proposals according to their emotions. It is a collaboration between an emotion engine and generative AI that maximally supports users' travels.

[0307] The processing flow will be described below.

[0308] Step 1:

[0309] The user inputs travel plan information such as the destination of the trip, the planned visit date, the desired experiences, and the means of transportation into the terminal.

[0310] Step 2:

[0311] The terminal receives the input travel plan information and sends it to the server.

[0312] Step 3:

[0313] The server analyzes the received travel plan information and generates an initial travel plan by comparing it with databases such as weather forecasts, traffic information, and tourism information.

[0314] Step 4:

[0315] The server sends the initial travel plan to the user's terminal and notifies the user.

[0316] Step 5:

[0317] The user checks the initial travel plan.

[0318] Step 6:

[0319] The server monitors the real-time weather information, traffic information, and tourism information at the travel destination during the travel period.

[0320] Step 7:

[0321] The server periodically retrieves the user's location information to understand their current status.

[0322] Step 8:

[0323] The server acquires the user's voice and image data from the camera, and uses an emotion engine to recognize the user's emotions.

[0324] Step 9:

[0325] When the server detects a change in circumstances (for example, a sudden change in weather, traffic congestion, or temporary closure of a tourist attraction), it uses generative artificial intelligence (AI) to generate an alternative travel plan.

[0326] Step 10:

[0327] Generative AI optimizes alternative plans based on the user's current emotions. For example, if the user is feeling stressed, it will suggest a plan that helps them relax.

[0328] Step 11:

[0329] The server sends the generated alternative travel plan to the user's terminal and notifies the user.

[0330] Step 12:

[0331] The device displays alternative travel plans to the user, allowing them to choose whether or not to accept them.

[0332] Step 13:

[0333] The user reviews the alternative travel plan and chooses whether to accept it.

[0334] Step 14:

[0335] The device will display the new schedule if the user accepts an alternative travel plan.

[0336] Step 15:

[0337] The server provides the necessary information (maps, navigation, details of destinations, etc.) based on the new schedule.

[0338] Step 16:

[0339] The user continues their journey according to the new schedule. During this time, they can check real-time information (traffic conditions, weather forecast, etc.) on their device as needed.

[0340] This system allows users to respond flexibly to unexpected situations and receive optimal suggestions tailored to their emotions, resulting in a better travel experience.

[0341] (Example 2)

[0342] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0343] Conventional travel planning support systems can provide alternative plans in response to real-time changes in circumstances, but they have the problem of not being able to adjust plans while considering the user's emotional state. As a result, users may experience stress or be unable to respond appropriately to unexpected situations, making it difficult to provide the optimal travel experience.

[0344] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0345] In this invention, the server includes means for receiving travel plan information entered by the user, means for generating an initial travel plan based on the travel plan information, means for monitoring weather information, traffic information, and sightseeing information in real time during the trip, generative artificial intelligence (AI) means for generating an alternative travel plan based on changes in circumstances, emotion recognition means for recognizing the user's emotional state and adjusting the alternative travel plan based on those emotions, means for notifying the user of the alternative travel plan, and means for allowing the user to choose whether or not to accept the alternative travel plan. As a result, the user can not only respond flexibly to real-time changes in circumstances but also receive an optimal travel plan that takes into account their own emotional state.

[0346] "Means for receiving travel plan information entered by the user" refers to a system that allows users to enter travel plan information via a terminal and send that information to a server.

[0347] "Travel plan information" is a general term for the information that users input regarding their trip, such as destination, planned dates of visit, desired experiences and tourist attractions, and mode of transportation.

[0348] "Means for generating initial travel plans" refers to a system for creating an initial travel schedule based on the received travel plan information.

[0349] "Means for monitoring weather, traffic, and tourist information in real time while traveling" refers to a system for continuously acquiring and monitoring the latest weather forecasts, traffic conditions, opening / closing information for tourist destinations, etc.

[0350] "Generative artificial intelligence (AI) means for generating alternative travel plans based on changes in circumstances" refers to artificial intelligence technology for generating new travel plans based on changes in circumstances detected by monitoring means.

[0351] "An emotion recognition system that recognizes the user's emotional state and adjusts alternative travel plans based on those emotions" refers to a system that uses cameras and microphones to analyze the user's emotions from their facial expressions and voice, and adjusts the travel plan accordingly.

[0352] "Means of notifying users of alternative travel plans" refers to a mechanism for informing users of the generated alternative travel plans. This includes push notifications and alert messages to the device.

[0353] "A means of allowing users to choose whether or not to accept alternative travel plans" refers to a mechanism that provides an interface where users can review proposed alternative travel plans and choose whether or not to accept them.

[0354] This invention is a system that provides maximum support for the user's travel experience, characterized by its ability to flexibly respond to real-time changes in circumstances and to recognize the user's emotions to adjust alternatives and notification content.

[0355] Overall system configuration

[0356] The system consists of the following main components:

[0357] User device: A device such as a smartphone or computer where the user inputs travel plans and receives notifications and alternative suggestions. This includes cameras and microphones for emotion recognition.

[0358] Server: A central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed. It also incorporates an emotion engine.

[0359] Generative AI: This is a generative artificial intelligence (AI) embedded within a server that provides optimal alternative solutions based on real-time changes in the situation.

[0360] Emotion Engine: A system that recognizes a user's emotions through methods such as voice analysis and camera image analysis.

[0361] Operation on the user terminal

[0362] The user enters information into the device to plan their trip. Specifically, they enter the following information:

[0363] Travel destination

[0364] Scheduled visit date

[0365] Desired experiences and tourist destinations

[0366] means of transportation

[0367] The entered plan information is sent from the user's terminal to the server. For example, when a user enters their destination and travel dates into a smartphone application and taps the "Send" button, the information is sent to the server.

[0368] Processing on the server

[0369] The server performs the following processes based on the received travel plan information:

[0370] The system analyzes the entered travel plan information and generates an initial travel plan by referencing databases such as weather forecasts, traffic information, and tourist information. For example, it retrieves weather information for specific days based on the travel dates and checks the operating status of tourist destinations to be visited.

[0371] The system monitors weather information, traffic information, and opening / closing status information for tourist attractions in real time, and detects changes in the situation based on this information. The server periodically updates this information via an API.

[0372] The system periodically acquires the user's location information to understand their current situation. Specifically, it collects GPS data from the device and sends it to the server.

[0373] Generating alternative plans using generative AI and emotion engines

[0374] The generative AI generates alternative travel plans based on data from the server's monitoring system, adapting to changing circumstances. This generation process includes:

[0375] Based on changing circumstances, the system suggests alternative destinations and timings, for example, in case of bad weather or traffic congestion. Specifically, it can flexibly adapt by changing from outdoor to indoor tourist attractions. For instance, if a planned outdoor activity is canceled due to rain, the system can generate a plan to visit a nearby museum.

[0376] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this, it suggests relaxing plans if the user is stressed, and active plans if they are excited. For example, if the system recognizes that the user is stressed based on their voice and facial expressions, it will suggest taking a break at a cafe.

[0377] Notification and selection of alternative plans

[0378] The server notifies the user's terminal of alternative plans created by generative AI. The content of the notification is also adjusted according to the user's emotions. For example, if the user is feeling stressed, the notification will be in a supportive tone.

[0379] The device provides an interface that displays alternative plans to the user and allows them to choose whether to accept or reject them. It displays push notifications and alert messages, allowing the user to tap to view details. For example, tapping a push notification displays detailed alternative plans and "Accept" or "Reject" buttons.

[0380] The user reviews the notified alternative plan and chooses whether to accept it. If the user accepts the alternative plan, the new schedule will be displayed on the device.

[0381] Specific example scenarios and prompt statements

[0382] As a specific scenario, let's consider a user planning a trip to Paris:

[0383] 1. The user plans a trip to Paris using their smartphone and wishes to visit the Louvre Museum.

[0384] 2. The terminal sends the entered information to the server.

[0385] 3. The server generates an initial travel plan based on the received information and notifies the user.

[0386] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[0387] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[0388] 6. The emotion engine recognizes the user's emotions from their voice and facial expressions, and adjusts the suggestions to match the user's feelings. For example, if the user is tired, it will suggest taking a break at a cafe.

[0389] 7. The server notifies the user of the adjusted alternative plan.

[0390] 8. The user reviews the alternative plan and chooses whether to accept it.

[0391] 9. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[0392] Example of a prompt

[0393] "I'm planning a trip to Paris, and I'd like some recommendations for indoor places to enjoy myself if the weather turns bad."

[0394] This system allows users to respond flexibly to unexpected situations while receiving optimal suggestions tailored to their individual emotions. The collaboration of an emotion engine and generative AI provides maximum support for the user's travels.

[0395] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0396] Step 1:

[0397] The user enters travel plan information into their device. This information includes destination, planned visit dates, desired experiences or attractions, and mode of transportation. After entering the information, tapping the "Submit" button saves the information to the device. This action prepares the entered information as data for use in the next processing step.

[0398] Step 2:

[0399] The terminal receives travel plan information entered by the user and sends it to the server. Specifically, the terminal uses its internet connection to send the user's entered travel plan information (destination, planned visit dates, desired experiences or sightseeing spots, and mode of transportation) to the server. After transmission, the server analyzes the received data.

[0400] Step 3:

[0401] The server generates an initial travel plan based on the received travel plan information. It takes travel plan information as input data and generates an initial travel plan as a processing result. Specifically, the server refers to databases such as weather forecasts, traffic information, and tourist information to create an optimal plan based on the travel dates and destination. This initial travel plan is then ready to be notified to the user.

[0402] Step 4:

[0403] The server monitors weather, traffic, and tourist information in real time. It acquires various real-time data via APIs as input and evaluates that information. Specifically, the server periodically updates this information and has the functionality to detect changes in the situation. This allows for the proactive detection of potential problems that may occur during a trip.

[0404] Step 5:

[0405] The server periodically acquires the user's location information to understand their current situation. It receives GPS data from the device as input data to determine the user's current location. Specifically, the device periodically sends GPS information to the server, which analyzes this data to check the user's movement.

[0406] Step 6:

[0407] The generative AI generates alternative travel plans based on data monitored on a server, adapting to changing circumstances. It takes real-time information and user location data as input and uses this to generate alternative plans. Specific examples include flexible plan generation, such as changing outdoor activities to indoor activities if bad weather is predicted.

[0408] Step 7:

[0409] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. It takes audio and image data as input and processes it to recognize the user's emotional state. Specifically, it uses image recognition technology to analyze facial expressions and speech recognition technology to evaluate voice tone and volume. This allows it to respond in ways such as suggesting relaxing plans if the user is feeling stressed.

[0410] Step 8:

[0411] The server notifies the user's terminal of the alternative plan created by the generative AI. It takes the alternative plan as input data and generates notification data as the processing result. Specifically, it sends push notifications or alert messages to the terminal so that the user can check them.

[0412] Step 9:

[0413] The device provides an interface that displays alternative plans to the user and allows them to choose whether to accept or reject them. It receives the alternative plan notified as input data and generates an interface that the user can select as a result of processing. Specifically, when the push notification is tapped, the detailed alternative plan is displayed, and "Accept" or "Reject" buttons are shown.

[0414] Step 10:

[0415] The user reviews the notified alternative plan and chooses whether to accept it. The notification details of the alternative plan are reviewed as input data, and the selection result is entered into the device as processing result. Specifically, if the user accepts, they tap the "Accept" button, and the new schedule is displayed on the device.

[0416] (Application Example 2)

[0417] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0418] Conventional travel planning support systems can provide alternatives to accommodate changes in circumstances during a trip, but they lack consideration for the user's emotions when generating alternatives or adjusting notification content, leading to problems such as users easily experiencing stress and dissatisfaction. Furthermore, the alternative suggestions are often mechanical, making it difficult to fully support the user's travel experience.

[0419] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving travel plan information input by the user, means for generating an initial travel plan based on the travel plan information, means for monitoring weather information, traffic information, and sightseeing information during the trip in real time, generative artificial intelligence (AI) means for generating an alternative travel plan based on the situation changes detected by the monitoring means, means for notifying the user of the alternative travel plan, emotion engine means for recognizing the user's emotions, means for adjusting the alternative plan and notification content according to the user's emotions based on the emotion engine means, and means for allowing the user to choose whether or not to accept the alternative travel plan. This makes it possible to always provide the optimal alternative plan that takes the user's emotions into consideration, and for the user to have a more comfortable travel experience.

[0420] A "user" is a system user who inputs travel plan information, receives notifications of alternative options, and makes a selection.

[0421] "Travel plan information" refers to information entered by the user, such as the travel destination, planned dates of visit, desired experiences or sightseeing spots, and mode of transportation.

[0422] An "initial travel plan" refers to the basic schedule and activities of a trip, which are generated based on the travel plan information entered by the user.

[0423] "Real-time monitoring" means keeping weather, traffic, and tourist information constantly updated to the latest state during your trip.

[0424] "Changes in circumstances" refer to changes in weather conditions, traffic conditions, and the opening / closing status of tourist attractions that may occur during your trip.

[0425] An "alternative travel plan" is a preliminary or modified travel plan that is generated in response to changes in circumstances monitored in real time.

[0426] "Generative artificial intelligence (AI) methods" refer to artificial intelligence systems that generate travel plans and alternative options based on collected data.

[0427] An "emotion engine" is a system that analyzes the user's voice and camera images to recognize the user's emotions.

[0428] "Means for adjusting alternatives and notification content" refers to means for appropriately changing alternatives and notification content based on the user's emotions recognized by the emotion engine.

[0429] "Location information" refers to information that indicates the user's current geographical location.

[0430] A "new schedule" is a new travel schedule generated based on the alternative travel plan accepted by the user.

[0431] This invention is a system that provides maximum support for the user's travel experience, characterized by its ability to flexibly respond to real-time changes in circumstances, recognize the user's emotions, and adjust alternatives and notifications accordingly. The system consists of the following main components.

[0432] Overall system configuration

[0433] 1. User terminal: A device such as a smartphone or personal computer where the user inputs travel plans and receives notifications and alternative suggestions. This terminal includes a camera and microphone for emotion recognition.

[0434] 2. Server: This is the central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed. The server incorporates generative artificial intelligence (AI) and an emotion engine.

[0435] Operation on the user terminal

[0436] The user enters travel plan information into the terminal. This information includes the travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation. This entered plan information is then sent from the user's terminal to the server.

[0437] Processing on the server

[0438] The server performs the following processing based on the received travel plan information.

[0439] The system analyzes the entered travel plan information and generates an initial travel plan by referencing databases of weather forecasts, traffic information, and tourist information. Furthermore, it monitors weather information, traffic information, and opening / closing status of tourist attractions in real time and detects changes in the situation based on this information.

[0440] The system periodically acquires the user's location information to understand their current situation.

[0441] Generating alternative plans using generative AI and emotion engines

[0442] The generative AI generates alternative travel plans that adapt to changing circumstances, based on data from the server's monitoring system. This generation process includes the following:

[0443] Based on changing circumstances, for example, in the case of bad weather or traffic congestion, we can suggest new destinations or timings. Specifically, we can flexibly respond by changing from outdoor to indoor tourist attractions.

[0444] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this, it suggests relaxing plans if the user is stressed, and active plans if they are excited.

[0445] Notification and selection of alternative plans

[0446] The server notifies the user's terminal of an alternative plan created by a generative AI. The content of the notification is also adjusted according to the user's emotions. For example, if the user is feeling stressed, the notification will be in a supportive tone. The user's terminal provides an interface that displays the alternative plan to the user and allows them to choose whether to accept it or not. If the user accepts the alternative plan, a new schedule is generated and displayed on the terminal.

[0447] Specific Scenario

[0448] For users planning a trip to Paris, the process would be as follows:

[0449] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[0450] 2. The terminal sends the entered information to the server.

[0451] 3. The server generates an initial travel plan based on the received information and notifies the user.

[0452] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[0453] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[0454] 6. The emotion engine recognizes the user's emotions from their voice and facial expressions, and adjusts the suggestions to match the user's mood. For example, if the user is tired, it will suggest an indoor cafe where they can rest.

[0455] 7. The server notifies the user of the adjusted alternative plan.

[0456] 8. The user reviews the alternative plan and chooses whether to accept it.

[0457] 9. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[0458] This system allows users to receive optimal suggestions tailored to their emotions, even in the face of unexpected situations, resulting in a better travel experience. The collaboration between an emotion engine and generative AI provides maximum support for the user's travels.

[0459] Example of a prompt:

[0460] "I want to build an application that suggests the best travel plan based on the user's emotions and weather information. Please suggest the best alternative for the following travel plan: Destination: Paris, Planned visits: Louvre Museum, Weather: Rain, User's emotions: Stressed."

[0461] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0462] Step 1:

[0463] The user enters travel plan information into the terminal. Specifically, they enter their travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation. Based on the entered data, travel plan information is generated. This travel plan information is then sent from the terminal to the server.

[0464] Input: Travel destination, planned visit dates, desired experiences, sightseeing spots, mode of transportation

[0465] Output: Generation and transmission of travel plan information

[0466] Step 2:

[0467] The server generates an initial travel plan based on the received travel plan information. Specifically, it refers to databases such as weather forecasts, traffic information, and tourist information, and plans a schedule and destinations according to the entered information.

[0468] Input: Travel plan information

[0469] Output: Initial travel plan

[0470] Step 3:

[0471] The server monitors weather, traffic, and tourist information in real time. This monitoring process retrieves necessary information via APIs and stores it in a database. It also periodically retrieves user location information to understand their current situation. This information is then analyzed to detect changes in the situation.

[0472] Input: Current weather information, traffic information, tourist information, location information

[0473] Output: Latest status data

[0474] Step 4:

[0475] The server's generative artificial intelligence (AI) system generates alternative travel plans based on the latest acquired situational data. It flexibly adapts to changing circumstances, modifying the initial travel plan and suggesting new destinations and schedules. For example, it might change from outdoor to indoor tourist attractions in case of bad weather.

[0476] Input: Latest situation data, initial travel plan

[0477] Output: Alternative travel plan

[0478] Step 5:

[0479] The server's emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this recognition, it suggests a relaxing plan if the user is feeling stressed, and an active plan if they are feeling excited.

[0480] Input: Camera data, audio data

[0481] Output: Emotion recognition result

[0482] Step 6:

[0483] The server adjusts alternative plans and notifications based on the user's emotions, as recognized by the emotion engine. For example, if the user is feeling stressed, it will send a supportive notification and add suggestions for places to rest.

[0484] Input: Emotion recognition results, alternative travel plans

[0485] Output: Adjusted alternative travel plan, notification content

[0486] Step 7:

[0487] The server notifies the user terminal of the adjusted alternative travel plan. The terminal provides an interface that displays this alternative plan to the user and allows them to choose whether to accept it or not.

[0488] Input: Adjusted alternative travel plan, notification details

[0489] Output: Notification to the user

[0490] Step 8:

[0491] The user reviews the notified alternative plan and chooses whether to accept it. If the user accepts the alternative plan, the server generates a new schedule and sends it to the device. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[0492] Input: Select an alternative plan

[0493] Output: New schedule, displayed to the user.

[0494] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0495] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0496] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0497] [Second Embodiment]

[0498] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0499] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0500] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0501] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0502] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0503] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0504] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0505] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0506] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0507] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0508] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0509] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0510] This invention is a system that provides maximum support for the user's travel experience, and is characterized in particular by real-time monitoring and the provision of alternative plans to respond to unexpected situations during travel.

[0511] Overall system configuration

[0512] The system mainly consists of the following components:

[0513] User device: A device such as a smartphone or computer where the user enters their travel plans and receives notifications and alternative suggestions.

[0514] Server: A central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed.

[0515] Generative AI: This is a generative artificial intelligence (AI) embedded within a server that provides optimal alternative solutions based on real-time changes in the situation.

[0516] Operation on the user terminal

[0517] The user enters the following information into the terminal to plan a specific trip:

[0518] Travel destination

[0519] Scheduled visit date

[0520] Desired experiences and tourist destinations

[0521] means of transportation

[0522] The entered plan information is sent from the user terminal to the server.

[0523] Processing on the server

[0524] The server performs the following processes based on the received travel plan information:

[0525] To generate an initial travel plan, the system cross-references databases such as weather forecasts, traffic information, and tourist information to determine the optimal schedule and order of visits.

[0526] After the initial plan is generated, weather information, traffic information, and opening / closing status of tourist attractions are monitored in real time. This allows for the timely detection of unexpected weather changes, traffic problems, closures of tourist attractions, and other changes in circumstances.

[0527] Generating alternative plans using generative AI

[0528] The generative AI performs the following processes based on data from the server monitoring system:

[0529] If a change in circumstances is detected, an alternative travel plan will be generated accordingly. This alternative travel plan will include the options best suited to the user's preferences and current situation.

[0530] Example: If the Louvre Museum is temporarily closed, we would suggest a shopping plan at the Musée d'Orsay or along the Champs-Élysées.

[0531] Notification and selection of alternative plans

[0532] The server notifies the user terminal of an alternative plan created by the generative AI. The user can choose whether or not to accept the notified alternative plan.

[0533] The device will display the new schedule as needed if the user accepts an alternative plan. Furthermore, it will support the user's travel experience by providing maps and navigation.

[0534] Specific Scenario

[0535] As an example, consider a user planning a trip to Paris:

[0536] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[0537] 2. The terminal sends the entered information to the server.

[0538] 3. The server generates an initial travel plan based on the received information and notifies the user.

[0539] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[0540] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[0541] 6. The server notifies the user of the generated alternative plan.

[0542] 7. The user selects to visit the Musée d'Orsay, and the device displays a new schedule.

[0543] This system allows users to make the most of their trip, even in the face of unexpected events. Real-time information monitoring and flexible responses powered by generative AI enrich the user's travel experience.

[0544] The following describes the processing flow.

[0545] Step 1:

[0546] The user enters travel plan information into the terminal, such as the travel destination, planned visit dates, desired experiences, and mode of transportation.

[0547] Step 2:

[0548] The terminal receives the entered travel plan information and sends it to the server.

[0549] Step 3:

[0550] The server analyzes the received travel plan information and compares it with databases containing weather forecasts, traffic information, and tourist information to generate an initial travel plan.

[0551] Step 4:

[0552] The server sends the initial travel plan to the user's terminal and notifies the user.

[0553] Step 5:

[0554] The user reviews their initial travel plan.

[0555] Step 6:

[0556] The server monitors weather, traffic, and tourist information for the travel destination in real time during the trip.

[0557] Step 7:

[0558] The server periodically retrieves the user's location information to understand their current status.

[0559] Step 8:

[0560] When the server detects a change in circumstances (for example, a sudden change in weather, traffic congestion, or temporary closure of a tourist attraction), it uses generative artificial intelligence (AI) to generate an alternative travel plan.

[0561] Step 9:

[0562] The server sends the generated alternative travel plan to the user's terminal and notifies the user.

[0563] Step 10:

[0564] The device displays alternative travel plans to the user, allowing them to choose whether or not to accept them.

[0565] Step 11:

[0566] The user reviews the alternative travel plan and chooses whether to accept it.

[0567] Step 12:

[0568] The device will display the new schedule if the user accepts an alternative travel plan.

[0569] Step 13:

[0570] The server provides the necessary information (maps, navigation, details of destinations, etc.) based on the new schedule.

[0571] Step 14:

[0572] The user continues their journey according to the new schedule. During this time, they can check real-time information (traffic conditions, weather forecast, etc.) on their device as needed.

[0573] This series of processes allows users to respond flexibly to unexpected situations and have a better travel experience.

[0574] (Example 1)

[0575] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0576] Current travel support systems struggle to respond quickly and appropriately to unexpected situations that may arise during travel, such as weather changes, transportation problems, or the closure of tourist attractions. As a result, this increases the stress and difficulty users experience when changing their travel plans, degrading the quality of their travel experience. Furthermore, the limited availability of alternative plans means that they cannot fully meet the diverse needs of users.

[0577] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0578] In this invention, the server includes means for receiving travel plan information entered by a user, means for generating an initial travel plan based on the travel plan information, means for monitoring weather information, traffic information, and tourist information in real time during the trip, generative artificial intelligence (AI) means for generating an alternative travel plan based on changes in circumstances detected by the monitoring means, means for notifying the user of the alternative travel plan, means for allowing the user to choose whether to accept the alternative travel plan, and if the user accepts the alternative travel plan, means for displaying the new schedule and providing maps and navigation information as necessary. This enables the user to respond quickly and appropriately to unexpected situations that occur during the trip, thereby improving the quality of the travel experience.

[0579] A "user" refers to an individual who uses the system to plan a trip and receives information and services provided by the system during their trip.

[0580] A "terminal" refers to a device such as a smartphone or personal computer used by the user, through which travel planning information is entered and notifications are received.

[0581] A "server" refers to a central system that receives travel plan information sent from user terminals, performs analysis, real-time monitoring, and generates and notifies users of alternative plans.

[0582] "Travel planning information" refers to information that users enter to plan their trip, such as the travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation.

[0583] "Initial travel plan" refers to the travel schedule generated by the server based on travel plan information entered by the user.

[0584] "Weather information" refers to information about the weather at the travel destination, which may affect your travel plans.

[0585] "Traffic information" refers to information about public transportation and road conditions between travel destinations.

[0586] "Tourism information" refers to all information related to users' tourism activities, such as opening and closing times of tourist attractions and event information.

[0587] "Generative artificial intelligence (AI)" refers to an AI system that has the function of generating optimal alternative travel plans in real time based on changing circumstances.

[0588] An "alternative travel plan" refers to a new travel plan generated by generative artificial intelligence (AI) to adapt to unexpected changes in circumstances, in addition to the initial travel plan.

[0589] "Notifications" refer to messages and alerts sent from the server to the user's terminal, including information about travel plans and alternative plans.

[0590] "Selection" refers to the action a user takes to decide whether or not to accept the alternative travel plan they have been notified about.

[0591] "New schedule" refers to the revised travel schedule that is generated when the user accepts an alternative travel plan.

[0592] "Maps" and "navigation information" refer to the directions and information necessary for users to travel to their next destination based on the new schedule.

[0593] This invention is a system designed to maximize the user's travel experience, particularly characterized by real-time monitoring and the provision of alternative plans to address unexpected situations during travel. The system consists of user terminals, a server, and generative artificial intelligence (AI) components.

[0594] Operation on the user terminal

[0595] Users input travel plan information using their smartphones or computers (user terminals). Input fields include travel destination, planned visit dates, desired experiences and sightseeing spots, and mode of transportation. Once the user enters this information into their terminal and presses the submit button, the information is transferred to the server.

[0596] Processing on the server

[0597] The server receives travel plan information sent from the user's terminal and begins analysis. The server generates an optimal initial travel plan by cross-referencing various databases, including weather forecasts, traffic information, and opening / closing information for tourist attractions. It also monitors weather, traffic, and tourist information in real time during the trip, detecting changes in conditions in a timely manner. The databases used include, for example, weather forecast APIs for weather information and public transport APIs for traffic information.

[0598] Generating alternative plans using generative AI

[0599] Generative AI generates alternative travel plans based on changes in circumstances detected by the server. The AI ​​provides the optimal alternative plan, taking into account the user's preferences and current situation. Generative AI uses machine learning models, enabling it to respond quickly to unexpected situations.

[0600] Notification and selection of alternative plans

[0601] The server notifies the user of alternative plans created by generative AI. These notifications are sent via push notifications, email, or other methods. The user can choose whether or not to accept the alternative plan. If the user accepts the alternative plan, the device displays the new schedule as needed and also provides maps and navigation information, thereby supporting the user's travel experience.

[0602] Specific Scenario

[0603] As an example, consider a user planning a trip to Paris:

[0604] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[0605] 2. The terminal sends the entered information to the server.

[0606] 3. The server generates an initial travel plan based on the received information and notifies the user.

[0607] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[0608] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[0609] 6. The server notifies the user of the generated alternative plan.

[0610] 7. The user selects to visit the Musée d'Orsay, and the device displays a new schedule.

[0611] Example of a prompt

[0612] "Please plan and provide a trip to Paris."

[0613] "Please propose alternative plans in case the Louvre Museum closes."

[0614] "Please generate the optimal travel schedule based on current traffic and weather information."

[0615] This invention enables users to respond quickly and appropriately to unexpected situations that may arise during their travels, allowing them to enjoy a comfortable and fulfilling travel experience. Real-time information monitoring and flexible responses powered by generative AI enrich the user's travel experience.

[0616] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0617] Step 1: Collecting user input information

[0618] The user enters travel plan information into the terminal, including the travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation. The entered data is saved on the user's terminal as "travel plan information," and when the send button is pressed, it is transferred to the server.

[0619] Step 2: Transferring information to the server

[0620] The device sends the travel plan information entered by the user to the server via the internet. Specifically, it sends data to the server's API endpoint using an HTTP request. The input is "travel plan information," and the output is "data transmission to the server."

[0621] Step 3: Generating an initial travel plan

[0622] The server uses the received travel plan information to compare it with databases of weather forecasts, traffic information, and tourist information. Specifically, it queries the "Weather Forecast API," "Traffic Information API," and "Tourist Attraction Opening / Closing Information Database," among others. As a result, with "travel plan information" as input, an "initial travel plan" is generated as output, ready to be notified to the user.

[0623] Step 4: Real-time monitoring

[0624] The server monitors weather, traffic, and tourist information in real time while the generated initial travel plan is being implemented. Specifically, it periodically queries various APIs (e.g., weather forecast API, traffic information API) to obtain updated information. The input is the "initial travel plan," and the output is the update of "real-time monitoring data."

[0625] Step 5: Detecting changes in the situation

[0626] The server analyzes data acquired through real-time monitoring to detect changes in conditions such as weather fluctuations, traffic problems, and temporary closures of tourist attractions. Specifically, it runs an algorithm that compares the monitoring data with the initial travel plan and detects any discrepancies. The inputs are "real-time monitoring data" and the "initial travel plan," and the output is "detected changes in conditions."

[0627] Step 6: Generate an alternative plan

[0628] The generative AI generates alternative travel plans based on detected situational changes. Specifically, it uses a machine learning model to consider the user's preferences and current situation, and proposes the most suitable alternative. The input consists of "detected situational changes" and "prompts from the generative AI model," and the output is an "alternative travel plan."

[0629] Step 7: Notify the user

[0630] The server notifies the user's device of the alternative plan generated by the generative AI. Specifically, the user is provided with the new plan via push notification or email. The input is an "alternative travel plan," and the output is a "notification to the user."

[0631] Step 8: User selection and presentation of the final plan

[0632] The user reviews the notification and chooses whether to accept the alternative travel plan. The input is "Notification of Alternative Plan," and the new schedule is confirmed based on the user's selection.

[0633] The device displays a new schedule and provides map and navigation information once the user accepts an alternative plan. Specifically, it displays route information in conjunction with map and navigation apps. The input is "user selection," and the output provides "new schedule" and "navigation information."

[0634] The above outlines the specific processing steps of this system's program.

[0635] (Application Example 1)

[0636] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0637] In food delivery services, delivery status and traffic conditions can change in real time, potentially leading to delivery delays and a decline in service quality. Furthermore, users may become dissatisfied with the quality and speed of alternatives offered. To address this, a system is needed that monitors the situation in real time and quickly generates and presents appropriate alternatives.

[0638] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0639] In this invention, the server includes means for receiving plan information entered by the user, means for generating an initial plan based on the plan information, means for monitoring situation information in real time, generative artificial intelligence (AI) means for generating alternative plans, means for notifying the user of the alternative plans, and means for allowing the user to choose whether or not to accept the alternative plans. This makes it possible to respond to real-time changes in the situation and provide the user with a quick and optimal alternative.

[0640] A "user" refers to a person who uses the system.

[0641] "Planning information" refers to order and delivery information entered by the user.

[0642] "Means" refers to the methods or equipment used to achieve a particular objective.

[0643] "Initial plan" refers to the delivery schedule and route initially generated based on the information entered by the user.

[0644] "Real-time" refers to information and situations occurring at a given moment.

[0645] "Status information" refers to real-time information that affects deliveries, such as traffic conditions during delivery or the state of the kitchen.

[0646] "To monitor" refers to continuously observing a specific object and detecting changes in it.

[0647] "Changes in circumstances" refer to events that affect the delivery schedule, such as unexpected traffic congestion or changes in weather.

[0648] An "alternative plan" refers to a new delivery plan generated when circumstances arise that make the initial plan difficult to implement.

[0649] "Generative artificial intelligence (AI)" refers to artificial intelligence technology that generates optimal alternative solutions based on situational information.

[0650] "To notify" refers to conveying information or messages to a user.

[0651] "To allow a user to choose" means enabling the user to select one option from several choices.

[0652] The embodiments for carrying out this invention will be described in detail.

[0653] System Configuration

[0654] This system is primarily composed of three components: a server, a terminal, and generative artificial intelligence (AI).

[0655] 1. Server

[0656] The server plays a central role in receiving user-entered planning information and generating an initial plan. The server includes the following functions:

[0657] Function to receive and analyze planning information

[0658] A feature that monitors the situation in real time (traffic, weather, kitchen status).

[0659] A function that generates alternative plans in conjunction with generative AI.

[0660] A feature to notify users of alternative plans.

[0661] 2. Terminal

[0662] The terminal is a device used by the user to input planning information and receive notifications and alternatives. Generally, smartphones or tablets are used. Specific operations are as follows:

[0663] The user places an order and enters the delivery address and desired delivery time.

[0664] Receive notifications of initial and alternative plans from the server.

[0665] Choose whether to accept the alternative.

[0666] 3. Generative artificial intelligence (AI)

[0667] Generative AI is an AI model embedded within a server that generates optimal alternatives in response to real-time changes in the situation. Common AI models used include GPT-4 and BERT. The AI ​​performs the following processes:

[0668] Perform data analysis based on situational information.

[0669] Generate the optimal alternative in response to changing circumstances.

[0670] Create a prompt message and notify the user.

[0671] Hardware and software

[0672] hardware

[0673] Smartphones and tablets (user devices)

[0674] Servers (cloud-based is common)

[0675] software

[0676] Server software (e.g., Flask, Django)

[0677] Real-time monitoring systems (e.g., Geolocation API)

[0678] Generative AI (e.g. GPT-4, BERT)

[0679] Specific example

[0680] The following is a scenario in which a user orders a pizza:

[0681] 1. The user orders a pizza using a terminal and enters the delivery address and desired delivery time.

[0682] 2. The server receives this, generates an initial delivery plan, and notifies the user.

[0683] 3. If traffic congestion occurs during delivery, the server's real-time monitoring system will detect it.

[0684] 4. Generative AI generates optimal alternative routes and new delivery time suggestions based on changes in the situation.

[0685] 5. This alternative will be communicated to the user via the server, and the user will choose whether to accept it on their device.

[0686] Example of a prompt

[0687] "User ID: 123, pizza ordered. Traffic congestion occurred during delivery. Please offer the user an alternative: a restaurant discount coupon or another food item."

[0688] This embodiment of the invention makes it possible to respond immediately to real-time changes in circumstances and quickly provide users with the most suitable alternative. This, in turn, can improve the quality of food delivery services.

[0689] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0690] Step 1:

[0691] The user uses a terminal to enter order information and submits the product (e.g., pizza), delivery address, and desired delivery time.

[0692] Input: Product information, delivery address, desired delivery time

[0693] Output: Initial order data

[0694] Operation: Form input on the terminal

[0695] Step 2:

[0696] The terminal sends the order information entered by the user to the server.

[0697] Input: Initial order data

[0698] Output: Order data sent to the server

[0699] Operation: Sends data using HTTP requests

[0700] Step 3:

[0701] The server receives the initial order data and generates an initial delivery plan.

[0702] Input: Initial order data

[0703] Output: Initial delivery plan

[0704] Operation: A planning generation algorithm is executed to calculate the optimal delivery route and time.

[0705] Step 4:

[0706] The server notifies the terminal of the initial delivery plan and displays it to the user.

[0707] Input: Initial delivery plan

[0708] Output: Delivery plan notification displayed to the user

[0709] Operation: Sends data to the terminal via HTTP response.

[0710] Step 5:

[0711] It monitors situational information (traffic, weather, etc.) in real time and sends the data to the server.

[0712] Input: Real-time status information

[0713] Output: Updated status data

[0714] Operation: The monitoring system periodically collects data and sends it to the server.

[0715] Step 6:

[0716] The server analyzes real-time status information, and if a change in the situation is detected, it uses generative artificial intelligence (AI) to generate an alternative plan.

[0717] Input: Updated status data

[0718] Output: Alternative plan

[0719] Operation: An AI model (e.g., GPT-4) analyzes the input data and generates the optimal alternative.

[0720] Step 7:

[0721] The server notifies the terminal of the generated alternative plan along with a prompt message.

[0722] Input: Alternative plan, prompt statement

[0723] Output: Alternatives notified to the user

[0724] Operation: Sends notification data to the device via HTTP response.

[0725] Step 8:

[0726] The device displays an alternative plan notification to the user, who then chooses whether to accept the alternative.

[0727] Input: Alternative proposal notification

[0728] Output: User Selection

[0729] Operation: Selection behavior in the user interface

[0730] Step 9:

[0731] The user's selection is sent to the server, which then confirms and notifies the user of the final delivery plan based on the selection.

[0732] Input: User Selection

[0733] Output: Final delivery plan

[0734] Operation: Receive selected data, generate a new schedule, and send another notification.

[0735] This allows for immediate response to real-time changes in the food delivery process and enables the rapid provision of the most suitable alternative to the user.

[0736] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0737] This invention is a system that provides maximum support for the user's travel experience, and is characterized by its ability to flexibly respond to real-time changes in circumstances, as well as recognizing the user's emotions and adjusting alternatives and notification content accordingly.

[0738] Overall system configuration

[0739] The system consists of the following main components:

[0740] User device: A device such as a smartphone or computer where the user inputs travel plans and receives notifications and alternative suggestions. This includes cameras and microphones for emotion recognition.

[0741] Server: A central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed. It also incorporates an emotion engine.

[0742] Generative AI: This is a generative artificial intelligence (AI) embedded within a server that provides optimal alternative solutions based on real-time changes in the situation.

[0743] Emotion Engine: A system that recognizes a user's emotions through methods such as voice analysis and camera image analysis.

[0744] Operation on the user terminal

[0745] The user enters information into the device to plan their trip. Specifically, they enter the following information:

[0746] Travel destination

[0747] Scheduled visit date

[0748] Desired experiences and tourist destinations

[0749] means of transportation

[0750] The entered plan information is sent from the user terminal to the server.

[0751] Processing on the server

[0752] The server performs the following processes based on the received travel plan information:

[0753] The system analyzes the entered travel plan information and generates an initial travel plan by referencing databases of weather forecasts, traffic information, and tourist information. Furthermore, it monitors weather information, traffic information, and opening / closing status of tourist attractions in real time and detects changes in the situation based on this information.

[0754] The system periodically acquires the user's location information to understand their current situation.

[0755] Generating alternative plans using generative AI and emotion engines

[0756] The generative AI generates alternative travel plans based on data from the server's monitoring system, adapting to changing circumstances. This generation process includes:

[0757] Based on changing circumstances, for example, in the case of bad weather or traffic congestion, we can suggest new destinations or timings. Specifically, we can flexibly respond by changing from outdoor to indoor tourist attractions.

[0758] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this, it suggests relaxing plans if the user is stressed, and active plans if they are excited.

[0759] Example: If the system detects that the user is experiencing stress based on their voice or facial expressions, it might suggest that they take a break at a cafe.

[0760] Notification and selection of alternative plans

[0761] The server notifies the user's terminal of alternative plans created by generative AI. The content of the notification is also adjusted according to the user's emotions. For example, if the user is feeling stressed, the notification will be in a supportive tone.

[0762] The device provides an interface that displays alternative plans to the user and allows them to choose whether to accept or reject them.

[0763] The user reviews the notified alternative plan and chooses whether to accept it. If the user accepts the alternative plan, a new schedule is generated and displayed on the device.

[0764] Specific Scenario

[0765] As an example, consider a user planning a trip to Paris:

[0766] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[0767] 2. The terminal sends the entered information to the server.

[0768] 3. The server generates an initial travel plan based on the received information and notifies the user.

[0769] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[0770] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[0771] 6. The emotion engine recognizes the user's emotions from their voice and facial expressions, and adjusts the suggestions to match the user's mood. For example, if the user is tired, it will suggest an indoor cafe where they can rest.

[0772] 7. The server notifies the user of the adjusted alternative plan.

[0773] 8. The user reviews the alternative plan and chooses whether to accept it.

[0774] 9. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[0775] This system allows users to receive optimal suggestions tailored to their emotions, even in the face of unexpected situations, resulting in a better travel experience. The collaboration between an emotion engine and generative AI provides maximum support for the user's travels.

[0776] The following describes the processing flow.

[0777] Step 1:

[0778] The user enters travel plan information into the terminal, such as the travel destination, planned visit dates, desired experiences, and mode of transportation.

[0779] Step 2:

[0780] The terminal receives the entered travel plan information and sends it to the server.

[0781] Step 3:

[0782] The server analyzes the received travel plan information and compares it with databases containing weather forecasts, traffic information, and tourist information to generate an initial travel plan.

[0783] Step 4:

[0784] The server sends the initial travel plan to the user's terminal and notifies the user.

[0785] Step 5:

[0786] The user reviews their initial travel plan.

[0787] Step 6:

[0788] The server monitors weather, traffic, and tourist information for the travel destination in real time during the trip.

[0789] Step 7:

[0790] The server periodically retrieves the user's location information to understand their current status.

[0791] Step 8:

[0792] The server acquires the user's voice and image data from the camera, and uses an emotion engine to recognize the user's emotions.

[0793] Step 9:

[0794] When the server detects a change in circumstances (for example, a sudden change in weather, traffic congestion, or temporary closure of a tourist attraction), it uses generative artificial intelligence (AI) to generate an alternative travel plan.

[0795] Step 10:

[0796] Generative AI optimizes alternative plans based on the user's current emotions. For example, if the user is feeling stressed, it will suggest a plan that helps them relax.

[0797] Step 11:

[0798] The server sends the generated alternative travel plan to the user's terminal and notifies the user.

[0799] Step 12:

[0800] The device displays alternative travel plans to the user, allowing them to choose whether or not to accept them.

[0801] Step 13:

[0802] The user reviews the alternative travel plan and chooses whether to accept it.

[0803] Step 14:

[0804] The device will display the new schedule if the user accepts an alternative travel plan.

[0805] Step 15:

[0806] The server provides the necessary information (maps, navigation, details of destinations, etc.) based on the new schedule.

[0807] Step 16:

[0808] The user continues their journey according to the new schedule. During this time, they can check real-time information (traffic conditions, weather forecast, etc.) on their device as needed.

[0809] This system allows users to respond flexibly to unexpected situations and receive optimal suggestions tailored to their emotions, resulting in a better travel experience.

[0810] (Example 2)

[0811] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0812] Conventional travel planning support systems can provide alternative plans in response to real-time changes in circumstances, but they have the problem of not being able to adjust plans while considering the user's emotional state. As a result, users may experience stress or be unable to respond appropriately to unexpected situations, making it difficult to provide the optimal travel experience.

[0813] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0814] In this invention, the server includes means for receiving travel plan information entered by the user, means for generating an initial travel plan based on the travel plan information, means for monitoring weather information, traffic information, and sightseeing information in real time during the trip, generative artificial intelligence (AI) means for generating an alternative travel plan based on changes in circumstances, emotion recognition means for recognizing the user's emotional state and adjusting the alternative travel plan based on those emotions, means for notifying the user of the alternative travel plan, and means for allowing the user to choose whether or not to accept the alternative travel plan. As a result, the user can not only respond flexibly to real-time changes in circumstances but also receive an optimal travel plan that takes into account their own emotional state.

[0815] "Means for receiving travel plan information entered by the user" refers to a system that allows users to enter travel plan information via a terminal and send that information to a server.

[0816] "Travel plan information" is a general term for the information that users input regarding their trip, such as destination, planned dates of visit, desired experiences and tourist attractions, and mode of transportation.

[0817] "Means for generating initial travel plans" refers to a system for creating an initial travel schedule based on the received travel plan information.

[0818] "Means for monitoring weather, traffic, and tourist information in real time while traveling" refers to a system for continuously acquiring and monitoring the latest weather forecasts, traffic conditions, opening / closing information for tourist destinations, etc.

[0819] "Generative artificial intelligence (AI) means for generating alternative travel plans based on changes in circumstances" refers to artificial intelligence technology for generating new travel plans based on changes in circumstances detected by monitoring means.

[0820] "An emotion recognition system that recognizes the user's emotional state and adjusts alternative travel plans based on those emotions" refers to a system that uses cameras and microphones to analyze the user's emotions from their facial expressions and voice, and adjusts the travel plan accordingly.

[0821] "Means of notifying users of alternative travel plans" refers to a mechanism for informing users of the generated alternative travel plans. This includes push notifications and alert messages to the device.

[0822] "A means of allowing users to choose whether or not to accept alternative travel plans" refers to a mechanism that provides an interface where users can review proposed alternative travel plans and choose whether or not to accept them.

[0823] This invention is a system that provides maximum support for the user's travel experience, characterized by its ability to flexibly respond to real-time changes in circumstances and to recognize the user's emotions to adjust alternatives and notification content.

[0824] Overall system configuration

[0825] The system consists of the following main components:

[0826] User device: A device such as a smartphone or computer where the user inputs travel plans and receives notifications and alternative suggestions. This includes cameras and microphones for emotion recognition.

[0827] Server: A central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed. It also incorporates an emotion engine.

[0828] Generative AI: This is a generative artificial intelligence (AI) embedded within a server that provides optimal alternative solutions based on real-time changes in the situation.

[0829] Emotion Engine: A system that recognizes a user's emotions through methods such as voice analysis and camera image analysis.

[0830] Operation on the user terminal

[0831] The user enters information into the device to plan their trip. Specifically, they enter the following information:

[0832] Travel destination

[0833] Scheduled visit date

[0834] Desired experiences and tourist destinations

[0835] means of transportation

[0836] The entered plan information is sent from the user's terminal to the server. For example, when a user enters their destination and travel dates into a smartphone application and taps the "Send" button, the information is sent to the server.

[0837] Processing on the server

[0838] The server performs the following processes based on the received travel plan information:

[0839] The system analyzes the entered travel plan information and generates an initial travel plan by referencing databases such as weather forecasts, traffic information, and tourist information. For example, it retrieves weather information for specific days based on the travel dates and checks the operating status of tourist destinations to be visited.

[0840] The system monitors weather information, traffic information, and opening / closing status information for tourist attractions in real time, and detects changes in the situation based on this information. The server periodically updates this information via an API.

[0841] The system periodically acquires the user's location information to understand their current situation. Specifically, it collects GPS data from the device and sends it to the server.

[0842] Generating alternative plans using generative AI and emotion engines

[0843] The generative AI generates alternative travel plans based on data from the server's monitoring system, adapting to changing circumstances. This generation process includes:

[0844] Based on changing circumstances, the system suggests alternative destinations and timings, for example, in case of bad weather or traffic congestion. Specifically, it can flexibly adapt by changing from outdoor to indoor tourist attractions. For instance, if a planned outdoor activity is canceled due to rain, the system can generate a plan to visit a nearby museum.

[0845] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this, it suggests relaxing plans if the user is stressed, and active plans if they are excited. For example, if the system recognizes that the user is stressed based on their voice and facial expressions, it will suggest taking a break at a cafe.

[0846] Notification and selection of alternative plans

[0847] The server notifies the user's terminal of alternative plans created by generative AI. The content of the notification is also adjusted according to the user's emotions. For example, if the user is feeling stressed, the notification will be in a supportive tone.

[0848] The device provides an interface that displays alternative plans to the user and allows them to choose whether to accept or reject them. It displays push notifications and alert messages, allowing the user to tap to view details. For example, tapping a push notification displays detailed alternative plans and "Accept" or "Reject" buttons.

[0849] The user reviews the notified alternative plan and chooses whether to accept it. If the user accepts the alternative plan, the new schedule will be displayed on the device.

[0850] Specific example scenarios and prompt statements

[0851] As a specific scenario, let's consider a user planning a trip to Paris:

[0852] 1. The user plans a trip to Paris using their smartphone and wishes to visit the Louvre Museum.

[0853] 2. The terminal sends the entered information to the server.

[0854] 3. The server generates an initial travel plan based on the received information and notifies the user.

[0855] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[0856] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[0857] 6. The emotion engine recognizes the user's emotions from their voice and facial expressions, and adjusts the suggestions to match the user's feelings. For example, if the user is tired, it will suggest taking a break at a cafe.

[0858] 7. The server notifies the user of the adjusted alternative plan.

[0859] 8. The user reviews the alternative plan and chooses whether to accept it.

[0860] 9. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[0861] Example of a prompt

[0862] "I'm planning a trip to Paris, and I'd like some recommendations for indoor places to enjoy myself if the weather turns bad."

[0863] This system allows users to respond flexibly to unexpected situations while receiving optimal suggestions tailored to their individual emotions. The collaboration of an emotion engine and generative AI provides maximum support for the user's travels.

[0864] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0865] Step 1:

[0866] The user enters travel plan information into their device. This information includes destination, planned visit dates, desired experiences or attractions, and mode of transportation. After entering the information, tapping the "Submit" button saves the information to the device. This action prepares the entered information as data for use in the next processing step.

[0867] Step 2:

[0868] The terminal receives travel plan information entered by the user and sends it to the server. Specifically, the terminal uses its internet connection to send the user's entered travel plan information (destination, planned visit dates, desired experiences or sightseeing spots, and mode of transportation) to the server. After transmission, the server analyzes the received data.

[0869] Step 3:

[0870] The server generates an initial travel plan based on the received travel plan information. It takes travel plan information as input data and generates an initial travel plan as a processing result. Specifically, the server refers to databases such as weather forecasts, traffic information, and tourist information to create an optimal plan based on the travel dates and destination. This initial travel plan is then ready to be notified to the user.

[0871] Step 4:

[0872] The server monitors weather, traffic, and tourist information in real time. It acquires various real-time data via APIs as input and evaluates that information. Specifically, the server periodically updates this information and has the functionality to detect changes in the situation. This allows for the proactive detection of potential problems that may occur during a trip.

[0873] Step 5:

[0874] The server periodically acquires the user's location information to understand their current situation. It receives GPS data from the device as input data to determine the user's current location. Specifically, the device periodically sends GPS information to the server, which analyzes this data to check the user's movement.

[0875] Step 6:

[0876] The generative AI generates alternative travel plans based on data monitored on a server, adapting to changing circumstances. It takes real-time information and user location data as input and uses this to generate alternative plans. Specific examples include flexible plan generation, such as changing outdoor activities to indoor activities if bad weather is predicted.

[0877] Step 7:

[0878] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. It takes audio and image data as input and processes it to recognize the user's emotional state. Specifically, it uses image recognition technology to analyze facial expressions and speech recognition technology to evaluate voice tone and volume. This allows it to respond in ways such as suggesting relaxing plans if the user is feeling stressed.

[0879] Step 8:

[0880] The server notifies the user's terminal of the alternative plan created by the generative AI. It takes the alternative plan as input data and generates notification data as the processing result. Specifically, it sends push notifications or alert messages to the terminal so that the user can check them.

[0881] Step 9:

[0882] The device provides an interface that displays alternative plans to the user and allows them to choose whether to accept or reject them. It receives the alternative plan notified as input data and generates an interface that the user can select as a result of processing. Specifically, when the push notification is tapped, the detailed alternative plan is displayed, and "Accept" or "Reject" buttons are shown.

[0883] Step 10:

[0884] The user reviews the notified alternative plan and chooses whether to accept it. The notification details of the alternative plan are reviewed as input data, and the selection result is entered into the device as processing result. Specifically, if the user accepts, they tap the "Accept" button, and the new schedule is displayed on the device.

[0885] (Application Example 2)

[0886] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0887] Conventional travel planning support systems can provide alternatives to accommodate changes in circumstances during a trip, but they lack consideration for the user's emotions when generating alternatives or adjusting notification content, leading to problems such as users easily experiencing stress and dissatisfaction. Furthermore, the alternative suggestions are often mechanical, making it difficult to fully support the user's travel experience.

[0888] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving travel plan information input by the user, means for generating an initial travel plan based on the travel plan information, means for monitoring weather information, traffic information, and sightseeing information during the trip in real time, generative artificial intelligence (AI) means for generating an alternative travel plan based on the situation changes detected by the monitoring means, means for notifying the user of the alternative travel plan, emotion engine means for recognizing the user's emotions, means for adjusting the alternative plan and notification content according to the user's emotions based on the emotion engine means, and means for allowing the user to choose whether or not to accept the alternative travel plan. This makes it possible to always provide the optimal alternative plan that takes the user's emotions into consideration, and for the user to have a more comfortable travel experience.

[0889] A "user" is a system user who inputs travel plan information, receives notifications of alternative options, and makes a selection.

[0890] "Travel plan information" refers to information entered by the user, such as the travel destination, planned dates of visit, desired experiences or sightseeing spots, and mode of transportation.

[0891] An "initial travel plan" refers to the basic schedule and activities of a trip, which are generated based on the travel plan information entered by the user.

[0892] "Real-time monitoring" means keeping weather, traffic, and tourist information constantly updated to the latest state during your trip.

[0893] "Changes in circumstances" refer to changes in weather conditions, traffic conditions, and the opening / closing status of tourist attractions that may occur during your trip.

[0894] An "alternative travel plan" is a preliminary or modified travel plan that is generated in response to changes in circumstances monitored in real time.

[0895] "Generative artificial intelligence (AI) methods" refer to artificial intelligence systems that generate travel plans and alternative options based on collected data.

[0896] An "emotion engine" is a system that analyzes the user's voice and camera images to recognize the user's emotions.

[0897] "Means for adjusting alternatives and notification content" refers to means for appropriately changing alternatives and notification content based on the user's emotions recognized by the emotion engine.

[0898] "Location information" refers to information that indicates the user's current geographical location.

[0899] A "new schedule" is a new travel schedule generated based on the alternative travel plan accepted by the user.

[0900] This invention is a system that provides maximum support for the user's travel experience, characterized by its ability to flexibly respond to real-time changes in circumstances, recognize the user's emotions, and adjust alternatives and notifications accordingly. The system consists of the following main components.

[0901] Overall system configuration

[0902] 1. User terminal: A device such as a smartphone or personal computer where the user inputs travel plans and receives notifications and alternative suggestions. This terminal includes a camera and microphone for emotion recognition.

[0903] 2. Server: This is the central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed. The server incorporates generative artificial intelligence (AI) and an emotion engine.

[0904] Operation on the user terminal

[0905] The user enters travel plan information into the terminal. This information includes the travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation. This entered plan information is then sent from the user's terminal to the server.

[0906] Processing on the server

[0907] The server performs the following processing based on the received travel plan information.

[0908] The system analyzes the entered travel plan information and generates an initial travel plan by referencing databases of weather forecasts, traffic information, and tourist information. Furthermore, it monitors weather information, traffic information, and opening / closing status of tourist attractions in real time and detects changes in the situation based on this information.

[0909] The system periodically acquires the user's location information to understand their current situation.

[0910] Generating alternative plans using generative AI and emotion engines

[0911] The generative AI generates alternative travel plans that adapt to changing circumstances, based on data from the server's monitoring system. This generation process includes the following:

[0912] Based on changing circumstances, for example, in the case of bad weather or traffic congestion, we can suggest new destinations or timings. Specifically, we can flexibly respond by changing from outdoor to indoor tourist attractions.

[0913] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this, it suggests relaxing plans if the user is stressed, and active plans if they are excited.

[0914] Notification and selection of alternative plans

[0915] The server notifies the user's terminal of an alternative plan created by a generative AI. The content of the notification is also adjusted according to the user's emotions. For example, if the user is feeling stressed, the notification will be in a supportive tone. The user's terminal provides an interface that displays the alternative plan to the user and allows them to choose whether to accept it or not. If the user accepts the alternative plan, a new schedule is generated and displayed on the terminal.

[0916] Specific Scenario

[0917] For users planning a trip to Paris, the process would be as follows:

[0918] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[0919] 2. The terminal sends the entered information to the server.

[0920] 3. The server generates an initial travel plan based on the received information and notifies the user.

[0921] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[0922] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[0923] 6. The emotion engine recognizes the user's emotions from their voice and facial expressions, and adjusts the suggestions to match the user's mood. For example, if the user is tired, it will suggest an indoor cafe where they can rest.

[0924] 7. The server notifies the user of the adjusted alternative plan.

[0925] 8. The user reviews the alternative plan and chooses whether to accept it.

[0926] 9. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[0927] This system allows users to receive optimal suggestions tailored to their emotions, even in the face of unexpected situations, resulting in a better travel experience. The collaboration between an emotion engine and generative AI provides maximum support for the user's travels.

[0928] Example of a prompt:

[0929] "I want to build an application that suggests the best travel plan based on the user's emotions and weather information. Please suggest the best alternative for the following travel plan: Destination: Paris, Planned visits: Louvre Museum, Weather: Rain, User's emotions: Stressed."

[0930] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0931] Step 1:

[0932] The user enters travel plan information into the terminal. Specifically, they enter their travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation. Based on the entered data, travel plan information is generated. This travel plan information is then sent from the terminal to the server.

[0933] Input: Travel destination, planned visit dates, desired experiences, sightseeing spots, mode of transportation

[0934] Output: Generation and transmission of travel plan information

[0935] Step 2:

[0936] The server generates an initial travel plan based on the received travel plan information. Specifically, it refers to databases such as weather forecasts, traffic information, and tourist information, and plans a schedule and destinations according to the entered information.

[0937] Input: Travel plan information

[0938] Output: Initial travel plan

[0939] Step 3:

[0940] The server monitors weather, traffic, and tourist information in real time. This monitoring process retrieves necessary information via APIs and stores it in a database. It also periodically retrieves user location information to understand their current situation. This information is then analyzed to detect changes in the situation.

[0941] Input: Current weather information, traffic information, tourist information, location information

[0942] Output: Latest status data

[0943] Step 4:

[0944] The server's generative artificial intelligence (AI) system generates alternative travel plans based on the latest acquired situational data. It flexibly adapts to changing circumstances, modifying the initial travel plan and suggesting new destinations and schedules. For example, it might change from outdoor to indoor tourist attractions in case of bad weather.

[0945] Input: Latest situation data, initial travel plan

[0946] Output: Alternative travel plan

[0947] Step 5:

[0948] The server's emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this recognition, it suggests a relaxing plan if the user is feeling stressed, and an active plan if they are feeling excited.

[0949] Input: Camera data, audio data

[0950] Output: Emotion recognition result

[0951] Step 6:

[0952] The server adjusts alternative plans and notifications based on the user's emotions, as recognized by the emotion engine. For example, if the user is feeling stressed, it will send a supportive notification and add suggestions for places to rest.

[0953] Input: Emotion recognition results, alternative travel plans

[0954] Output: Adjusted alternative travel plan, notification content

[0955] Step 7:

[0956] The server notifies the user terminal of the adjusted alternative travel plan. The terminal provides an interface that displays this alternative plan to the user and allows them to choose whether to accept it or not.

[0957] Input: Adjusted alternative travel plan, notification details

[0958] Output: Notification to the user

[0959] Step 8:

[0960] The user reviews the notified alternative plan and chooses whether to accept it. If the user accepts the alternative plan, the server generates a new schedule and sends it to the device. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[0961] Input: Select an alternative plan

[0962] Output: New schedule, displayed to the user.

[0963] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0964] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0965] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0966] [Third Embodiment]

[0967] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0968] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0969] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0970] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0971] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0972] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0973] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0974] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0975] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0976] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0977] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0978] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0979] This invention is a system that provides maximum support for the user's travel experience, and is characterized in particular by real-time monitoring and the provision of alternative plans to respond to unexpected situations during travel.

[0980] Overall system configuration

[0981] The system mainly consists of the following components:

[0982] User device: A device such as a smartphone or computer where the user enters their travel plans and receives notifications and alternative suggestions.

[0983] Server: A central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed.

[0984] Generative AI: This is a generative artificial intelligence (AI) embedded within a server that provides optimal alternative solutions based on real-time changes in the situation.

[0985] Operation on the user terminal

[0986] The user enters the following information into the terminal to plan a specific trip:

[0987] Travel destination

[0988] Scheduled visit date

[0989] Desired experiences and tourist destinations

[0990] means of transportation

[0991] The entered plan information is sent from the user terminal to the server.

[0992] Processing on the server

[0993] The server performs the following processes based on the received travel plan information:

[0994] To generate an initial travel plan, the system cross-references databases such as weather forecasts, traffic information, and tourist information to determine the optimal schedule and order of visits.

[0995] After the initial plan is generated, weather information, traffic information, and opening / closing status of tourist attractions are monitored in real time. This allows for the timely detection of unexpected weather changes, traffic problems, closures of tourist attractions, and other changes in circumstances.

[0996] Generating alternative plans using generative AI

[0997] The generative AI performs the following processes based on data from the server monitoring system:

[0998] If a change in circumstances is detected, an alternative travel plan will be generated accordingly. This alternative travel plan will include the options best suited to the user's preferences and current situation.

[0999] Example: If the Louvre Museum is temporarily closed, we would suggest a shopping plan at the Musée d'Orsay or along the Champs-Élysées.

[1000] Notification and selection of alternative plans

[1001] The server notifies the user terminal of an alternative plan created by the generative AI. The user can choose whether or not to accept the notified alternative plan.

[1002] The device will display the new schedule as needed if the user accepts an alternative plan. Furthermore, it will support the user's travel experience by providing maps and navigation.

[1003] Specific Scenario

[1004] As an example, consider a user planning a trip to Paris:

[1005] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[1006] 2. The terminal sends the entered information to the server.

[1007] 3. The server generates an initial travel plan based on the received information and notifies the user.

[1008] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[1009] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[1010] 6. The server notifies the user of the generated alternative plan.

[1011] 7. The user selects to visit the Musée d'Orsay, and the device displays a new schedule.

[1012] This system allows users to make the most of their trip, even in the face of unexpected events. Real-time information monitoring and flexible responses powered by generative AI enrich the user's travel experience.

[1013] The following describes the processing flow.

[1014] Step 1:

[1015] The user enters travel plan information into the terminal, such as the travel destination, planned visit dates, desired experiences, and mode of transportation.

[1016] Step 2:

[1017] The terminal receives the entered travel plan information and sends it to the server.

[1018] Step 3:

[1019] The server analyzes the received travel plan information and compares it with databases containing weather forecasts, traffic information, and tourist information to generate an initial travel plan.

[1020] Step 4:

[1021] The server sends the initial travel plan to the user's terminal and notifies the user.

[1022] Step 5:

[1023] The user reviews their initial travel plan.

[1024] Step 6:

[1025] The server monitors weather, traffic, and tourist information for the travel destination in real time during the trip.

[1026] Step 7:

[1027] The server periodically retrieves the user's location information to understand their current status.

[1028] Step 8:

[1029] When the server detects a change in circumstances (for example, a sudden change in weather, traffic congestion, or temporary closure of a tourist attraction), it uses generative artificial intelligence (AI) to generate an alternative travel plan.

[1030] Step 9:

[1031] The server sends the generated alternative travel plan to the user's terminal and notifies the user.

[1032] Step 10:

[1033] The device displays alternative travel plans to the user, allowing them to choose whether or not to accept them.

[1034] Step 11:

[1035] The user reviews the alternative travel plan and chooses whether to accept it.

[1036] Step 12:

[1037] The device will display the new schedule if the user accepts an alternative travel plan.

[1038] Step 13:

[1039] The server provides the necessary information (maps, navigation, details of destinations, etc.) based on the new schedule.

[1040] Step 14:

[1041] The user continues their journey according to the new schedule. During this time, they can check real-time information (traffic conditions, weather forecast, etc.) on their device as needed.

[1042] This series of processes allows users to respond flexibly to unexpected situations and have a better travel experience.

[1043] (Example 1)

[1044] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1045] Current travel support systems struggle to respond quickly and appropriately to unexpected situations that may arise during travel, such as weather changes, transportation problems, or the closure of tourist attractions. As a result, this increases the stress and difficulty users experience when changing their travel plans, degrading the quality of their travel experience. Furthermore, the limited availability of alternative plans means that they cannot fully meet the diverse needs of users.

[1046] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1047] In this invention, the server includes means for receiving travel plan information entered by a user, means for generating an initial travel plan based on the travel plan information, means for monitoring weather information, traffic information, and tourist information in real time during the trip, generative artificial intelligence (AI) means for generating an alternative travel plan based on changes in circumstances detected by the monitoring means, means for notifying the user of the alternative travel plan, means for allowing the user to choose whether to accept the alternative travel plan, and if the user accepts the alternative travel plan, means for displaying the new schedule and providing maps and navigation information as necessary. This enables the user to respond quickly and appropriately to unexpected situations that occur during the trip, thereby improving the quality of the travel experience.

[1048] A "user" refers to an individual who uses the system to plan a trip and receives information and services provided by the system during their trip.

[1049] A "terminal" refers to a device such as a smartphone or personal computer used by the user, through which travel planning information is entered and notifications are received.

[1050] A "server" refers to a central system that receives travel plan information sent from user terminals, performs analysis, real-time monitoring, and generates and notifies users of alternative plans.

[1051] "Travel planning information" refers to information that users enter to plan their trip, such as the travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation.

[1052] "Initial travel plan" refers to the travel schedule generated by the server based on travel plan information entered by the user.

[1053] "Weather information" refers to information about the weather at the travel destination, which may affect your travel plans.

[1054] "Traffic information" refers to information about public transportation and road conditions between travel destinations.

[1055] "Tourism information" refers to all information related to users' tourism activities, such as opening and closing times of tourist attractions and event information.

[1056] "Generative artificial intelligence (AI)" refers to an AI system that has the function of generating optimal alternative travel plans in real time based on changing circumstances.

[1057] An "alternative travel plan" refers to a new travel plan generated by generative artificial intelligence (AI) to adapt to unexpected changes in circumstances, in addition to the initial travel plan.

[1058] "Notifications" refer to messages and alerts sent from the server to the user's terminal, including information about travel plans and alternative plans.

[1059] "Selection" refers to the action a user takes to decide whether or not to accept the alternative travel plan they have been notified about.

[1060] "New schedule" refers to the revised travel schedule that is generated when the user accepts an alternative travel plan.

[1061] "Maps" and "navigation information" refer to the directions and information necessary for users to travel to their next destination based on the new schedule.

[1062] This invention is a system designed to maximize the user's travel experience, particularly characterized by real-time monitoring and the provision of alternative plans to address unexpected situations during travel. The system consists of user terminals, a server, and generative artificial intelligence (AI) components.

[1063] Operation on the user terminal

[1064] Users input travel plan information using their smartphones or computers (user terminals). Input fields include travel destination, planned visit dates, desired experiences and sightseeing spots, and mode of transportation. Once the user enters this information into their terminal and presses the submit button, the information is transferred to the server.

[1065] Processing on the server

[1066] The server receives travel plan information sent from the user's terminal and begins analysis. The server generates an optimal initial travel plan by cross-referencing various databases, including weather forecasts, traffic information, and opening / closing information for tourist attractions. It also monitors weather, traffic, and tourist information in real time during the trip, detecting changes in conditions in a timely manner. The databases used include, for example, weather forecast APIs for weather information and public transport APIs for traffic information.

[1067] Generating alternative plans using generative AI

[1068] Generative AI generates alternative travel plans based on changes in circumstances detected by the server. The AI ​​provides the optimal alternative plan, taking into account the user's preferences and current situation. Generative AI uses machine learning models, enabling it to respond quickly to unexpected situations.

[1069] Notification and selection of alternative plans

[1070] The server notifies the user of alternative plans created by generative AI. These notifications are sent via push notifications, email, or other methods. The user can choose whether or not to accept the alternative plan. If the user accepts the alternative plan, the device displays the new schedule as needed and also provides maps and navigation information, thereby supporting the user's travel experience.

[1071] Specific Scenario

[1072] As an example, consider a user planning a trip to Paris:

[1073] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[1074] 2. The terminal sends the entered information to the server.

[1075] 3. The server generates an initial travel plan based on the received information and notifies the user.

[1076] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[1077] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[1078] 6. The server notifies the user of the generated alternative plan.

[1079] 7. The user selects to visit the Musée d'Orsay, and the device displays a new schedule.

[1080] Example of a prompt

[1081] "Please plan and provide a trip to Paris."

[1082] "Please propose alternative plans in case the Louvre Museum closes."

[1083] "Please generate the optimal travel schedule based on current traffic and weather information."

[1084] This invention enables users to respond quickly and appropriately to unexpected situations that may arise during their travels, allowing them to enjoy a comfortable and fulfilling travel experience. Real-time information monitoring and flexible responses powered by generative AI enrich the user's travel experience.

[1085] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1086] Step 1: Collecting user input information

[1087] The user enters travel plan information into the terminal, including the travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation. The entered data is saved on the user's terminal as "travel plan information," and when the send button is pressed, it is transferred to the server.

[1088] Step 2: Transferring information to the server

[1089] The device sends the travel plan information entered by the user to the server via the internet. Specifically, it sends data to the server's API endpoint using an HTTP request. The input is "travel plan information," and the output is "data transmission to the server."

[1090] Step 3: Generating an initial travel plan

[1091] The server uses the received travel plan information to compare it with databases of weather forecasts, traffic information, and tourist information. Specifically, it queries the "Weather Forecast API," "Traffic Information API," and "Tourist Attraction Opening / Closing Information Database," among others. As a result, with "travel plan information" as input, an "initial travel plan" is generated as output, ready to be notified to the user.

[1092] Step 4: Real-time monitoring

[1093] The server monitors weather, traffic, and tourist information in real time while the generated initial travel plan is being implemented. Specifically, it periodically queries various APIs (e.g., weather forecast API, traffic information API) to obtain updated information. The input is the "initial travel plan," and the output is the update of "real-time monitoring data."

[1094] Step 5: Detecting changes in the situation

[1095] The server analyzes data acquired through real-time monitoring to detect changes in conditions such as weather fluctuations, traffic problems, and temporary closures of tourist attractions. Specifically, it runs an algorithm that compares the monitoring data with the initial travel plan and detects any discrepancies. The inputs are "real-time monitoring data" and the "initial travel plan," and the output is "detected changes in conditions."

[1096] Step 6: Generate an alternative plan

[1097] The generative AI generates alternative travel plans based on detected situational changes. Specifically, it uses a machine learning model to consider the user's preferences and current situation, and proposes the most suitable alternative. The input consists of "detected situational changes" and "prompts from the generative AI model," and the output is an "alternative travel plan."

[1098] Step 7: Notify the user

[1099] The server notifies the user's device of the alternative plan generated by the generative AI. Specifically, the user is provided with the new plan via push notification or email. The input is an "alternative travel plan," and the output is a "notification to the user."

[1100] Step 8: User selection and presentation of the final plan

[1101] The user reviews the notification and chooses whether to accept the alternative travel plan. The input is "Notification of Alternative Plan," and the new schedule is confirmed based on the user's selection.

[1102] The device displays a new schedule and provides map and navigation information once the user accepts an alternative plan. Specifically, it displays route information in conjunction with map and navigation apps. The input is "user selection," and the output provides "new schedule" and "navigation information."

[1103] The above outlines the specific processing steps of this system's program.

[1104] (Application Example 1)

[1105] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1106] In food delivery services, delivery status and traffic conditions can change in real time, potentially leading to delivery delays and a decline in service quality. Furthermore, users may become dissatisfied with the quality and speed of alternatives offered. To address this, a system is needed that monitors the situation in real time and quickly generates and presents appropriate alternatives.

[1107] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1108] In this invention, the server includes means for receiving plan information entered by the user, means for generating an initial plan based on the plan information, means for monitoring situation information in real time, generative artificial intelligence (AI) means for generating alternative plans, means for notifying the user of the alternative plans, and means for allowing the user to choose whether or not to accept the alternative plans. This makes it possible to respond to real-time changes in the situation and provide the user with a quick and optimal alternative.

[1109] A "user" refers to a person who uses the system.

[1110] "Planning information" refers to order and delivery information entered by the user.

[1111] "Means" refers to the methods or equipment used to achieve a particular objective.

[1112] "Initial plan" refers to the delivery schedule and route initially generated based on the information entered by the user.

[1113] "Real-time" refers to information and situations occurring at a given moment.

[1114] "Status information" refers to real-time information that affects deliveries, such as traffic conditions during delivery or the state of the kitchen.

[1115] "To monitor" refers to continuously observing a specific object and detecting changes in it.

[1116] "Changes in circumstances" refer to events that affect the delivery schedule, such as unexpected traffic congestion or changes in weather.

[1117] An "alternative plan" refers to a new delivery plan generated when circumstances arise that make the initial plan difficult to implement.

[1118] "Generative artificial intelligence (AI)" refers to artificial intelligence technology that generates optimal alternative solutions based on situational information.

[1119] "To notify" refers to conveying information or messages to a user.

[1120] "To allow a user to choose" means enabling the user to select one option from several choices.

[1121] The embodiments for carrying out this invention will be described in detail.

[1122] System Configuration

[1123] This system is primarily composed of three components: a server, a terminal, and generative artificial intelligence (AI).

[1124] 1. Server

[1125] The server plays a central role in receiving user-entered planning information and generating an initial plan. The server includes the following functions:

[1126] Function to receive and analyze planning information

[1127] A feature that monitors the situation in real time (traffic, weather, kitchen status).

[1128] A function that generates alternative plans in conjunction with generative AI.

[1129] A feature to notify users of alternative plans.

[1130] 2. Terminal

[1131] The terminal is a device used by the user to input planning information and receive notifications and alternatives. Generally, smartphones or tablets are used. Specific operations are as follows:

[1132] The user places an order and enters the delivery address and desired delivery time.

[1133] Receive notifications of initial and alternative plans from the server.

[1134] Choose whether to accept the alternative.

[1135] 3. Generative artificial intelligence (AI)

[1136] Generative AI is an AI model embedded within a server that generates optimal alternatives in response to real-time changes in the situation. Common AI models used include GPT-4 and BERT. The AI ​​performs the following processes:

[1137] Perform data analysis based on situational information.

[1138] Generate the optimal alternative in response to changing circumstances.

[1139] Create a prompt message and notify the user.

[1140] Hardware and software

[1141] hardware

[1142] Smartphones and tablets (user devices)

[1143] Servers (cloud-based is common)

[1144] software

[1145] Server software (e.g., Flask, Django)

[1146] Real-time monitoring systems (e.g., Geolocation API)

[1147] Generative AI (e.g. GPT-4, BERT)

[1148] Specific example

[1149] The following is a scenario in which a user orders a pizza:

[1150] 1. The user orders a pizza using a terminal and enters the delivery address and desired delivery time.

[1151] 2. The server receives this, generates an initial delivery plan, and notifies the user.

[1152] 3. If traffic congestion occurs during delivery, the server's real-time monitoring system will detect it.

[1153] 4. Generative AI generates optimal alternative routes and new delivery time suggestions based on changes in the situation.

[1154] 5. This alternative will be communicated to the user via the server, and the user will choose whether to accept it on their device.

[1155] Example of a prompt

[1156] "User ID: 123, pizza ordered. Traffic congestion occurred during delivery. Please offer the user an alternative: a restaurant discount coupon or another food item."

[1157] This embodiment of the invention makes it possible to respond immediately to real-time changes in circumstances and quickly provide users with the most suitable alternative. This, in turn, can improve the quality of food delivery services.

[1158] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1159] Step 1:

[1160] The user uses a terminal to enter order information and submits the product (e.g., pizza), delivery address, and desired delivery time.

[1161] Input: Product information, delivery address, desired delivery time

[1162] Output: Initial order data

[1163] Operation: Form input on the terminal

[1164] Step 2:

[1165] The terminal sends the order information entered by the user to the server.

[1166] Input: Initial order data

[1167] Output: Order data sent to the server

[1168] Operation: Sends data using HTTP requests

[1169] Step 3:

[1170] The server receives the initial order data and generates an initial delivery plan.

[1171] Input: Initial order data

[1172] Output: Initial delivery plan

[1173] Operation: A planning generation algorithm is executed to calculate the optimal delivery route and time.

[1174] Step 4:

[1175] The server notifies the terminal of the initial delivery plan and displays it to the user.

[1176] Input: Initial delivery plan

[1177] Output: Delivery plan notification displayed to the user

[1178] Operation: Sends data to the terminal via HTTP response.

[1179] Step 5:

[1180] It monitors situational information (traffic, weather, etc.) in real time and sends the data to the server.

[1181] Input: Real-time status information

[1182] Output: Updated status data

[1183] Operation: The monitoring system periodically collects data and sends it to the server.

[1184] Step 6:

[1185] The server analyzes real-time status information, and if a change in the situation is detected, it uses generative artificial intelligence (AI) to generate an alternative plan.

[1186] Input: Updated status data

[1187] Output: Alternative plan

[1188] Operation: An AI model (e.g., GPT-4) analyzes the input data and generates the optimal alternative.

[1189] Step 7:

[1190] The server notifies the terminal of the generated alternative plan along with a prompt message.

[1191] Input: Alternative plan, prompt statement

[1192] Output: Alternatives notified to the user

[1193] Operation: Sends notification data to the device via HTTP response.

[1194] Step 8:

[1195] The device displays an alternative plan notification to the user, who then chooses whether to accept the alternative.

[1196] Input: Alternative proposal notification

[1197] Output: User Selection

[1198] Operation: Selection behavior in the user interface

[1199] Step 9:

[1200] The user's selection is sent to the server, which then confirms and notifies the user of the final delivery plan based on the selection.

[1201] Input: User Selection

[1202] Output: Final delivery plan

[1203] Operation: Receive selected data, generate a new schedule, and send another notification.

[1204] This allows for immediate response to real-time changes in the food delivery process and enables the rapid provision of the most suitable alternative to the user.

[1205] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1206] This invention is a system that provides maximum support for the user's travel experience, and is characterized by its ability to flexibly respond to real-time changes in circumstances, as well as recognizing the user's emotions and adjusting alternatives and notification content accordingly.

[1207] Overall system configuration

[1208] The system consists of the following main components:

[1209] User device: A device such as a smartphone or computer where the user inputs travel plans and receives notifications and alternative suggestions. This includes cameras and microphones for emotion recognition.

[1210] Server: A central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed. It also incorporates an emotion engine.

[1211] Generative AI: This is a generative artificial intelligence (AI) embedded within a server that provides optimal alternative solutions based on real-time changes in the situation.

[1212] Emotion Engine: A system that recognizes a user's emotions through methods such as voice analysis and camera image analysis.

[1213] Operation on the user terminal

[1214] The user enters information into the device to plan their trip. Specifically, they enter the following information:

[1215] Travel destination

[1216] Scheduled visit date

[1217] Desired experiences and tourist destinations

[1218] means of transportation

[1219] The entered plan information is sent from the user terminal to the server.

[1220] Processing on the server

[1221] The server performs the following processes based on the received travel plan information:

[1222] The system analyzes the entered travel plan information and generates an initial travel plan by referencing databases of weather forecasts, traffic information, and tourist information. Furthermore, it monitors weather information, traffic information, and opening / closing status of tourist attractions in real time and detects changes in the situation based on this information.

[1223] The system periodically acquires the user's location information to understand their current situation.

[1224] Generating alternative plans using generative AI and emotion engines

[1225] The generative AI generates alternative travel plans based on data from the server's monitoring system, adapting to changing circumstances. This generation process includes:

[1226] Based on changing circumstances, for example, in the case of bad weather or traffic congestion, we can suggest new destinations or timings. Specifically, we can flexibly respond by changing from outdoor to indoor tourist attractions.

[1227] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this, it suggests relaxing plans if the user is stressed, and active plans if they are excited.

[1228] Example: If the system detects that the user is experiencing stress based on their voice or facial expressions, it might suggest that they take a break at a cafe.

[1229] Notification and selection of alternative plans

[1230] The server notifies the user's terminal of alternative plans created by generative AI. The content of the notification is also adjusted according to the user's emotions. For example, if the user is feeling stressed, the notification will be in a supportive tone.

[1231] The device provides an interface that displays alternative plans to the user and allows them to choose whether to accept or reject them.

[1232] The user reviews the notified alternative plan and chooses whether to accept it. If the user accepts the alternative plan, a new schedule is generated and displayed on the device.

[1233] Specific Scenario

[1234] As an example, consider a user planning a trip to Paris:

[1235] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[1236] 2. The terminal sends the entered information to the server.

[1237] 3. The server generates an initial travel plan based on the received information and notifies the user.

[1238] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[1239] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[1240] 6. The emotion engine recognizes the user's emotions from their voice and facial expressions, and adjusts the suggestions to match the user's mood. For example, if the user is tired, it will suggest an indoor cafe where they can rest.

[1241] 7. The server notifies the user of the adjusted alternative plan.

[1242] 8. The user reviews the alternative plan and chooses whether to accept it.

[1243] 9. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[1244] This system allows users to receive optimal suggestions tailored to their emotions, even in the face of unexpected situations, resulting in a better travel experience. The collaboration between an emotion engine and generative AI provides maximum support for the user's travels.

[1245] The following describes the processing flow.

[1246] Step 1:

[1247] The user enters travel plan information into the terminal, such as the travel destination, planned visit dates, desired experiences, and mode of transportation.

[1248] Step 2:

[1249] The terminal receives the entered travel plan information and sends it to the server.

[1250] Step 3:

[1251] The server analyzes the received travel plan information and compares it with databases containing weather forecasts, traffic information, and tourist information to generate an initial travel plan.

[1252] Step 4:

[1253] The server sends the initial travel plan to the user's terminal and notifies the user.

[1254] Step 5:

[1255] The user reviews their initial travel plan.

[1256] Step 6:

[1257] The server monitors weather, traffic, and tourist information for the travel destination in real time during the trip.

[1258] Step 7:

[1259] The server periodically retrieves the user's location information to understand their current status.

[1260] Step 8:

[1261] The server acquires the user's voice and image data from the camera, and uses an emotion engine to recognize the user's emotions.

[1262] Step 9:

[1263] When the server detects a change in circumstances (for example, a sudden change in weather, traffic congestion, or temporary closure of a tourist attraction), it uses generative artificial intelligence (AI) to generate an alternative travel plan.

[1264] Step 10:

[1265] Generative AI optimizes alternative plans based on the user's current emotions. For example, if the user is feeling stressed, it will suggest a plan that helps them relax.

[1266] Step 11:

[1267] The server sends the generated alternative travel plan to the user's terminal and notifies the user.

[1268] Step 12:

[1269] The device displays alternative travel plans to the user, allowing them to choose whether or not to accept them.

[1270] Step 13:

[1271] The user reviews the alternative travel plan and chooses whether to accept it.

[1272] Step 14:

[1273] The device will display the new schedule if the user accepts an alternative travel plan.

[1274] Step 15:

[1275] The server provides the necessary information (maps, navigation, details of destinations, etc.) based on the new schedule.

[1276] Step 16:

[1277] The user continues their journey according to the new schedule. During this time, they can check real-time information (traffic conditions, weather forecast, etc.) on their device as needed.

[1278] This system allows users to respond flexibly to unexpected situations and receive optimal suggestions tailored to their emotions, resulting in a better travel experience.

[1279] (Example 2)

[1280] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1281] Conventional travel planning support systems can provide alternative plans in response to real-time changes in circumstances, but they have the problem of not being able to adjust plans while considering the user's emotional state. As a result, users may experience stress or be unable to respond appropriately to unexpected situations, making it difficult to provide the optimal travel experience.

[1282] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1283] In this invention, the server includes means for receiving travel plan information entered by the user, means for generating an initial travel plan based on the travel plan information, means for monitoring weather information, traffic information, and sightseeing information in real time during the trip, generative artificial intelligence (AI) means for generating an alternative travel plan based on changes in circumstances, emotion recognition means for recognizing the user's emotional state and adjusting the alternative travel plan based on those emotions, means for notifying the user of the alternative travel plan, and means for allowing the user to choose whether or not to accept the alternative travel plan. As a result, the user can not only respond flexibly to real-time changes in circumstances but also receive an optimal travel plan that takes into account their own emotional state.

[1284] "Means for receiving travel plan information entered by the user" refers to a system that allows users to enter travel plan information via a terminal and send that information to a server.

[1285] "Travel plan information" is a general term for the information that users input regarding their trip, such as destination, planned dates of visit, desired experiences and tourist attractions, and mode of transportation.

[1286] "Means for generating initial travel plans" refers to a system for creating an initial travel schedule based on the received travel plan information.

[1287] "Means for monitoring weather, traffic, and tourist information in real time while traveling" refers to a system for continuously acquiring and monitoring the latest weather forecasts, traffic conditions, opening / closing information for tourist destinations, etc.

[1288] "Generative artificial intelligence (AI) means for generating alternative travel plans based on changes in circumstances" refers to artificial intelligence technology for generating new travel plans based on changes in circumstances detected by monitoring means.

[1289] "An emotion recognition system that recognizes the user's emotional state and adjusts alternative travel plans based on those emotions" refers to a system that uses cameras and microphones to analyze the user's emotions from their facial expressions and voice, and adjusts the travel plan accordingly.

[1290] "Means of notifying users of alternative travel plans" refers to a mechanism for informing users of the generated alternative travel plans. This includes push notifications and alert messages to the device.

[1291] "A means of allowing users to choose whether or not to accept alternative travel plans" refers to a mechanism that provides an interface where users can review proposed alternative travel plans and choose whether or not to accept them.

[1292] This invention is a system that provides maximum support for the user's travel experience, characterized by its ability to flexibly respond to real-time changes in circumstances and to recognize the user's emotions to adjust alternatives and notification content.

[1293] Overall system configuration

[1294] The system consists of the following main components:

[1295] User device: A device such as a smartphone or computer where the user inputs travel plans and receives notifications and alternative suggestions. This includes cameras and microphones for emotion recognition.

[1296] Server: A central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed. It also incorporates an emotion engine.

[1297] Generative AI: This is a generative artificial intelligence (AI) embedded within a server that provides optimal alternative solutions based on real-time changes in the situation.

[1298] Emotion Engine: A system that recognizes a user's emotions through methods such as voice analysis and camera image analysis.

[1299] Operation on the user terminal

[1300] The user enters information into the device to plan their trip. Specifically, they enter the following information:

[1301] Travel destination

[1302] Scheduled visit date

[1303] Desired experiences and tourist destinations

[1304] means of transportation

[1305] The entered plan information is sent from the user's terminal to the server. For example, when a user enters their destination and travel dates into a smartphone application and taps the "Send" button, the information is sent to the server.

[1306] Processing on the server

[1307] The server performs the following processes based on the received travel plan information:

[1308] The system analyzes the entered travel plan information and generates an initial travel plan by referencing databases such as weather forecasts, traffic information, and tourist information. For example, it retrieves weather information for specific days based on the travel dates and checks the operating status of tourist destinations to be visited.

[1309] The system monitors weather information, traffic information, and opening / closing status information for tourist attractions in real time, and detects changes in the situation based on this information. The server periodically updates this information via an API.

[1310] The system periodically acquires the user's location information to understand their current situation. Specifically, it collects GPS data from the device and sends it to the server.

[1311] Generating alternative plans using generative AI and emotion engines

[1312] The generative AI generates alternative travel plans based on data from the server's monitoring system, adapting to changing circumstances. This generation process includes:

[1313] Based on changing circumstances, the system suggests alternative destinations and timings, for example, in case of bad weather or traffic congestion. Specifically, it can flexibly adapt by changing from outdoor to indoor tourist attractions. For instance, if a planned outdoor activity is canceled due to rain, the system can generate a plan to visit a nearby museum.

[1314] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this, it suggests relaxing plans if the user is stressed, and active plans if they are excited. For example, if the system recognizes that the user is stressed based on their voice and facial expressions, it will suggest taking a break at a cafe.

[1315] Notification and selection of alternative plans

[1316] The server notifies the user's terminal of alternative plans created by generative AI. The content of the notification is also adjusted according to the user's emotions. For example, if the user is feeling stressed, the notification will be in a supportive tone.

[1317] The device provides an interface that displays alternative plans to the user and allows them to choose whether to accept or reject them. It displays push notifications and alert messages, allowing the user to tap to view details. For example, tapping a push notification displays detailed alternative plans and "Accept" or "Reject" buttons.

[1318] The user reviews the notified alternative plan and chooses whether to accept it. If the user accepts the alternative plan, the new schedule will be displayed on the device.

[1319] Specific example scenarios and prompt statements

[1320] As a specific scenario, let's consider a user planning a trip to Paris:

[1321] 1. The user plans a trip to Paris using their smartphone and wishes to visit the Louvre Museum.

[1322] 2. The terminal sends the entered information to the server.

[1323] 3. The server generates an initial travel plan based on the received information and notifies the user.

[1324] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[1325] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[1326] 6. The emotion engine recognizes the user's emotions from their voice and facial expressions, and adjusts the suggestions to match the user's feelings. For example, if the user is tired, it will suggest taking a break at a cafe.

[1327] 7. The server notifies the user of the adjusted alternative plan.

[1328] 8. The user reviews the alternative plan and chooses whether to accept it.

[1329] 9. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[1330] Example of a prompt

[1331] "I'm planning a trip to Paris, and I'd like some recommendations for indoor places to enjoy myself if the weather turns bad."

[1332] This system allows users to respond flexibly to unexpected situations while receiving optimal suggestions tailored to their individual emotions. The collaboration of an emotion engine and generative AI provides maximum support for the user's travels.

[1333] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1334] Step 1:

[1335] The user enters travel plan information into their device. This information includes destination, planned visit dates, desired experiences or attractions, and mode of transportation. After entering the information, tapping the "Submit" button saves the information to the device. This action prepares the entered information as data for use in the next processing step.

[1336] Step 2:

[1337] The terminal receives travel plan information entered by the user and sends it to the server. Specifically, the terminal uses its internet connection to send the user's entered travel plan information (destination, planned visit dates, desired experiences or sightseeing spots, and mode of transportation) to the server. After transmission, the server analyzes the received data.

[1338] Step 3:

[1339] The server generates an initial travel plan based on the received travel plan information. It takes travel plan information as input data and generates an initial travel plan as a processing result. Specifically, the server refers to databases such as weather forecasts, traffic information, and tourist information to create an optimal plan based on the travel dates and destination. This initial travel plan is then ready to be notified to the user.

[1340] Step 4:

[1341] The server monitors weather, traffic, and tourist information in real time. It acquires various real-time data via APIs as input and evaluates that information. Specifically, the server periodically updates this information and has the functionality to detect changes in the situation. This allows for the proactive detection of potential problems that may occur during a trip.

[1342] Step 5:

[1343] The server periodically acquires the user's location information to understand their current situation. It receives GPS data from the device as input data to determine the user's current location. Specifically, the device periodically sends GPS information to the server, which analyzes this data to check the user's movement.

[1344] Step 6:

[1345] The generative AI generates alternative travel plans based on data monitored on a server, adapting to changing circumstances. It takes real-time information and user location data as input and uses this to generate alternative plans. Specific examples include flexible plan generation, such as changing outdoor activities to indoor activities if bad weather is predicted.

[1346] Step 7:

[1347] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. It takes audio and image data as input and processes it to recognize the user's emotional state. Specifically, it uses image recognition technology to analyze facial expressions and speech recognition technology to evaluate voice tone and volume. This allows it to respond in ways such as suggesting relaxing plans if the user is feeling stressed.

[1348] Step 8:

[1349] The server notifies the user's terminal of the alternative plan created by the generative AI. It takes the alternative plan as input data and generates notification data as the processing result. Specifically, it sends push notifications or alert messages to the terminal so that the user can check them.

[1350] Step 9:

[1351] The device provides an interface that displays alternative plans to the user and allows them to choose whether to accept or reject them. It receives the alternative plan notified as input data and generates an interface that the user can select as a result of processing. Specifically, when the push notification is tapped, the detailed alternative plan is displayed, and "Accept" or "Reject" buttons are shown.

[1352] Step 10:

[1353] The user reviews the notified alternative plan and chooses whether to accept it. The notification details of the alternative plan are reviewed as input data, and the selection result is entered into the device as processing result. Specifically, if the user accepts, they tap the "Accept" button, and the new schedule is displayed on the device.

[1354] (Application Example 2)

[1355] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1356] Conventional travel planning support systems can provide alternatives to accommodate changes in circumstances during a trip, but they lack consideration for the user's emotions when generating alternatives or adjusting notification content, leading to problems such as users easily experiencing stress and dissatisfaction. Furthermore, the alternative suggestions are often mechanical, making it difficult to fully support the user's travel experience.

[1357] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving travel plan information input by the user, means for generating an initial travel plan based on the travel plan information, means for monitoring weather information, traffic information, and sightseeing information during the trip in real time, generative artificial intelligence (AI) means for generating an alternative travel plan based on the situation changes detected by the monitoring means, means for notifying the user of the alternative travel plan, emotion engine means for recognizing the user's emotions, means for adjusting the alternative plan and notification content according to the user's emotions based on the emotion engine means, and means for allowing the user to choose whether or not to accept the alternative travel plan. This makes it possible to always provide the optimal alternative plan that takes the user's emotions into consideration, and for the user to have a more comfortable travel experience.

[1358] A "user" is a system user who inputs travel plan information, receives notifications of alternative options, and makes a selection.

[1359] "Travel plan information" refers to information entered by the user, such as the travel destination, planned dates of visit, desired experiences or sightseeing spots, and mode of transportation.

[1360] An "initial travel plan" refers to the basic schedule and activities of a trip, which are generated based on the travel plan information entered by the user.

[1361] "Real-time monitoring" means keeping weather, traffic, and tourist information constantly updated to the latest state during your trip.

[1362] "Changes in circumstances" refer to changes in weather conditions, traffic conditions, and the opening / closing status of tourist attractions that may occur during your trip.

[1363] An "alternative travel plan" is a preliminary or modified travel plan that is generated in response to changes in circumstances monitored in real time.

[1364] "Generative artificial intelligence (AI) methods" refer to artificial intelligence systems that generate travel plans and alternative options based on collected data.

[1365] An "emotion engine" is a system that analyzes the user's voice and camera images to recognize the user's emotions.

[1366] "Means for adjusting alternatives and notification content" refers to means for appropriately changing alternatives and notification content based on the user's emotions recognized by the emotion engine.

[1367] "Location information" refers to information that indicates the user's current geographical location.

[1368] A "new schedule" is a new travel schedule generated based on the alternative travel plan accepted by the user.

[1369] This invention is a system that provides maximum support for the user's travel experience, characterized by its ability to flexibly respond to real-time changes in circumstances, recognize the user's emotions, and adjust alternatives and notifications accordingly. The system consists of the following main components.

[1370] Overall system configuration

[1371] 1. User terminal: A device such as a smartphone or personal computer where the user inputs travel plans and receives notifications and alternative suggestions. This terminal includes a camera and microphone for emotion recognition.

[1372] 2. Server: This is the central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed. The server incorporates generative artificial intelligence (AI) and an emotion engine.

[1373] Operation on the user terminal

[1374] The user enters travel plan information into the terminal. This information includes the travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation. This entered plan information is then sent from the user's terminal to the server.

[1375] Processing on the server

[1376] The server performs the following processing based on the received travel plan information.

[1377] The system analyzes the entered travel plan information and generates an initial travel plan by referencing databases of weather forecasts, traffic information, and tourist information. Furthermore, it monitors weather information, traffic information, and opening / closing status of tourist attractions in real time and detects changes in the situation based on this information.

[1378] The system periodically acquires the user's location information to understand their current situation.

[1379] Generating alternative plans using generative AI and emotion engines

[1380] The generative AI generates alternative travel plans that adapt to changing circumstances, based on data from the server's monitoring system. This generation process includes the following:

[1381] Based on changing circumstances, for example, in the case of bad weather or traffic congestion, we can suggest new destinations or timings. Specifically, we can flexibly respond by changing from outdoor to indoor tourist attractions.

[1382] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this, it suggests relaxing plans if the user is stressed, and active plans if they are excited.

[1383] Notification and selection of alternative plans

[1384] The server notifies the user's terminal of an alternative plan created by a generative AI. The content of the notification is also adjusted according to the user's emotions. For example, if the user is feeling stressed, the notification will be in a supportive tone. The user's terminal provides an interface that displays the alternative plan to the user and allows them to choose whether to accept it or not. If the user accepts the alternative plan, a new schedule is generated and displayed on the terminal.

[1385] Specific Scenario

[1386] For users planning a trip to Paris, the process would be as follows:

[1387] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[1388] 2. The terminal sends the entered information to the server.

[1389] 3. The server generates an initial travel plan based on the received information and notifies the user.

[1390] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[1391] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[1392] 6. The emotion engine recognizes the user's emotions from their voice and facial expressions, and adjusts the suggestions to match the user's mood. For example, if the user is tired, it will suggest an indoor cafe where they can rest.

[1393] 7. The server notifies the user of the adjusted alternative plan.

[1394] 8. The user reviews the alternative plan and chooses whether to accept it.

[1395] 9. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[1396] This system allows users to receive optimal suggestions tailored to their emotions, even in the face of unexpected situations, resulting in a better travel experience. The collaboration between an emotion engine and generative AI provides maximum support for the user's travels.

[1397] Example of a prompt:

[1398] "I want to build an application that suggests the best travel plan based on the user's emotions and weather information. Please suggest the best alternative for the following travel plan: Destination: Paris, Planned visits: Louvre Museum, Weather: Rain, User's emotions: Stressed."

[1399] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1400] Step 1:

[1401] The user enters travel plan information into the terminal. Specifically, they enter their travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation. Based on the entered data, travel plan information is generated. This travel plan information is then sent from the terminal to the server.

[1402] Input: Travel destination, planned visit dates, desired experiences, sightseeing spots, mode of transportation

[1403] Output: Generation and transmission of travel plan information

[1404] Step 2:

[1405] The server generates an initial travel plan based on the received travel plan information. Specifically, it refers to databases such as weather forecasts, traffic information, and tourist information, and plans a schedule and destinations according to the entered information.

[1406] Input: Travel plan information

[1407] Output: Initial travel plan

[1408] Step 3:

[1409] The server monitors weather, traffic, and tourist information in real time. This monitoring process retrieves necessary information via APIs and stores it in a database. It also periodically retrieves user location information to understand their current situation. This information is then analyzed to detect changes in the situation.

[1410] Input: Current weather information, traffic information, tourist information, location information

[1411] Output: Latest status data

[1412] Step 4:

[1413] The server's generative artificial intelligence (AI) system generates alternative travel plans based on the latest acquired situational data. It flexibly adapts to changing circumstances, modifying the initial travel plan and suggesting new destinations and schedules. For example, it might change from outdoor to indoor tourist attractions in case of bad weather.

[1414] Input: Latest situation data, initial travel plan

[1415] Output: Alternative travel plan

[1416] Step 5:

[1417] The server's emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this recognition, it suggests a relaxing plan if the user is feeling stressed, and an active plan if they are feeling excited.

[1418] Input: Camera data, audio data

[1419] Output: Emotion recognition result

[1420] Step 6:

[1421] The server adjusts alternative plans and notifications based on the user's emotions, as recognized by the emotion engine. For example, if the user is feeling stressed, it will send a supportive notification and add suggestions for places to rest.

[1422] Input: Emotion recognition results, alternative travel plans

[1423] Output: Adjusted alternative travel plan, notification content

[1424] Step 7:

[1425] The server notifies the user terminal of the adjusted alternative travel plan. The terminal provides an interface that displays this alternative plan to the user and allows them to choose whether to accept it or not.

[1426] Input: Adjusted alternative travel plan, notification details

[1427] Output: Notification to the user

[1428] Step 8:

[1429] The user reviews the notified alternative plan and chooses whether to accept it. If the user accepts the alternative plan, the server generates a new schedule and sends it to the device. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[1430] Input: Select an alternative plan

[1431] Output: New schedule, displayed to the user.

[1432] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1433] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1434] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1435] [Fourth Embodiment]

[1436] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1437] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1438] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1439] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1440] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1441] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1442] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1443] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1444] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1445] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1446] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1447] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1448] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1449] This invention is a system that provides maximum support for the user's travel experience, and is characterized in particular by real-time monitoring and the provision of alternative plans to respond to unexpected situations during travel.

[1450] Overall system configuration

[1451] The system mainly consists of the following components:

[1452] User device: A device such as a smartphone or computer where the user enters their travel plans and receives notifications and alternative suggestions.

[1453] Server: A central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed.

[1454] Generative AI: This is a generative artificial intelligence (AI) embedded within a server that provides optimal alternative solutions based on real-time changes in the situation.

[1455] Operation on the user terminal

[1456] The user enters the following information into the terminal to plan a specific trip:

[1457] Travel destination

[1458] Scheduled visit date

[1459] Desired experiences and tourist destinations

[1460] means of transportation

[1461] The entered plan information is sent from the user terminal to the server.

[1462] Processing on the server

[1463] The server performs the following processes based on the received travel plan information:

[1464] To generate an initial travel plan, the system cross-references databases such as weather forecasts, traffic information, and tourist information to determine the optimal schedule and order of visits.

[1465] After the initial plan is generated, weather information, traffic information, and opening / closing status of tourist attractions are monitored in real time. This allows for the timely detection of unexpected weather changes, traffic problems, closures of tourist attractions, and other changes in circumstances.

[1466] Generating alternative plans using generative AI

[1467] The generative AI performs the following processes based on data from the server monitoring system:

[1468] If a change in circumstances is detected, an alternative travel plan will be generated accordingly. This alternative travel plan will include the options best suited to the user's preferences and current situation.

[1469] Example: If the Louvre Museum is temporarily closed, we would suggest a shopping plan at the Musée d'Orsay or along the Champs-Élysées.

[1470] Notification and selection of alternative plans

[1471] The server notifies the user terminal of an alternative plan created by the generative AI. The user can choose whether or not to accept the notified alternative plan.

[1472] The device will display the new schedule as needed if the user accepts an alternative plan. Furthermore, it will support the user's travel experience by providing maps and navigation.

[1473] Specific Scenario

[1474] As an example, consider a user planning a trip to Paris:

[1475] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[1476] 2. The terminal sends the entered information to the server.

[1477] 3. The server generates an initial travel plan based on the received information and notifies the user.

[1478] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[1479] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[1480] 6. The server notifies the user of the generated alternative plan.

[1481] 7. The user selects to visit the Musée d'Orsay, and the device displays a new schedule.

[1482] This system allows users to make the most of their trip, even in the face of unexpected events. Real-time information monitoring and flexible responses powered by generative AI enrich the user's travel experience.

[1483] The following describes the processing flow.

[1484] Step 1:

[1485] The user enters travel plan information into the terminal, such as the travel destination, planned visit dates, desired experiences, and mode of transportation.

[1486] Step 2:

[1487] The terminal receives the entered travel plan information and sends it to the server.

[1488] Step 3:

[1489] The server analyzes the received travel plan information and compares it with databases containing weather forecasts, traffic information, and tourist information to generate an initial travel plan.

[1490] Step 4:

[1491] The server sends the initial travel plan to the user's terminal and notifies the user.

[1492] Step 5:

[1493] The user reviews their initial travel plan.

[1494] Step 6:

[1495] The server monitors weather, traffic, and tourist information for the travel destination in real time during the trip.

[1496] Step 7:

[1497] The server periodically retrieves the user's location information to understand their current status.

[1498] Step 8:

[1499] When the server detects a change in circumstances (for example, a sudden change in weather, traffic congestion, or temporary closure of a tourist attraction), it uses generative artificial intelligence (AI) to generate an alternative travel plan.

[1500] Step 9:

[1501] The server sends the generated alternative travel plan to the user's terminal and notifies the user.

[1502] Step 10:

[1503] The device displays alternative travel plans to the user, allowing them to choose whether or not to accept them.

[1504] Step 11:

[1505] The user reviews the alternative travel plan and chooses whether to accept it.

[1506] Step 12:

[1507] The device will display the new schedule if the user accepts an alternative travel plan.

[1508] Step 13:

[1509] The server provides the necessary information (maps, navigation, details of destinations, etc.) based on the new schedule.

[1510] Step 14:

[1511] The user continues their journey according to the new schedule. During this time, they can check real-time information (traffic conditions, weather forecast, etc.) on their device as needed.

[1512] This series of processes allows users to respond flexibly to unexpected situations and have a better travel experience.

[1513] (Example 1)

[1514] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1515] Current travel support systems struggle to respond quickly and appropriately to unexpected situations that may arise during travel, such as weather changes, transportation problems, or the closure of tourist attractions. As a result, this increases the stress and difficulty users experience when changing their travel plans, degrading the quality of their travel experience. Furthermore, the limited availability of alternative plans means that they cannot fully meet the diverse needs of users.

[1516] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1517] In this invention, the server includes means for receiving travel plan information entered by a user, means for generating an initial travel plan based on the travel plan information, means for monitoring weather information, traffic information, and tourist information in real time during the trip, generative artificial intelligence (AI) means for generating an alternative travel plan based on changes in circumstances detected by the monitoring means, means for notifying the user of the alternative travel plan, means for allowing the user to choose whether to accept the alternative travel plan, and if the user accepts the alternative travel plan, means for displaying the new schedule and providing maps and navigation information as necessary. This enables the user to respond quickly and appropriately to unexpected situations that occur during the trip, thereby improving the quality of the travel experience.

[1518] A "user" refers to an individual who uses the system to plan a trip and receives information and services provided by the system during their trip.

[1519] A "terminal" refers to a device such as a smartphone or personal computer used by the user, through which travel planning information is entered and notifications are received.

[1520] A "server" refers to a central system that receives travel plan information sent from user terminals, performs analysis, real-time monitoring, and generates and notifies users of alternative plans.

[1521] "Travel planning information" refers to information that users enter to plan their trip, such as the travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation.

[1522] "Initial travel plan" refers to the travel schedule generated by the server based on travel plan information entered by the user.

[1523] "Weather information" refers to information about the weather at the travel destination, which may affect your travel plans.

[1524] "Traffic information" refers to information about public transportation and road conditions between travel destinations.

[1525] "Tourism information" refers to all information related to users' tourism activities, such as opening and closing times of tourist attractions and event information.

[1526] "Generative artificial intelligence (AI)" refers to an AI system that has the function of generating optimal alternative travel plans in real time based on changing circumstances.

[1527] An "alternative travel plan" refers to a new travel plan generated by generative artificial intelligence (AI) to adapt to unexpected changes in circumstances, in addition to the initial travel plan.

[1528] "Notifications" refer to messages and alerts sent from the server to the user's terminal, including information about travel plans and alternative plans.

[1529] "Selection" refers to the action a user takes to decide whether or not to accept the alternative travel plan they have been notified about.

[1530] "New schedule" refers to the revised travel schedule that is generated when the user accepts an alternative travel plan.

[1531] "Maps" and "navigation information" refer to the directions and information necessary for users to travel to their next destination based on the new schedule.

[1532] This invention is a system designed to maximize the user's travel experience, particularly characterized by real-time monitoring and the provision of alternative plans to address unexpected situations during travel. The system consists of user terminals, a server, and generative artificial intelligence (AI) components.

[1533] Operation on the user terminal

[1534] Users input travel plan information using their smartphones or computers (user terminals). Input fields include travel destination, planned visit dates, desired experiences and sightseeing spots, and mode of transportation. Once the user enters this information into their terminal and presses the submit button, the information is transferred to the server.

[1535] Processing on the server

[1536] The server receives travel plan information sent from the user's terminal and begins analysis. The server generates an optimal initial travel plan by cross-referencing various databases, including weather forecasts, traffic information, and opening / closing information for tourist attractions. It also monitors weather, traffic, and tourist information in real time during the trip, detecting changes in conditions in a timely manner. The databases used include, for example, weather forecast APIs for weather information and public transport APIs for traffic information.

[1537] Generating alternative plans using generative AI

[1538] Generative AI generates alternative travel plans based on changes in circumstances detected by the server. The AI ​​provides the optimal alternative plan, taking into account the user's preferences and current situation. Generative AI uses machine learning models, enabling it to respond quickly to unexpected situations.

[1539] Notification and selection of alternative plans

[1540] The server notifies the user of alternative plans created by generative AI. These notifications are sent via push notifications, email, or other methods. The user can choose whether or not to accept the alternative plan. If the user accepts the alternative plan, the device displays the new schedule as needed and also provides maps and navigation information, thereby supporting the user's travel experience.

[1541] Specific Scenario

[1542] As an example, consider a user planning a trip to Paris:

[1543] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[1544] 2. The terminal sends the entered information to the server.

[1545] 3. The server generates an initial travel plan based on the received information and notifies the user.

[1546] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[1547] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[1548] 6. The server notifies the user of the generated alternative plan.

[1549] 7. The user selects to visit the Musée d'Orsay, and the device displays a new schedule.

[1550] Example of a prompt

[1551] "Please plan and provide a trip to Paris."

[1552] "Please propose alternative plans in case the Louvre Museum closes."

[1553] "Please generate the optimal travel schedule based on current traffic and weather information."

[1554] This invention enables users to respond quickly and appropriately to unexpected situations that may arise during their travels, allowing them to enjoy a comfortable and fulfilling travel experience. Real-time information monitoring and flexible responses powered by generative AI enrich the user's travel experience.

[1555] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1556] Step 1: Collecting user input information

[1557] The user enters travel plan information into the terminal, including the travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation. The entered data is saved on the user's terminal as "travel plan information," and when the send button is pressed, it is transferred to the server.

[1558] Step 2: Transferring information to the server

[1559] The device sends the travel plan information entered by the user to the server via the internet. Specifically, it sends data to the server's API endpoint using an HTTP request. The input is "travel plan information," and the output is "data transmission to the server."

[1560] Step 3: Generating an initial travel plan

[1561] The server uses the received travel plan information to compare it with databases of weather forecasts, traffic information, and tourist information. Specifically, it queries the "Weather Forecast API," "Traffic Information API," and "Tourist Attraction Opening / Closing Information Database," among others. As a result, with "travel plan information" as input, an "initial travel plan" is generated as output, ready to be notified to the user.

[1562] Step 4: Real-time monitoring

[1563] The server monitors weather, traffic, and tourist information in real time while the generated initial travel plan is being implemented. Specifically, it periodically queries various APIs (e.g., weather forecast API, traffic information API) to obtain updated information. The input is the "initial travel plan," and the output is the update of "real-time monitoring data."

[1564] Step 5: Detecting changes in the situation

[1565] The server analyzes data acquired through real-time monitoring to detect changes in conditions such as weather fluctuations, traffic problems, and temporary closures of tourist attractions. Specifically, it runs an algorithm that compares the monitoring data with the initial travel plan and detects any discrepancies. The inputs are "real-time monitoring data" and the "initial travel plan," and the output is "detected changes in conditions."

[1566] Step 6: Generate an alternative plan

[1567] The generative AI generates alternative travel plans based on detected situational changes. Specifically, it uses a machine learning model to consider the user's preferences and current situation, and proposes the most suitable alternative. The input consists of "detected situational changes" and "prompts from the generative AI model," and the output is an "alternative travel plan."

[1568] Step 7: Notify the user

[1569] The server notifies the user's device of the alternative plan generated by the generative AI. Specifically, the user is provided with the new plan via push notification or email. The input is an "alternative travel plan," and the output is a "notification to the user."

[1570] Step 8: User selection and presentation of the final plan

[1571] The user reviews the notification and chooses whether to accept the alternative travel plan. The input is "Notification of Alternative Plan," and the new schedule is confirmed based on the user's selection.

[1572] The device displays a new schedule and provides map and navigation information once the user accepts an alternative plan. Specifically, it displays route information in conjunction with map and navigation apps. The input is "user selection," and the output provides "new schedule" and "navigation information."

[1573] The above outlines the specific processing steps of this system's program.

[1574] (Application Example 1)

[1575] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1576] In food delivery services, delivery status and traffic conditions can change in real time, potentially leading to delivery delays and a decline in service quality. Furthermore, users may become dissatisfied with the quality and speed of alternatives offered. To address this, a system is needed that monitors the situation in real time and quickly generates and presents appropriate alternatives.

[1577] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1578] In this invention, the server includes means for receiving plan information entered by the user, means for generating an initial plan based on the plan information, means for monitoring situation information in real time, generative artificial intelligence (AI) means for generating alternative plans, means for notifying the user of the alternative plans, and means for allowing the user to choose whether or not to accept the alternative plans. This makes it possible to respond to real-time changes in the situation and provide the user with a quick and optimal alternative.

[1579] A "user" refers to a person who uses the system.

[1580] "Planning information" refers to order and delivery information entered by the user.

[1581] "Means" refers to the methods or equipment used to achieve a particular objective.

[1582] "Initial plan" refers to the delivery schedule and route initially generated based on the information entered by the user.

[1583] "Real-time" refers to information and situations occurring at a given moment.

[1584] "Status information" refers to real-time information that affects deliveries, such as traffic conditions during delivery or the state of the kitchen.

[1585] "To monitor" refers to continuously observing a specific object and detecting changes in it.

[1586] "Changes in circumstances" refer to events that affect the delivery schedule, such as unexpected traffic congestion or changes in weather.

[1587] An "alternative plan" refers to a new delivery plan generated when circumstances arise that make the initial plan difficult to implement.

[1588] "Generative artificial intelligence (AI)" refers to artificial intelligence technology that generates optimal alternative solutions based on situational information.

[1589] "To notify" refers to conveying information or messages to a user.

[1590] "To allow a user to choose" means enabling the user to select one option from several choices.

[1591] The embodiments for carrying out this invention will be described in detail.

[1592] System Configuration

[1593] This system is primarily composed of three components: a server, a terminal, and generative artificial intelligence (AI).

[1594] 1. Server

[1595] The server plays a central role in receiving user-entered planning information and generating an initial plan. The server includes the following functions:

[1596] Function to receive and analyze planning information

[1597] A feature that monitors the situation in real time (traffic, weather, kitchen status).

[1598] A function that generates alternative plans in conjunction with generative AI.

[1599] A feature to notify users of alternative plans.

[1600] 2. Terminal

[1601] The terminal is a device used by the user to input planning information and receive notifications and alternatives. Generally, smartphones or tablets are used. Specific operations are as follows:

[1602] The user places an order and enters the delivery address and desired delivery time.

[1603] Receive notifications of initial and alternative plans from the server.

[1604] Choose whether to accept the alternative.

[1605] 3. Generative artificial intelligence (AI)

[1606] Generative AI is an AI model embedded within a server that generates optimal alternatives in response to real-time changes in the situation. Common AI models used include GPT-4 and BERT. The AI ​​performs the following processes:

[1607] Perform data analysis based on situational information.

[1608] Generate the optimal alternative in response to changing circumstances.

[1609] Create a prompt message and notify the user.

[1610] Hardware and software

[1611] hardware

[1612] Smartphones and tablets (user devices)

[1613] Servers (cloud-based is common)

[1614] software

[1615] Server software (e.g., Flask, Django)

[1616] Real-time monitoring systems (e.g., Geolocation API)

[1617] Generative AI (e.g. GPT-4, BERT)

[1618] Specific example

[1619] The following is a scenario in which a user orders a pizza:

[1620] 1. The user orders a pizza using a terminal and enters the delivery address and desired delivery time.

[1621] 2. The server receives this, generates an initial delivery plan, and notifies the user.

[1622] 3. If traffic congestion occurs during delivery, the server's real-time monitoring system will detect it.

[1623] 4. Generative AI generates optimal alternative routes and new delivery time suggestions based on changes in the situation.

[1624] 5. This alternative will be communicated to the user via the server, and the user will choose whether to accept it on their device.

[1625] Example of a prompt

[1626] "User ID: 123, pizza ordered. Traffic congestion occurred during delivery. Please offer the user an alternative: a restaurant discount coupon or another food item."

[1627] This embodiment of the invention makes it possible to respond immediately to real-time changes in circumstances and quickly provide users with the most suitable alternative. This, in turn, can improve the quality of food delivery services.

[1628] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1629] Step 1:

[1630] The user uses a terminal to enter order information and submits the product (e.g., pizza), delivery address, and desired delivery time.

[1631] Input: Product information, delivery address, desired delivery time

[1632] Output: Initial order data

[1633] Operation: Form input on the terminal

[1634] Step 2:

[1635] The terminal sends the order information entered by the user to the server.

[1636] Input: Initial order data

[1637] Output: Order data sent to the server

[1638] Operation: Sends data using HTTP requests

[1639] Step 3:

[1640] The server receives the initial order data and generates an initial delivery plan.

[1641] Input: Initial order data

[1642] Output: Initial delivery plan

[1643] Operation: A planning generation algorithm is executed to calculate the optimal delivery route and time.

[1644] Step 4:

[1645] The server notifies the terminal of the initial delivery plan and displays it to the user.

[1646] Input: Initial delivery plan

[1647] Output: Delivery plan notification displayed to the user

[1648] Operation: Sends data to the terminal via HTTP response.

[1649] Step 5:

[1650] It monitors situational information (traffic, weather, etc.) in real time and sends the data to the server.

[1651] Input: Real-time status information

[1652] Output: Updated status data

[1653] Operation: The monitoring system periodically collects data and sends it to the server.

[1654] Step 6:

[1655] The server analyzes real-time status information, and if a change in the situation is detected, it uses generative artificial intelligence (AI) to generate an alternative plan.

[1656] Input: Updated status data

[1657] Output: Alternative plan

[1658] Operation: An AI model (e.g., GPT-4) analyzes the input data and generates the optimal alternative.

[1659] Step 7:

[1660] The server notifies the terminal of the generated alternative plan along with a prompt message.

[1661] Input: Alternative plan, prompt statement

[1662] Output: Alternatives notified to the user

[1663] Operation: Sends notification data to the device via HTTP response.

[1664] Step 8:

[1665] The device displays an alternative plan notification to the user, who then chooses whether to accept the alternative.

[1666] Input: Alternative proposal notification

[1667] Output: User Selection

[1668] Operation: Selection behavior in the user interface

[1669] Step 9:

[1670] The user's selection is sent to the server, which then confirms and notifies the user of the final delivery plan based on the selection.

[1671] Input: User Selection

[1672] Output: Final delivery plan

[1673] Operation: Receive selected data, generate a new schedule, and send another notification.

[1674] This allows for immediate response to real-time changes in the food delivery process and enables the rapid provision of the most suitable alternative to the user.

[1675] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1676] This invention is a system that provides maximum support for the user's travel experience, and is characterized by its ability to flexibly respond to real-time changes in circumstances, as well as recognizing the user's emotions and adjusting alternatives and notification content accordingly.

[1677] Overall system configuration

[1678] The system consists of the following main components:

[1679] User device: A device such as a smartphone or computer where the user inputs travel plans and receives notifications and alternative suggestions. This includes cameras and microphones for emotion recognition.

[1680] Server: A central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed. It also incorporates an emotion engine.

[1681] Generative AI: This is a generative artificial intelligence (AI) embedded within a server that provides optimal alternative solutions based on real-time changes in the situation.

[1682] Emotion Engine: A system that recognizes a user's emotions through methods such as voice analysis and camera image analysis.

[1683] Operation on the user terminal

[1684] The user enters information into the device to plan their trip. Specifically, they enter the following information:

[1685] Travel destination

[1686] Scheduled visit date

[1687] Desired experiences and tourist destinations

[1688] means of transportation

[1689] The entered plan information is sent from the user terminal to the server.

[1690] Processing on the server

[1691] The server performs the following processes based on the received travel plan information:

[1692] The system analyzes the entered travel plan information and generates an initial travel plan by referencing databases of weather forecasts, traffic information, and tourist information. Furthermore, it monitors weather information, traffic information, and opening / closing status of tourist attractions in real time and detects changes in the situation based on this information.

[1693] The system periodically acquires the user's location information to understand their current situation.

[1694] Generating alternative plans using generative AI and emotion engines

[1695] The generative AI generates alternative travel plans based on data from the server's monitoring system, adapting to changing circumstances. This generation process includes:

[1696] Based on changing circumstances, for example, in the case of bad weather or traffic congestion, we can suggest new destinations or timings. Specifically, we can flexibly respond by changing from outdoor to indoor tourist attractions.

[1697] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this, it suggests relaxing plans if the user is stressed, and active plans if they are excited.

[1698] Example: If the system detects that the user is experiencing stress based on their voice or facial expressions, it might suggest that they take a break at a cafe.

[1699] Notification and selection of alternative plans

[1700] The server notifies the user's terminal of alternative plans created by generative AI. The content of the notification is also adjusted according to the user's emotions. For example, if the user is feeling stressed, the notification will be in a supportive tone.

[1701] The device provides an interface that displays alternative plans to the user and allows them to choose whether to accept or reject them.

[1702] The user reviews the notified alternative plan and chooses whether to accept it. If the user accepts the alternative plan, a new schedule is generated and displayed on the device.

[1703] Specific Scenario

[1704] As an example, consider a user planning a trip to Paris:

[1705] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[1706] 2. The terminal sends the entered information to the server.

[1707] 3. The server generates an initial travel plan based on the received information and notifies the user.

[1708] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[1709] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[1710] 6. The emotion engine recognizes the user's emotions from their voice and facial expressions, and adjusts the suggestions to match the user's mood. For example, if the user is tired, it will suggest an indoor cafe where they can rest.

[1711] 7. The server notifies the user of the adjusted alternative plan.

[1712] 8. The user reviews the alternative plan and chooses whether to accept it.

[1713] 9. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[1714] This system allows users to receive optimal suggestions tailored to their emotions, even in the face of unexpected situations, resulting in a better travel experience. The collaboration between an emotion engine and generative AI provides maximum support for the user's travels.

[1715] The following describes the processing flow.

[1716] Step 1:

[1717] The user enters travel plan information into the terminal, such as the travel destination, planned visit dates, desired experiences, and mode of transportation.

[1718] Step 2:

[1719] The terminal receives the entered travel plan information and sends it to the server.

[1720] Step 3:

[1721] The server analyzes the received travel plan information and compares it with databases containing weather forecasts, traffic information, and tourist information to generate an initial travel plan.

[1722] Step 4:

[1723] The server sends the initial travel plan to the user's terminal and notifies the user.

[1724] Step 5:

[1725] The user reviews their initial travel plan.

[1726] Step 6:

[1727] The server monitors weather, traffic, and tourist information for the travel destination in real time during the trip.

[1728] Step 7:

[1729] The server periodically retrieves the user's location information to understand their current status.

[1730] Step 8:

[1731] The server acquires the user's voice and image data from the camera, and uses an emotion engine to recognize the user's emotions.

[1732] Step 9:

[1733] When the server detects a change in circumstances (for example, a sudden change in weather, traffic congestion, or temporary closure of a tourist attraction), it uses generative artificial intelligence (AI) to generate an alternative travel plan.

[1734] Step 10:

[1735] Generative AI optimizes alternative plans based on the user's current emotions. For example, if the user is feeling stressed, it will suggest a plan that helps them relax.

[1736] Step 11:

[1737] The server sends the generated alternative travel plan to the user's terminal and notifies the user.

[1738] Step 12:

[1739] The device displays alternative travel plans to the user, allowing them to choose whether or not to accept them.

[1740] Step 13:

[1741] The user reviews the alternative travel plan and chooses whether to accept it.

[1742] Step 14:

[1743] The device will display the new schedule if the user accepts an alternative travel plan.

[1744] Step 15:

[1745] The server provides the necessary information (maps, navigation, details of destinations, etc.) based on the new schedule.

[1746] Step 16:

[1747] The user continues their journey according to the new schedule. During this time, they can check real-time information (traffic conditions, weather forecast, etc.) on their device as needed.

[1748] This system allows users to respond flexibly to unexpected situations and receive optimal suggestions tailored to their emotions, resulting in a better travel experience.

[1749] (Example 2)

[1750] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1751] Conventional travel planning support systems can provide alternative plans in response to real-time changes in circumstances, but they have the problem of not being able to adjust plans while considering the user's emotional state. As a result, users may experience stress or be unable to respond appropriately to unexpected situations, making it difficult to provide the optimal travel experience.

[1752] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1753] In this invention, the server includes means for receiving travel plan information entered by the user, means for generating an initial travel plan based on the travel plan information, means for monitoring weather information, traffic information, and sightseeing information in real time during the trip, generative artificial intelligence (AI) means for generating an alternative travel plan based on changes in circumstances, emotion recognition means for recognizing the user's emotional state and adjusting the alternative travel plan based on those emotions, means for notifying the user of the alternative travel plan, and means for allowing the user to choose whether or not to accept the alternative travel plan. As a result, the user can not only respond flexibly to real-time changes in circumstances but also receive an optimal travel plan that takes into account their own emotional state.

[1754] "Means for receiving travel plan information entered by the user" refers to a system that allows users to enter travel plan information via a terminal and send that information to a server.

[1755] "Travel plan information" is a general term for the information that users input regarding their trip, such as destination, planned dates of visit, desired experiences and tourist attractions, and mode of transportation.

[1756] "Means for generating initial travel plans" refers to a system for creating an initial travel schedule based on the received travel plan information.

[1757] "Means for monitoring weather, traffic, and tourist information in real time while traveling" refers to a system for continuously acquiring and monitoring the latest weather forecasts, traffic conditions, opening / closing information for tourist destinations, etc.

[1758] "Generative artificial intelligence (AI) means for generating alternative travel plans based on changes in circumstances" refers to artificial intelligence technology for generating new travel plans based on changes in circumstances detected by monitoring means.

[1759] "An emotion recognition system that recognizes the user's emotional state and adjusts alternative travel plans based on those emotions" refers to a system that uses cameras and microphones to analyze the user's emotions from their facial expressions and voice, and adjusts the travel plan accordingly.

[1760] "Means of notifying users of alternative travel plans" refers to a mechanism for informing users of the generated alternative travel plans. This includes push notifications and alert messages to the device.

[1761] "A means of allowing users to choose whether or not to accept alternative travel plans" refers to a mechanism that provides an interface where users can review proposed alternative travel plans and choose whether or not to accept them.

[1762] This invention is a system that provides maximum support for the user's travel experience, characterized by its ability to flexibly respond to real-time changes in circumstances and to recognize the user's emotions to adjust alternatives and notification content.

[1763] Overall system configuration

[1764] The system consists of the following main components:

[1765] User device: A device such as a smartphone or computer where the user inputs travel plans and receives notifications and alternative suggestions. This includes cameras and microphones for emotion recognition.

[1766] Server: A central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed. It also incorporates an emotion engine.

[1767] Generative AI: This is a generative artificial intelligence (AI) embedded within a server that provides optimal alternative solutions based on real-time changes in the situation.

[1768] Emotion Engine: A system that recognizes a user's emotions through methods such as voice analysis and camera image analysis.

[1769] Operation on the user terminal

[1770] The user enters information into the device to plan their trip. Specifically, they enter the following information:

[1771] Travel destination

[1772] Scheduled visit date

[1773] Desired experiences and tourist destinations

[1774] means of transportation

[1775] The entered plan information is sent from the user's terminal to the server. For example, when a user enters their destination and travel dates into a smartphone application and taps the "Send" button, the information is sent to the server.

[1776] Processing on the server

[1777] The server performs the following processes based on the received travel plan information:

[1778] The system analyzes the entered travel plan information and generates an initial travel plan by referencing databases such as weather forecasts, traffic information, and tourist information. For example, it retrieves weather information for specific days based on the travel dates and checks the operating status of tourist destinations to be visited.

[1779] The system monitors weather information, traffic information, and opening / closing status information for tourist attractions in real time, and detects changes in the situation based on this information. The server periodically updates this information via an API.

[1780] The system periodically acquires the user's location information to understand their current situation. Specifically, it collects GPS data from the device and sends it to the server.

[1781] Generating alternative plans using generative AI and emotion engines

[1782] The generative AI generates alternative travel plans based on data from the server's monitoring system, adapting to changing circumstances. This generation process includes:

[1783] Based on changing circumstances, the system suggests alternative destinations and timings, for example, in case of bad weather or traffic congestion. Specifically, it can flexibly adapt by changing from outdoor to indoor tourist attractions. For instance, if a planned outdoor activity is canceled due to rain, the system can generate a plan to visit a nearby museum.

[1784] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this, it suggests relaxing plans if the user is stressed, and active plans if they are excited. For example, if the system recognizes that the user is stressed based on their voice and facial expressions, it will suggest taking a break at a cafe.

[1785] Notification and selection of alternative plans

[1786] The server notifies the user's terminal of alternative plans created by generative AI. The content of the notification is also adjusted according to the user's emotions. For example, if the user is feeling stressed, the notification will be in a supportive tone.

[1787] The device provides an interface that displays alternative plans to the user and allows them to choose whether to accept or reject them. It displays push notifications and alert messages, allowing the user to tap to view details. For example, tapping a push notification displays detailed alternative plans and "Accept" or "Reject" buttons.

[1788] The user reviews the notified alternative plan and chooses whether to accept it. If the user accepts the alternative plan, the new schedule will be displayed on the device.

[1789] Specific example scenarios and prompt statements

[1790] As a specific scenario, let's consider a user planning a trip to Paris:

[1791] 1. The user plans a trip to Paris using their smartphone and wishes to visit the Louvre Museum.

[1792] 2. The terminal sends the entered information to the server.

[1793] 3. The server generates an initial travel plan based on the received information and notifies the user.

[1794] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[1795] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[1796] 6. The emotion engine recognizes the user's emotions from their voice and facial expressions, and adjusts the suggestions to match the user's feelings. For example, if the user is tired, it will suggest taking a break at a cafe.

[1797] 7. The server notifies the user of the adjusted alternative plan.

[1798] 8. The user reviews the alternative plan and chooses whether to accept it.

[1799] 9. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[1800] Example of a prompt

[1801] "I'm planning a trip to Paris, and I'd like some recommendations for indoor places to enjoy myself if the weather turns bad."

[1802] This system allows users to respond flexibly to unexpected situations while receiving optimal suggestions tailored to their individual emotions. The collaboration of an emotion engine and generative AI provides maximum support for the user's travels.

[1803] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1804] Step 1:

[1805] The user enters travel plan information into their device. This information includes destination, planned visit dates, desired experiences or attractions, and mode of transportation. After entering the information, tapping the "Submit" button saves the information to the device. This action prepares the entered information as data for use in the next processing step.

[1806] Step 2:

[1807] The terminal receives travel plan information entered by the user and sends it to the server. Specifically, the terminal uses its internet connection to send the user's entered travel plan information (destination, planned visit dates, desired experiences or sightseeing spots, and mode of transportation) to the server. After transmission, the server analyzes the received data.

[1808] Step 3:

[1809] The server generates an initial travel plan based on the received travel plan information. It takes travel plan information as input data and generates an initial travel plan as a processing result. Specifically, the server refers to databases such as weather forecasts, traffic information, and tourist information to create an optimal plan based on the travel dates and destination. This initial travel plan is then ready to be notified to the user.

[1810] Step 4:

[1811] The server monitors weather, traffic, and tourist information in real time. It acquires various real-time data via APIs as input and evaluates that information. Specifically, the server periodically updates this information and has the functionality to detect changes in the situation. This allows for the proactive detection of potential problems that may occur during a trip.

[1812] Step 5:

[1813] The server periodically acquires the user's location information to understand their current situation. It receives GPS data from the device as input data to determine the user's current location. Specifically, the device periodically sends GPS information to the server, which analyzes this data to check the user's movement.

[1814] Step 6:

[1815] The generative AI generates alternative travel plans based on data monitored on a server, adapting to changing circumstances. It takes real-time information and user location data as input and uses this to generate alternative plans. Specific examples include flexible plan generation, such as changing outdoor activities to indoor activities if bad weather is predicted.

[1816] Step 7:

[1817] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. It takes audio and image data as input and processes it to recognize the user's emotional state. Specifically, it uses image recognition technology to analyze facial expressions and speech recognition technology to evaluate voice tone and volume. This allows it to respond in ways such as suggesting relaxing plans if the user is feeling stressed.

[1818] Step 8:

[1819] The server notifies the user's terminal of the alternative plan created by the generative AI. It takes the alternative plan as input data and generates notification data as the processing result. Specifically, it sends push notifications or alert messages to the terminal so that the user can check them.

[1820] Step 9:

[1821] The device provides an interface that displays alternative plans to the user and allows them to choose whether to accept or reject them. It receives the alternative plan notified as input data and generates an interface that the user can select as a result of processing. Specifically, when the push notification is tapped, the detailed alternative plan is displayed, and "Accept" or "Reject" buttons are shown.

[1822] Step 10:

[1823] The user reviews the notified alternative plan and chooses whether to accept it. The notification details of the alternative plan are reviewed as input data, and the selection result is entered into the device as processing result. Specifically, if the user accepts, they tap the "Accept" button, and the new schedule is displayed on the device.

[1824] (Application Example 2)

[1825] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1826] Conventional travel planning support systems can provide alternatives to accommodate changes in circumstances during a trip, but they lack consideration for the user's emotions when generating alternatives or adjusting notification content, leading to problems such as users easily experiencing stress and dissatisfaction. Furthermore, the alternative suggestions are often mechanical, making it difficult to fully support the user's travel experience.

[1827] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving travel plan information input by the user, means for generating an initial travel plan based on the travel plan information, means for monitoring weather information, traffic information, and sightseeing information during the trip in real time, generative artificial intelligence (AI) means for generating an alternative travel plan based on the situation changes detected by the monitoring means, means for notifying the user of the alternative travel plan, emotion engine means for recognizing the user's emotions, means for adjusting the alternative plan and notification content according to the user's emotions based on the emotion engine means, and means for allowing the user to choose whether or not to accept the alternative travel plan. This makes it possible to always provide the optimal alternative plan that takes the user's emotions into consideration, and for the user to have a more comfortable travel experience.

[1828] A "user" is a system user who inputs travel plan information, receives notifications of alternative options, and makes a selection.

[1829] "Travel plan information" refers to information entered by the user, such as the travel destination, planned dates of visit, desired experiences or sightseeing spots, and mode of transportation.

[1830] An "initial travel plan" refers to the basic schedule and activities of a trip, which are generated based on the travel plan information entered by the user.

[1831] "Real-time monitoring" means keeping weather, traffic, and tourist information constantly updated to the latest state during your trip.

[1832] "Changes in circumstances" refer to changes in weather conditions, traffic conditions, and the opening / closing status of tourist attractions that may occur during your trip.

[1833] An "alternative travel plan" is a preliminary or modified travel plan that is generated in response to changes in circumstances monitored in real time.

[1834] "Generative artificial intelligence (AI) methods" refer to artificial intelligence systems that generate travel plans and alternative options based on collected data.

[1835] An "emotion engine" is a system that analyzes the user's voice and camera images to recognize the user's emotions.

[1836] "Means for adjusting alternatives and notification content" refers to means for appropriately changing alternatives and notification content based on the user's emotions recognized by the emotion engine.

[1837] "Location information" refers to information that indicates the user's current geographical location.

[1838] A "new schedule" is a new travel schedule generated based on the alternative travel plan accepted by the user.

[1839] This invention is a system that provides maximum support for the user's travel experience, characterized by its ability to flexibly respond to real-time changes in circumstances, recognize the user's emotions, and adjust alternatives and notifications accordingly. The system consists of the following main components.

[1840] Overall system configuration

[1841] 1. User terminal: A device such as a smartphone or personal computer where the user inputs travel plans and receives notifications and alternative suggestions. This terminal includes a camera and microphone for emotion recognition.

[1842] 2. Server: This is the central system that receives and analyzes travel planning information, monitors and evaluates weather, traffic, and sightseeing information in real time, and generates alternative plans as needed. The server incorporates generative artificial intelligence (AI) and an emotion engine.

[1843] Operation on the user terminal

[1844] The user enters travel plan information into the terminal. This information includes the travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation. This entered plan information is then sent from the user's terminal to the server.

[1845] Processing on the server

[1846] The server performs the following processing based on the received travel plan information.

[1847] The system analyzes the entered travel plan information and generates an initial travel plan by referencing databases of weather forecasts, traffic information, and tourist information. Furthermore, it monitors weather information, traffic information, and opening / closing status of tourist attractions in real time and detects changes in the situation based on this information.

[1848] The system periodically acquires the user's location information to understand their current situation.

[1849] Generating alternative plans using generative AI and emotion engines

[1850] The generative AI generates alternative travel plans that adapt to changing circumstances, based on data from the server's monitoring system. This generation process includes the following:

[1851] Based on changing circumstances, for example, in the case of bad weather or traffic congestion, we can suggest new destinations or timings. Specifically, we can flexibly respond by changing from outdoor to indoor tourist attractions.

[1852] The emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this, it suggests relaxing plans if the user is stressed, and active plans if they are excited.

[1853] Notification and selection of alternative plans

[1854] The server notifies the user's terminal of an alternative plan created by a generative AI. The content of the notification is also adjusted according to the user's emotions. For example, if the user is feeling stressed, the notification will be in a supportive tone. The user's terminal provides an interface that displays the alternative plan to the user and allows them to choose whether to accept it or not. If the user accepts the alternative plan, a new schedule is generated and displayed on the terminal.

[1855] Specific Scenario

[1856] For users planning a trip to Paris, the process would be as follows:

[1857] 1. The user plans a trip to Paris using their device and wishes to visit the Louvre Museum.

[1858] 2. The terminal sends the entered information to the server.

[1859] 3. The server generates an initial travel plan based on the received information and notifies the user.

[1860] 4. If bad weather is expected on the day of travel, the server will monitor the weather in real time and detect any changes in conditions.

[1861] 5. The generative AI suggests a visit to the Musée d'Orsay, an indoor attraction, as an alternative plan to cope with bad weather.

[1862] 6. The emotion engine recognizes the user's emotions from their voice and facial expressions, and adjusts the suggestions to match the user's mood. For example, if the user is tired, it will suggest an indoor cafe where they can rest.

[1863] 7. The server notifies the user of the adjusted alternative plan.

[1864] 8. The user reviews the alternative plan and chooses whether to accept it.

[1865] 9. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[1866] This system allows users to receive optimal suggestions tailored to their emotions, even in the face of unexpected situations, resulting in a better travel experience. The collaboration between an emotion engine and generative AI provides maximum support for the user's travels.

[1867] Example of a prompt:

[1868] "I want to build an application that suggests the best travel plan based on the user's emotions and weather information. Please suggest the best alternative for the following travel plan: Destination: Paris, Planned visits: Louvre Museum, Weather: Rain, User's emotions: Stressed."

[1869] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1870] Step 1:

[1871] The user enters travel plan information into the terminal. Specifically, they enter their travel destination, planned dates of visit, desired experiences and sightseeing spots, and mode of transportation. Based on the entered data, travel plan information is generated. This travel plan information is then sent from the terminal to the server.

[1872] Input: Travel destination, planned visit dates, desired experiences, sightseeing spots, mode of transportation

[1873] Output: Generation and transmission of travel plan information

[1874] Step 2:

[1875] The server generates an initial travel plan based on the received travel plan information. Specifically, it refers to databases such as weather forecasts, traffic information, and tourist information, and plans a schedule and destinations according to the entered information.

[1876] Input: Travel plan information

[1877] Output: Initial travel plan

[1878] Step 3:

[1879] The server monitors weather, traffic, and tourist information in real time. This monitoring process retrieves necessary information via APIs and stores it in a database. It also periodically retrieves user location information to understand their current situation. This information is then analyzed to detect changes in the situation.

[1880] Input: Current weather information, traffic information, tourist information, location information

[1881] Output: Latest status data

[1882] Step 4:

[1883] The server's generative artificial intelligence (AI) system generates alternative travel plans based on the latest acquired situational data. It flexibly adapts to changing circumstances, modifying the initial travel plan and suggesting new destinations and schedules. For example, it might change from outdoor to indoor tourist attractions in case of bad weather.

[1884] Input: Latest situation data, initial travel plan

[1885] Output: Alternative travel plan

[1886] Step 5:

[1887] The server's emotion engine analyzes data acquired from the user's camera and microphone to recognize the user's emotions. Based on this recognition, it suggests a relaxing plan if the user is feeling stressed, and an active plan if they are feeling excited.

[1888] Input: Camera data, audio data

[1889] Output: Emotion recognition result

[1890] Step 6:

[1891] The server adjusts alternative plans and notifications based on the user's emotions, as recognized by the emotion engine. For example, if the user is feeling stressed, it will send a supportive notification and add suggestions for places to rest.

[1892] Input: Emotion recognition results, alternative travel plans

[1893] Output: Adjusted alternative travel plan, notification content

[1894] Step 7:

[1895] The server notifies the user terminal of the adjusted alternative travel plan. The terminal provides an interface that displays this alternative plan to the user and allows them to choose whether to accept it or not.

[1896] Input: Adjusted alternative travel plan, notification details

[1897] Output: Notification to the user

[1898] Step 8:

[1899] The user reviews the notified alternative plan and chooses whether to accept it. If the user accepts the alternative plan, the server generates a new schedule and sends it to the device. The device displays the new schedule to the user, and the user continues their trip according to the new plan.

[1900] Input: Select an alternative plan

[1901] Output: New schedule, displayed to the user.

[1902] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1903] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1904] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1905] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1906] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1907] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1908] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1909] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1910] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1911] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1912] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1913] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1914] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1915] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1916] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1917] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1918] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1919] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1920] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1921] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1922] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1923] The following is further disclosed regarding the embodiments described above.

[1924] (Claim 1)

[1925] A means for receiving travel plan information entered by the user,

[1926] Means for generating an initial travel plan based on the aforementioned travel plan information,

[1927] A means of monitoring weather information, traffic information, and tourist information in real time while traveling,

[1928] Generative artificial intelligence (AI) means for generating an alternative travel plan based on the changes in circumstances detected by the monitoring means,

[1929] Means for notifying the user of the aforementioned alternative travel plan,

[1930] A means for allowing the user to choose whether or not to accept an alternative travel plan,

[1931] A system that includes this.

[1932] (Claim 2)

[1933] The system according to claim 1, wherein the generative artificial intelligence (AI) means for generating the alternative travel plan provides the optimal alternative using the user's location information.

[1934] (Claim 3)

[1935] The system according to claim 1, wherein the means for allowing the user to choose whether to accept an alternative travel plan further includes means for generating a new schedule based on the user's choice and notifying the user.

[1936] "Example 1"

[1937] (Claim 1)

[1938] A means for receiving travel plan information entered by the user,

[1939] Means for generating an initial travel plan based on the aforementioned travel plan information,

[1940] A means of monitoring weather information, traffic information, and tourist information in real time while traveling,

[1941] Generative artificial intelligence (AI) means for generating an alternative travel plan based on the changes in circumstances detected by the monitoring means,

[1942] Means for notifying the user of the aforementioned alternative travel plan,

[1943] A means for allowing the user to choose whether or not to accept an alternative travel plan,

[1944] If the user accepts an alternative travel plan, the means of displaying the new schedule and providing maps and navigation information as needed,

[1945] A system that includes this.

[1946] (Claim 2)

[1947] The system according to ...

Claims

1. A means for receiving travel plan information entered by the user, Means for generating an initial travel plan based on the aforementioned travel plan information, A means of monitoring weather, traffic, and tourist information in real time while traveling, A generative artificial intelligence means for generating an alternative travel plan based on the changes in circumstances detected by the monitoring means, Means for notifying the user of the aforementioned alternative travel plan, A means for allowing the user to choose whether or not to accept an alternative travel plan, A system that includes this.

2. The system according to claim 1, wherein the generative artificial intelligence means for generating the alternative travel plan provides the optimal alternative using the user's location information.

3. The system according to claim 1, wherein the means for allowing the user to choose whether to accept an alternative travel plan further includes means for generating a new schedule based on the user's choice and notifying the user.

Images