system

Abstract

We provide the system. [Solution] Means for obtaining input data from users, Means of obtaining real-time data through external services, A means of integrating acquired data and generating multiple routes, A means of converting the generated route into an image format, A means of outputting the optimal route for the user terminal, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the prior art, it has taken a lot of effort and time to easily obtain routes and travel plans suitable for an individual's lifestyle and current conditions. Also, it has been difficult to effectively integrate real-time information from multiple data sources and provide individualized route guidance. Therefore, there has been a demand for a system that can provide a user with a maximally comfortable and meaningful mobility experience.

Means for Solving the Problems

[0005] This invention provides a system that generates travel plans optimized for each user by acquiring input data from users and real-time data from external services, and integrating this data. Specifically, it integrates the acquired data, generates multiple routes via an AI model, and outputs them to the user's terminal in image format. Furthermore, it can utilize user feedback to improve the overall accuracy and usability of the system. As a result, users can enjoy a personalized travel experience through an intuitively usable platform.

[0006] "User input data" refers to data that users provide to the system, including location information, destinations, planned activities, and personal preference information.

[0007] "External services" refer to services operated by third parties that provide information such as traffic information, weather forecasts, and facility congestion status.

[0008] "Real-time data" refers to information that changes over time, such as current traffic conditions, weather conditions, and event information, and is obtained through external services.

[0009] "Integration" refers to the process of combining user input data with real-time data obtained from external services and organizing it into a format that can be analyzed and processed.

[0010] "Generating a route" means optimizing the means of transportation and routes to a destination based on integrated data, and calculating multiple routes to suggest to the user.

[0011] "Converting to an image format" refers to the process of visualizing the generated route information in a format that is easy to understand visually, such as a map or graphic.

[0012] A "user terminal" is a device used by a user to access the system, and includes smartphones, tablets, and personal computers.

[0013] "Feedback" is the act of users returning evaluations and opinions about their experience to the system, and this information is used to improve the system.

[0014] "Profile information" refers to personal information provided by users in advance, and is data used to provide services based on their behavioral patterns and preferences.

[0015] "Evaluation criteria" refer to the elements and standards that are emphasized in route selection and plan generation, and include aspects such as travel time, cost, and comfort. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention is a system that generates individually customized travel plans by having users input data specifying their destination, preferences, and mode of transportation, and combining that data with real-time data obtained from multiple external services. This system runs as a program on both the server and the terminal.

[0038] Users input their desired destinations and activities into the app using a smartphone or tablet. The entered data is immediately sent from the device to the server, which then stores this data in a database.

[0039] Next, the server communicates with external traffic information and weather forecast services via API to obtain real-time data. This data includes current traffic congestion, delay information for transportation services, the latest weather forecasts, and congestion levels at destination facilities.

[0040] The server processes the acquired data and analyzes it in combination with the user's input data. This analysis includes a process that uses an AI model to suggest the optimal travel route and travel plan. In particular, it generates multiple route options that take into account the user's preferences and conditions, and combines them to derive the most efficient and satisfying plan.

[0041] The server then converts these route suggestions into an image format that is easy to understand visually and sends it to the terminal. On the terminal, the route suggestions are visualized as a map or action plan, and the user can review the plan.

[0042] As a concrete example, let's consider a scenario where a user wants to visit a popular tourist destination on a holiday. The user enters the name of the tourist destination and the desired activities into the app. The server then considers traffic, weather, and facility congestion to suggest and provide the user with the optimal route combining various modes of transportation such as car, train, and walking.

[0043] This system allows users to easily obtain flexible travel plans tailored to their needs and conditions, resulting in a better travel experience.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] The user opens the app on their device and enters information such as their desired destination, activities, budget, and mode of transportation. The device then saves this input data locally as temporary data.

[0047] Step 2:

[0048] The terminal sends the entered data to the server. This data includes the user's pre-profile information and newly entered data each time. The server receives the data and stores it in the database.

[0049] Step 3:

[0050] The server obtains real-time data such as traffic information, weather forecasts, and destination congestion via external APIs. This aggregates the latest external information, which can then be used by users for planning.

[0051] Step 4:

[0052] The server integrates user input data with real-time data and generates route suggestions using an AI model. The calculated route suggestions are divided into multiple options based on evaluation criteria such as travel time, cost, and comfort.

[0053] Step 5:

[0054] The server selects the most optimal route and converts it into a visual image format. The visualized route plan is then prepared to be sent to the terminal as a map or schedule.

[0055] Step 6:

[0056] The device receives route suggestions sent from the server and presents them to the user. The user reviews these suggestions and makes adjustments as needed. At this point, the user can also receive alerts regarding congestion or delays.

[0057] Step 7:

[0058] After a trip or travel experience, users send feedback via their device. This feedback is then sent to a server, which uses it to improve the AI ​​algorithm. This results in more accurate plan generation for future trips.

[0059] (Example 1)

[0060] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0061] In modern travel, providing efficient routes tailored to individual needs quickly presents technical and infrastructure challenges. Furthermore, generating optimal travel plans based on user preferences and requests while considering fluctuating factors such as traffic conditions and weather is difficult.

[0062] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0063] In this invention, the server includes means for obtaining requests from users, means for obtaining time-series data through external information sources, means for combining the obtained data to generate multiple travel routes, means for converting the generated routes into visualization information, means for presenting the optimal route to the user's device, and means for optimizing the route according to the user's preferences using a generation AI model. This makes it possible to quickly provide users with the optimal itinerary, taking into account traffic conditions and weather conditions.

[0064] A "user" is an entity that inputs information and uses the system to obtain travel routes and travel plans.

[0065] A "request" is information that a user enters into an application to identify their destination, activity, and mode of transportation.

[0066] "External information sources" refer to third-party information systems that provide real-time data, such as traffic information services and weather forecast services.

[0067] "Time-series data" refers to data that changes over time, such as traffic conditions, weather information, and facility congestion levels.

[0068] "Combining acquired data" is the process of combining user request information with real-time data collected from external sources and performing a comprehensive analysis.

[0069] "Generating travel routes" refers to the process of formulating multiple possible travel plans based on user requests and real-time data.

[0070] "Visualized information" refers to information that converts generated travel routes and plans into formats that are easy for users to intuitively understand, such as maps and timelines.

[0071] "To present" means to display information on a device so that the user can review its contents and make a decision.

[0072] A "generative AI model" is an artificial intelligence technology used to learn user preferences and conditions and propose optimal travel routes and travel plans.

[0073] This invention relates to a system in which a user specifies their travel destination, preferences, and mode of transportation using a terminal, and a server generates an optimized travel plan based on this information. The system consists of a terminal and a server, and the specific hardware includes smartphones, tablets, and server computers. The software primarily consists of an application responsible for generating the travel plan and a server program.

[0074] First, the user uses a smartphone or tablet application to enter details such as their desired destination, activities, and mode of transportation. The user's request is then displayed as a prompt within the application. For example, the user might enter, "I'm planning a café hopping trip in Kyoto next weekend and would like to enjoy some traditional Japanese sweets there."

[0075] The terminal sends the entered data to the server. This server is responsible for obtaining time-series data from external sources via APIs. Real-time data, including traffic conditions, weather conditions, and facility congestion information, is sent to the server from external sources.

[0076] On the server, a generative AI model is used to analyze this acquired data and user input information to generate a travel plan. The generative AI model learns from the user's past data and preferences, enabling it to propose the most efficient and satisfying travel route.

[0077] For example, if a user expresses a desire to "enjoy surfing at Shonan Beach," the server can consider the congestion levels of surfing spots and transportation options to those locations, and then suggest recommended times and routes.

[0078] The plan generated by the server is converted into a visually easy-to-understand format and sent to the terminal. The plan received on the terminal is displayed as a map and a sequential action plan, allowing the user to immediately review the proposed plan and prepare to take action.

[0079] Thus, the present invention enables users to easily and accurately obtain travel plans and realize travel and transportation experiences optimized for them.

[0080] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0081] Step 1:

[0082] The user launches an application on their smartphone or tablet and enters their desired destination, activities, and mode of transportation. The application processes the entered information as text data and generates prompt statements. These prompt statements serve as foundational data for subsequent processing.

[0083] Step 2:

[0084] The terminal sends the generated prompt message to the server via internet communication. This transmission operation transfers the input data from the terminal to the server, preparing it for the next processing step. As a result, the user's request information is stored on the server.

[0085] Step 3:

[0086] The server, based on the user's request data received, collaborates with external information sources to collect time-series data. This process utilizes APIs from traffic and weather information services to obtain the latest traffic conditions, weather conditions, and facility congestion information. The input data is a prompt statement, and the output data is time-series data.

[0087] Step 4:

[0088] The server utilizes a generative AI model to integrate user input data with acquired time-series data to generate travel routes. In this process, the AI ​​model analyzes user preferences and conditions and generates multiple optimized travel plans. Specifically, it proposes several of the most efficient routes from the input data and prepares them for comparison and evaluation.

[0089] Step 5:

[0090] The server converts the generated travel plan into a format that is easy for the user to understand visually. Here, data processing is performed to visually represent the proposed route as a map or action plan. The input is the generated plan data, and the output is the visualized information.

[0091] Step 6:

[0092] The server sends the visualized information to the user's terminal. The user reviews the information received on the terminal and prepares to create a concrete action plan. The terminal displays prompts on the screen suggesting a specific travel plan, providing the foundation for proceeding to the next step.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] Traditional delivery systems struggled to provide optimal delivery routes in response to dynamic traffic conditions and weather changes. Furthermore, unexpected situations requiring route changes necessitated manual adjustments, leading to inefficiencies. This resulted in delivery personnel being unable to deliver smoothly, potentially lowering customer satisfaction.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes means for collecting input information from the user, means for acquiring dynamic data through an external information source, and means for integrating the acquired information and generating multiple routes. This enables the immediate output of the optimal route based on the dynamic data to the delivery person's terminal and the presentation of alternative routes in response to unexpected situations.

[0098] "User input information" refers to the details of the order received by the delivery person, and the information necessary for delivery.

[0099] "Dynamic data obtained through external sources" refers to communication methods for acquiring real-time, fluctuating external situational data, such as traffic conditions and weather information.

[0100] "Means for generating multiple routes" refers to algorithms that propose multiple efficient delivery routes for delivery personnel based on acquired dynamic data.

[0101] "Means of converting to a visual format" refers to methods of converting generated route information into maps or graphs that are easy for delivery personnel to understand.

[0102] A "delivery driver terminal" is an information processing device used by delivery drivers for their work, and includes smartphones and head-mounted displays.

[0103] "Means of suggesting alternative routes based on dynamic data" refers to a system that responds to real-time changes in traffic and weather information and proposes the most efficient alternative route.

[0104] "Response" refers to the reaction or feedback received from the user or delivery person.

[0105] "Means of improving the engine" refers to methods for improving the route generation algorithm based on the received response.

[0106] An "information storage system" is a data storage system used to store user characteristics and past data.

[0107] "Characteristic information" refers to data based on user preferences and behavioral history, and is used to customize services.

[0108] "Means for setting judgment criteria" refers to a method of determining criteria for selecting the optimal route based on user characteristic information, etc.

[0109] "Means for route re-evaluation" refers to a system that responds to unexpected traffic conditions or weather changes and immediately revises route selection.

[0110] To realize this invention, the delivery system uses a program consisting of a server and terminals. The server has the function of collecting input information from the user and, based on that, acquires real-time dynamic data from external information sources. For example, it uses the Google® Maps API to obtain traffic information and the OpenWeatherMap API to obtain weather information. The server aggregates and analyzes this data and uses an AI model to generate and evaluate multiple routes.

[0111] This generated route information is converted into a visual format and transmitted to the delivery person's terminal. The terminal is implemented in various forms, such as a smartphone or head-mounted display, providing the delivery person with visualized information. This information includes the optimal delivery route based on real-time dynamic data, as well as alternative routes for unexpected situations.

[0112] For example, when a delivery person is making multiple deliveries in a busy area, alternative routes to avoid congestion are suggested. In case of a sudden change in weather, alternative routes to avoid rain are also considered.

[0113] An example of a prompt for the generating AI model is, "Provide the shortest and most efficient delivery route based on current traffic, weather, and destination information obtained from the API." Based on this prompt, the AI ​​model demonstrates its ability to generate the optimal delivery strategy in real time, tailored to the situation.

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] The user enters order information using the delivery app. Information such as order details, delivery address, and desired delivery time is entered and sent from the device to the server. Based on this input information, the server prepares the initial data for generating the delivery route.

[0117] Step 2:

[0118] The server obtains real-time dynamic data via external APIs. It uses the Google Maps API to collect traffic information and the OpenWeatherMap API to obtain weather information. This data is stored in a database as variables necessary for generating delivery routes.

[0119] Step 3:

[0120] The server integrates order information from users with acquired movement data and generates multiple delivery routes using a generative AI model. Given the prompt "Generate the optimal route based on current traffic and weather data," the AI ​​model proposes an efficient delivery plan. This results in the server outputting the route with the highest delivery efficiency.

[0121] Step 4:

[0122] The delivery routes generated by the server are visualized and sent to the terminal. The terminal overlays this information on a map, displaying it in a format that is easy for delivery personnel to understand. As a result, the optimal delivery route is clearly visualized.

[0123] Step 5:

[0124] During delivery, the user's device reacquires real-time movement data and sends updated information to the server. Based on this information, the server uses an AI model to re-evaluate the route as needed and generate alternative routes. At this time, the AI ​​model is given the prompt message "Re-evaluate the route based on the newly changed movement information," and the optimal route is immediately readjusted.

[0125] Step 6:

[0126] The terminal overlays the regenerated route information onto the map again, providing delivery personnel with the latest travel instructions. This ensures that the optimal route is always guaranteed in real time.

[0127] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0128] This invention is a system aimed at providing travel and transportation plans that take user emotions into consideration, and incorporates an emotion recognition engine. In addition to user input data and real-time data acquired from external sources, this system identifies the user's emotional state in real time and generates a route suggestion that is adjusted based on that.

[0129] When users input their desired destinations or activities using their device, they can also indicate their current emotional state through voice or text. The device uses an emotion recognition engine to extract emotional data from these inputs and send it to the server.

[0130] The server collects real-time traffic information, weather data, and other data from external services and integrates it with sentiment data sent by users. This allows it to utilize an AI model to generate the route that best matches the user's emotional state when sentiment is explicitly expressed.

[0131] For example, if a user expresses a desire to relax, the server can suggest quiet routes or travel plans surrounded by nature. Conversely, if a user expresses a desire to be active, the server will guide them to routes and destinations with plenty of activities.

[0132] The generated route is converted into a visually easy-to-understand image format and sent to the device. The user uses this information to navigate, and after the experience, can provide feedback via the device on how well the navigation met their emotional needs.

[0133] The server receives user feedback, reruns the emotion engine analysis, and stores it as training data for the entire system. This allows the system to continuously improve so that future suggestions are better aligned with the user's emotions.

[0134] Thus, the present invention is a system that makes it possible to provide an unprecedentedly customized travel experience by directly taking into account the user's emotions.

[0135] The following describes the processing flow.

[0136] Step 1:

[0137] Users use their devices to input desired destinations and activities into the app. To reflect emotions, the app also provides features for easily adding emotions via voice or text input. The device temporarily stores the entered data, and the emotional data is analyzed by an emotion recognition engine.

[0138] Step 2:

[0139] The terminal sends the analyzed emotion data and basic input information to the server. The server receives this information, records it in a database, and prepares route calculations that reflect the emotions in real time.

[0140] Step 3:

[0141] The server uses external APIs to obtain real-time data such as the latest traffic information, weather forecasts, and congestion levels at tourist attractions. This data is integrated with sentiment data and used as analytical base data to provide users with the best possible options.

[0142] Step 4:

[0143] The server uses an AI model to generate multiple route suggestions that take into account the user's emotional state. Specifically, if the user is seeking relaxation, a quiet and less crowded route will be considered; if they are seeking excitement, a route offering an active experience will be selected.

[0144] Step 5:

[0145] The server generates an image of the optimal route it has selected for visual display and sends this information to the terminal. The terminal receives this and presents it to the user as a map or plan. The user confirms the selected route.

[0146] Step 6:

[0147] If a user's emotions change during a trip or journey, the device can provide a function to re-enter those emotions and send them to the server. The server receives this change, recalculates the route if necessary, and sends the updated route to the device.

[0148] Step 7:

[0149] After a user enjoys their trip, they send feedback about their overall experience and emotional satisfaction to a server via their device. The server analyzes this information using an emotion engine and adjusts the AI ​​model so that it is reflected in future route generation. This allows the system to continuously improve and enable more accurate and personalized suggestions.

[0150] (Example 2)

[0151] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0152] Conventional travel and transportation planning systems propose uniform routes without considering user emotions, making it difficult to customize them to meet diverse emotional needs. Furthermore, there is little opportunity for improvement based on feedback, resulting in insufficient quality suggestions to enhance user satisfaction.

[0153] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0154] In this invention, the server includes means for acquiring user emotions and destination input, means for identifying the user's emotional state using an emotion recognition engine, and means for acquiring real-time environmental data from an external information provider. This makes it possible to generate an optimal travel and transportation plan that corresponds to the user's emotions and to improve it with each suggestion.

[0155] "Means for obtaining user sentiment and destination input" refers to a function that receives sentiment information and desired destination information entered by the user through a terminal.

[0156] An "emotion recognition engine" is a software architecture that analyzes a user's emotional state from voice and text data and identifies it in real time.

[0157] An "external information provider" is an external service that can acquire environmental data such as traffic conditions and weather information in real time via the internet.

[0158] A "generative artificial intelligence model" is an artificial intelligence algorithm that uses accumulated data to calculate the optimal route.

[0159] "Means of converting to visual information format" refers to the process of converting the generated route information into an image format that can be easily understood by the user.

[0160] An "information storage device" is a memory structure that holds user input information and feedback, and uses it for subsequent data processing and algorithm improvement.

[0161] "Means for selecting the optimal route" refers to a method for determining the best travel path for a user based on their emotional state and real-time data.

[0162] To implement this invention, the user uses a terminal to input information about desired destinations and activities in voice or text format, and to indicate their current emotional state. The terminal receives this input and extracts emotional data using an emotion recognition engine. This emotion recognition engine may also use PyTorch or other similar processing software and a high-performance computing device to analyze the voice data.

[0163] The device sends emotion data and destination data to the server. The server obtains real-time traffic and weather information from external services. This may include using the Google Maps API or other information-providing APIs. Based on the data thus collected, the server generates the optimal route using a generative AI model. This generative AI model is specifically tailored to calculate routes that are appropriate for the user's emotions and is implemented using deep learning libraries such as TENSORFLOW®.

[0164] The generated route is converted into a visual information format using OpenCV or a similar image processing library. This image is designed to allow the user to intuitively understand the route. The converted visual information is sent to the user's terminal via the internet and displayed on the terminal. This allows the user to visually confirm a concrete travel plan.

[0165] After the journey, users can provide feedback on how well the experience met their emotional needs. This feedback is entered from the terminal and sent to the server. The server analyzes this feedback, performs a re-analysis using the emotion engine, and stores it as training data for the entire system, thereby continuously improving the generative AI model.

[0166] For example, if a user is feeling "I want to relax," a possible prompt might be, "Please suggest a travel plan that is best suited for when the user is feeling relaxed." Based on this prompt, the server can utilize a generative AI model to suggest a route that passes through a quiet natural environment.

[0167] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0168] Step 1:

[0169] Users input their desired destinations and activities via text or voice using their device, and also indicate their current emotional state. Examples of data users might input include "I want to relax" or "I want to be active." This input data is processed by the device's emotion recognition engine.

[0170] Step 2:

[0171] The device receives voice and text data input from the user. It then uses an emotion recognition engine to extract emotional data. Specifically, voice data is converted to text via a speech processing algorithm, and then emotional components are analyzed using natural language processing techniques. The output generates data indicating the user's emotional state.

[0172] Step 3:

[0173] The device sends the extracted sentiment data and destination data to the server. A secure and reliable data communication protocol is used for this transmission. The output data includes the user's sentiment data and destination information.

[0174] Step 4:

[0175] The server obtains real-time traffic and weather information from external information providers based on the received sentiment and destination data. Specifically, it collects data from external services using APIs and stores it in an internal database. Traffic conditions and weather information are provided as output.

[0176] Step 5:

[0177] The server utilizes a generative AI model to integrate user sentiment data, destination information, and real-time external data. Once the input data is provided to the AI ​​model, it executes a predictive algorithm to calculate the optimal route. During this process, it receives a prompt instructing it to "suggest the optimal route based on the sentiment expressed by the user." The optimal route is then generated as output.

[0178] Step 6:

[0179] The server converts the generated optimal route into a visual information format. This conversion uses an image processing library to generate a map image. The output is a route display image that the user can intuitively understand.

[0180] Step 7:

[0181] The server sends the converted route image to the terminal. The communication method used here is a network protocol that enables real-time data transfer. Once the output data reaches the terminal, the user displays it to confirm the next traverse plan.

[0182] Step 8:

[0183] Users actually travel based on the proposed travel plan. After the trip, they input feedback via their device about how well the experienced route met their emotional needs. The input data is provided as subjective emotional evaluations and opinions on the proposed plan.

[0184] Step 9:

[0185] The server receives user feedback and performs a re-analysis using the sentiment engine. The received feedback is stored as system training data, contributing to the continuous improvement of the AI ​​model. To improve the quality of future route suggestions, the generative AI model is updated based on a new training cycle.

[0186] (Application Example 2)

[0187] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0188] Traditional travel experiences often provide a uniform route without considering the user's emotions, making it difficult to offer an optimal travel plan that meets the individual emotional needs of each user. Therefore, there is a growing demand for personalized routes that cater to users' emotions, such as whether they want to relax or be active.

[0189] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0190] In this invention, the server includes means for acquiring input data from the user and identifying their emotional state, means for acquiring real-time data through an external information source, and means for integrating the acquired data and generating multiple travel routes based on the user's emotional state. This makes it possible to provide a personalized travel plan that matches the user's emotions.

[0191] "Input data" refers to information collected from users and is used to identify their emotional state.

[0192] "Emotional state" refers to the user's psychological state, which can change in response to suggested travel routes.

[0193] "External information sources" refer to external data providers that offer real-time data such as traffic information and weather information.

[0194] A "visual format" is a graphical format displayed on a digital screen, provided in a form that is easily recognizable to the user.

[0195] An "information storage device" refers to a device or system that stores acquired data and keeps it in a state where it can be used for future analysis and reference.

[0196] An "emotional state profile" refers to a set of individual datasets that accumulate users' past emotional input data and are used in suggesting travel routes.

[0197] A "generative algorithm" refers to a procedure or method for calculating the optimal travel route based on input data.

[0198] The system that realizes this application primarily uses an emotion recognition engine, an external data acquisition module, an AI route generation engine, and a feedback learning module. The system is installed in an autonomous vehicle equipped with an onboard computer.

[0199] The server uses an emotion recognition engine to identify the user's emotional state based on the data they input. Specifically, it analyzes voice and text input using an emotion recognition API (e.g., Google's Speech Emotion Recognition API) to identify the emotions the user is currently experiencing.

[0200] Next, the server obtains real-time traffic and weather information from external sources via an external data acquisition module. This module utilizes information sources such as the Google Maps API and the Weather Data API.

[0201] The AI ​​route generation engine integrates acquired emotional state data with external information to generate multiple travel routes that best match the user's emotions. A custom model developed using TensorFlow is used for this process. The generated routes are converted into a visual format and displayed on the in-car display.

[0202] Users travel using the provided routes and provide feedback from their devices on how well the travel experience met their emotional needs. The server receives this feedback and stores it as data to improve the accuracy of the generation algorithm using a feedback learning module.

[0203] For example, if a user planning a family drive one afternoon inputs the emotion "I want to enjoy nature," the system can suggest a relaxing route. In this way, the system provides a travel experience tailored to the individual user's emotional needs.

[0204] An example of a prompt message is as follows:

[0205] Receive emotional input from the user such as "I want to relax," and use an AI model to generate the optimal route while considering available traffic and weather information, then display it on the in-car display.

[0206] This makes it possible to smoothly provide personalized travel plans that match the user's emotions.

[0207] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0208] Step 1:

[0209] The user inputs their emotions via voice or text using an in-vehicle terminal. This input is received by the emotion recognition engine. The input data is provided to the system as voice or text, and the emotional state is identified through the emotion recognition API. As a result, the user's emotions are identified, and analyzed emotional state data is generated.

[0210] Step 2:

[0211] The server obtains real-time traffic and weather information from external sources. This information is obtained using the Google Maps API and other weather data APIs. The server sends requests to the APIs and receives information about traffic conditions and weather in the response. It sends traffic and weather requests as input and receives the latest traffic and weather data as output.

[0212] Step 3:

[0213] The server integrates the data obtained in Step 1 and Step 2 and inputs it into the AI ​​route generation engine. The AI ​​route generation engine is a custom model using TensorFlow and generates multiple travel routes. Emotional state data and real-time data are passed to the AI ​​engine as input, and multiple route suggestions that best match the user's emotions are generated as output.

[0214] Step 4:

[0215] The server converts the generated route into a visual format and outputs it to the in-vehicle display. This process transforms the proposed route into a user-friendly graphical representation. It accepts route information in text or data format as input and provides the terminal with the corresponding image or map output.

[0216] Step 5:

[0217] Users experience a suggested travel route and then send feedback from their device to the server. The feedback evaluates satisfaction with the travel experience and how well it met their emotional needs, and the evaluation data is sent to the server as input.

[0218] Step 6:

[0219] The server analyzes the received feedback using a feedback learning module to improve the generation algorithm. The analysis extracts information that contributes to improving the accuracy of the AI ​​model, and this information is fed back into the AI ​​model. It receives user feedback data as input and produces an improved generation model as output.

[0220] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0221] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0222] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0223] [Second Embodiment]

[0224] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0225] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0226] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0227] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0228] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0229] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0230] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0231] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0232] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0233] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0234] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0235] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0236] This invention is a system that generates individually customized travel plans by having users input data specifying their destination, preferences, and mode of transportation, and combining that data with real-time data obtained from multiple external services. This system runs as a program on both the server and the terminal.

[0237] Users input their desired destinations and activities into the app using a smartphone or tablet. The entered data is immediately sent from the device to the server, which then stores this data in a database.

[0238] Next, the server communicates with external traffic information and weather forecast services via API to obtain real-time data. This data includes current traffic congestion, delay information for transportation services, the latest weather forecasts, and congestion levels at destination facilities.

[0239] The server processes the acquired data and analyzes it in combination with the user's input data. This analysis includes a process that uses an AI model to suggest the optimal travel route and travel plan. In particular, it generates multiple route options that take into account the user's preferences and conditions, and combines them to derive the most efficient and satisfying plan.

[0240] The server then converts these route suggestions into an image format that is easy to understand visually and sends it to the terminal. On the terminal, the route suggestions are visualized as a map or action plan, and the user can review the plan.

[0241] As a concrete example, let's consider a scenario where a user wants to visit a popular tourist destination on a holiday. The user enters the name of the tourist destination and the desired activities into the app. The server then considers traffic, weather, and facility congestion to suggest and provide the user with the optimal route combining various modes of transportation such as car, train, and walking.

[0242] This system allows users to easily obtain flexible travel plans tailored to their needs and conditions, resulting in a better travel experience.

[0243] The following describes the processing flow.

[0244] Step 1:

[0245] The user opens the app on their device and enters information such as their desired destination, activities, budget, and mode of transportation. The device then saves this input data locally as temporary data.

[0246] Step 2:

[0247] The terminal sends the entered data to the server. This data includes the user's pre-profile information and newly entered data each time. The server receives the data and stores it in the database.

[0248] Step 3:

[0249] The server obtains real-time data such as traffic information, weather forecasts, and destination congestion via external APIs. This aggregates the latest external information, which can then be used by users for planning.

[0250] Step 4:

[0251] The server integrates user input data with real-time data and generates route suggestions using an AI model. The calculated route suggestions are divided into multiple options based on evaluation criteria such as travel time, cost, and comfort.

[0252] Step 5:

[0253] The server selects the most optimal route and converts it into a visual image format. The visualized route plan is then prepared to be sent to the terminal as a map or schedule.

[0254] Step 6:

[0255] The device receives route suggestions sent from the server and presents them to the user. The user reviews these suggestions and makes adjustments as needed. At this point, the user can also receive alerts regarding congestion or delays.

[0256] Step 7:

[0257] After a trip or travel experience, users send feedback via their device. This feedback is then sent to a server, which uses it to improve the AI ​​algorithm. This results in more accurate plan generation for future trips.

[0258] (Example 1)

[0259] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0260] In modern travel, providing efficient routes tailored to individual needs quickly presents technical and infrastructure challenges. Furthermore, generating optimal travel plans based on user preferences and requests while considering fluctuating factors such as traffic conditions and weather is difficult.

[0261] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0262] In this invention, the server includes means for obtaining requests from users, means for obtaining time-series data through external information sources, means for combining the obtained data to generate multiple travel routes, means for converting the generated routes into visualization information, means for presenting the optimal route to the user's device, and means for optimizing the route according to the user's preferences using a generation AI model. This makes it possible to quickly provide users with the optimal itinerary, taking into account traffic conditions and weather conditions.

[0263] A "user" is an entity that inputs information and uses the system to obtain travel routes and travel plans.

[0264] A "request" is information that a user enters into an application to identify their destination, activity, and mode of transportation.

[0265] "External information sources" refer to third-party information systems that provide real-time data, such as traffic information services and weather forecast services.

[0266] "Time-series data" refers to data that changes over time, such as traffic conditions, weather information, and facility congestion levels.

[0267] "Combining acquired data" is the process of combining user request information with real-time data collected from external sources and performing a comprehensive analysis.

[0268] "Generating travel routes" refers to the process of formulating multiple possible travel plans based on user requests and real-time data.

[0269] "Visualized information" refers to information that converts generated travel routes and plans into formats that are easy for users to intuitively understand, such as maps and timelines.

[0270] "To present" means to display information on a device so that the user can review its contents and make a decision.

[0271] A "generative AI model" is an artificial intelligence technology used to learn user preferences and conditions and propose optimal travel routes and travel plans.

[0272] This invention relates to a system in which a user specifies their travel destination, preferences, and mode of transportation using a terminal, and a server generates an optimized travel plan based on this information. The system consists of a terminal and a server, and the specific hardware includes smartphones, tablets, and server computers. The software primarily consists of an application responsible for generating the travel plan and a server program.

[0273] First, the user uses a smartphone or tablet application to enter details such as their desired destination, activities, and mode of transportation. The user's request is then displayed as a prompt within the application. For example, the user might enter, "I'm planning a café hopping trip in Kyoto next weekend and would like to enjoy some traditional Japanese sweets there."

[0274] The terminal sends the entered data to the server. This server is responsible for obtaining time-series data from external sources via APIs. Real-time data, including traffic conditions, weather conditions, and facility congestion information, is sent to the server from external sources.

[0275] On the server, a generative AI model is used to analyze this acquired data and user input information to generate a travel plan. The generative AI model learns from the user's past data and preferences, enabling it to propose the most efficient and satisfying travel route.

[0276] For example, if a user expresses a desire to "enjoy surfing at Shonan Beach," the server can consider the congestion levels of surfing spots and transportation options to those locations, and then suggest recommended times and routes.

[0277] The plan generated by the server is converted into a visually easy-to-understand format and sent to the terminal. The plan received on the terminal is displayed as a map and a sequential action plan, allowing the user to immediately review the proposed plan and prepare to take action.

[0278] Thus, the present invention enables users to easily and accurately obtain travel plans and realize travel and transportation experiences optimized for them.

[0279] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0280] Step 1:

[0281] The user launches an application on their smartphone or tablet and enters their desired destination, activities, and mode of transportation. The application processes the entered information as text data and generates prompt statements. These prompt statements serve as foundational data for subsequent processing.

[0282] Step 2:

[0283] The terminal sends the generated prompt sentence to the server via Internet communication. Through this sending operation, the input data from the terminal is passed to the server, and preparations for the next processing step are completed. As a result, the user's request information is stored in the server.

[0284] Step 3:

[0285] Based on the received user request data, the server collaborates with external information sources for time-series data collection. In this process, APIs of traffic information services and weather information services are used to obtain the latest traffic conditions, weather conditions, and congestion information of facilities. The input data is the prompt sentence, and time-series data is generated as the output data.

[0286] Step 4:

[0287] The server utilizes the generative AI model to integrate the user's input data and the obtained time-series data and generate a travel route. In this process, the AI model analyzes the user's preferences and conditions and generates multiple optimized travel plans. Specifically, several of the most efficient routes are proposed from the input data, and preparations are made for comparing and considering them.

[0288] Step 5:

[0289] The server converts the generated travel plan into a format that is visually easy for the user to understand. Here, data processing is performed to visually represent the route plan in a map format or as an action plan. The input is the generated plan data, and the output is the visualized information.

[0290] Step 6:

[0291] The server sends the visualized information to the user's terminal. The user reviews the information received on the terminal and prepares to create a concrete action plan. The terminal displays prompts on the screen suggesting a specific travel plan, providing the foundation for proceeding to the next step.

[0292] (Application Example 1)

[0293] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0294] Traditional delivery systems struggled to provide optimal delivery routes in response to dynamic traffic conditions and weather changes. Furthermore, unexpected situations requiring route changes necessitated manual adjustments, leading to inefficiencies. This resulted in delivery personnel being unable to deliver smoothly, potentially lowering customer satisfaction.

[0295] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0296] In this invention, the server includes means for collecting input information from the user, means for acquiring dynamic data through an external information source, and means for integrating the acquired information and generating multiple routes. This enables the immediate output of the optimal route based on the dynamic data to the delivery person's terminal and the presentation of alternative routes in response to unexpected situations.

[0297] "User input information" refers to the details of the order received by the delivery person, and the information necessary for delivery.

[0298] "Dynamic data obtained through external sources" refers to communication methods for acquiring real-time, fluctuating external situational data, such as traffic conditions and weather information.

[0299] The "means for generating multiple routes" refers to an algorithm for proposing multiple efficient delivery routes for delivery staff based on the acquired dynamic data.

[0300] The "means for converting to a visual format" refers to a method for converting the generated route information into a map or graph format that is easy for delivery staff to understand.

[0301] The "delivery staff terminal" is an information processing device used by delivery staff, such as a smartphone or a head-mounted display.

[0302] The "means for presenting an alternative route based on dynamic data" refers to a system that proposes the most efficient alternative route in response to real-time changes in traffic and weather information.

[0303] "Response" refers to the reaction or feedback received from the user or delivery staff.

[0304] The "means for improving the engine" refers to a method for improving the route generation algorithm based on the received response.

[0305] The "information repository" is a data storage system for storing user characteristics and past data.

[0306] "Characteristic information" is data based on user preferences and behavior history, and is used for customizing services.

[0307] The "means for setting criteria for determination" refers to a method for determining criteria for selecting an optimal route based on user characteristic information, etc.

[0308] The "means for re-evaluating the route" refers to a system that immediately reconsiders the route selection in response to unexpected traffic conditions and weather changes.

[0309] To realize this invention, the delivery system uses a program consisting of a server and terminals. The server has the function of collecting input information from the user and, based on that, acquires real-time dynamic data from external information sources. For example, it uses the Google Maps API to obtain traffic information and the OpenWeatherMap API to obtain weather information. The server aggregates and analyzes this data and uses an AI model to generate and evaluate multiple routes.

[0310] This generated route information is converted into a visual format and transmitted to the delivery person's terminal. The terminal is implemented in various forms, such as a smartphone or head-mounted display, providing the delivery person with visualized information. This information includes the optimal delivery route based on real-time dynamic data, as well as alternative routes for unexpected situations.

[0311] For example, when a delivery person is making multiple deliveries in a busy area, alternative routes to avoid congestion are suggested. In case of a sudden change in weather, alternative routes to avoid rain are also considered.

[0312] An example of a prompt for the generating AI model is, "Provide the shortest and most efficient delivery route based on current traffic, weather, and destination information obtained from the API." Based on this prompt, the AI ​​model demonstrates its ability to generate the optimal delivery strategy in real time, tailored to the situation.

[0313] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0314] Step 1:

[0315] The user enters order information using the delivery app. Information such as order details, delivery address, and desired delivery time is entered and sent from the device to the server. Based on this input information, the server prepares the initial data for generating the delivery route.

[0316] Step 2:

[0317] The server obtains real-time dynamic data via external APIs. It uses the Google Maps API to collect traffic information and the OpenWeatherMap API to obtain weather information. This data is stored in a database as variables necessary for generating delivery routes.

[0318] Step 3:

[0319] The server integrates order information from users with acquired movement data and generates multiple delivery routes using a generative AI model. Given the prompt "Generate the optimal route based on current traffic and weather data," the AI ​​model proposes an efficient delivery plan. This results in the server outputting the route with the highest delivery efficiency.

[0320] Step 4:

[0321] The delivery routes generated by the server are visualized and sent to the terminal. The terminal overlays this information on a map, displaying it in a format that is easy for delivery personnel to understand. As a result, the optimal delivery route is clearly visualized.

[0322] Step 5:

[0323] During delivery, the user's device reacquires real-time movement data and sends updated information to the server. Based on this information, the server uses an AI model to re-evaluate the route as needed and generate alternative routes. At this time, the AI ​​model is given the prompt message "Re-evaluate the route based on the newly changed movement information," and the optimal route is immediately readjusted.

[0324] Step 6:

[0325] The terminal overlays the regenerated route information onto the map again, providing delivery personnel with the latest travel instructions. This ensures that the optimal route is always guaranteed in real time.

[0326] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0327] This invention is a system aimed at providing travel and transportation plans that take user emotions into consideration, and incorporates an emotion recognition engine. In addition to user input data and real-time data acquired from external sources, this system identifies the user's emotional state in real time and generates a route suggestion that is adjusted based on that.

[0328] When users input their desired destinations or activities using their device, they can also indicate their current emotional state through voice or text. The device uses an emotion recognition engine to extract emotional data from these inputs and send it to the server.

[0329] The server collects real-time traffic information, weather data, and other data from external services and integrates it with sentiment data sent by users. This allows it to utilize an AI model to generate the route that best matches the user's emotional state when sentiment is explicitly expressed.

[0330] For example, if a user expresses a desire to relax, the server can suggest quiet routes or travel plans surrounded by nature. Conversely, if a user expresses a desire to be active, the server will guide them to routes and destinations with plenty of activities.

[0331] The generated route is converted into a visually easy-to-understand image format and sent to the device. The user uses this information to navigate, and after the experience, can provide feedback via the device on how well the navigation met their emotional needs.

[0332] The server receives user feedback, reruns the emotion engine analysis, and stores it as training data for the entire system. This allows the system to continuously improve so that future suggestions are better aligned with the user's emotions.

[0333] Thus, the present invention is a system that makes it possible to provide an unprecedentedly customized travel experience by directly taking into account the user's emotions.

[0334] The following describes the processing flow.

[0335] Step 1:

[0336] Users use their devices to input desired destinations and activities into the app. To reflect emotions, the app also provides features for easily adding emotions via voice or text input. The device temporarily stores the entered data, and the emotional data is analyzed by an emotion recognition engine.

[0337] Step 2:

[0338] The terminal sends the analyzed emotion data and basic input information to the server. The server receives this information, records it in a database, and prepares route calculations that reflect the emotions in real time.

[0339] Step 3:

[0340] The server uses external APIs to obtain real-time data such as the latest traffic information, weather forecasts, and congestion levels at tourist attractions. This data is integrated with sentiment data and used as analytical base data to provide users with the best possible options.

[0341] Step 4:

[0342] The server uses an AI model to generate multiple route suggestions that take into account the user's emotional state. Specifically, if the user is seeking relaxation, a quiet and less crowded route will be considered; if they are seeking excitement, a route offering an active experience will be selected.

[0343] Step 5:

[0344] The server generates an image of the optimal route it has selected for visual display and sends this information to the terminal. The terminal receives this and presents it to the user as a map or plan. The user confirms the selected route.

[0345] Step 6:

[0346] If a user's emotions change during a trip or journey, the device can provide a function to re-enter those emotions and send them to the server. The server receives this change, recalculates the route if necessary, and sends the updated route to the device.

[0347] Step 7:

[0348] After a user enjoys their trip, they send feedback about their overall experience and emotional satisfaction to a server via their device. The server analyzes this information using an emotion engine and adjusts the AI ​​model so that it is reflected in future route generation. This allows the system to continuously improve and enable more accurate and personalized suggestions.

[0349] (Example 2)

[0350] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0351] Conventional travel and transportation planning systems propose uniform routes without considering user emotions, making it difficult to customize them to meet diverse emotional needs. Furthermore, there is little opportunity for improvement based on feedback, resulting in insufficient quality suggestions to enhance user satisfaction.

[0352] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0353] In this invention, the server includes means for acquiring user emotions and destination input, means for identifying the user's emotional state using an emotion recognition engine, and means for acquiring real-time environmental data from an external information provider. This makes it possible to generate an optimal travel and transportation plan that corresponds to the user's emotions and to improve it with each suggestion.

[0354] "Means for obtaining user sentiment and destination input" refers to a function that receives sentiment information and desired destination information entered by the user through a terminal.

[0355] An "emotion recognition engine" is a software architecture that analyzes a user's emotional state from voice and text data and identifies it in real time.

[0356] An "external information provider" is an external service that can acquire environmental data such as traffic conditions and weather information in real time via the internet.

[0357] A "generative artificial intelligence model" is an artificial intelligence algorithm that uses accumulated data to calculate the optimal route.

[0358] "Means of converting to visual information format" refers to the process of converting the generated route information into an image format that can be easily understood by the user.

[0359] An "information storage device" is a memory structure that holds user input information and feedback, and uses it for subsequent data processing and algorithm improvement.

[0360] "Means for selecting the optimal route" refers to a method for determining the best travel path for a user based on their emotional state and real-time data.

[0361] To implement this invention, the user uses a terminal to input information about desired destinations and activities in voice or text format, and to indicate their current emotional state. The terminal receives this input and extracts emotional data using an emotion recognition engine. This emotion recognition engine may also use PyTorch or other similar processing software and a high-performance computing device to analyze the voice data.

[0362] The device sends emotion data and destination data to the server. The server obtains real-time traffic and weather information from external services. This may include using the Google Maps API or other information-providing APIs. Based on the data thus collected, the server generates the optimal route using a generative AI model. This generative AI model is specifically tailored to calculate routes that are appropriate for the user's emotions and is implemented using deep learning libraries such as TensorFlow.

[0363] The generated route is converted into a visual information format using OpenCV or a similar image processing library. This image is designed to allow the user to intuitively understand the route. The converted visual information is sent to the user's terminal via the internet and displayed on the terminal. This allows the user to visually confirm a concrete travel plan.

[0364] After the journey, users can provide feedback on how well the experience met their emotional needs. This feedback is entered from the terminal and sent to the server. The server analyzes this feedback, performs a re-analysis using the emotion engine, and stores it as training data for the entire system, thereby continuously improving the generative AI model.

[0365] For example, if a user is feeling "I want to relax," a possible prompt might be, "Please suggest a travel plan that is best suited for when the user is feeling relaxed." Based on this prompt, the server can utilize a generative AI model to suggest a route that passes through a quiet natural environment.

[0366] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0367] Step 1:

[0368] Users input their desired destinations and activities via text or voice using their device, and also indicate their current emotional state. Examples of data users might input include "I want to relax" or "I want to be active." This input data is processed by the device's emotion recognition engine.

[0369] Step 2:

[0370] The device receives voice and text data input from the user. It then uses an emotion recognition engine to extract emotional data. Specifically, voice data is converted to text via a speech processing algorithm, and then emotional components are analyzed using natural language processing techniques. The output generates data indicating the user's emotional state.

[0371] Step 3:

[0372] The device sends the extracted sentiment data and destination data to the server. A secure and reliable data communication protocol is used for this transmission. The output data includes the user's sentiment data and destination information.

[0373] Step 4:

[0374] The server obtains real-time traffic and weather information from external information providers based on the received sentiment and destination data. Specifically, it collects data from external services using APIs and stores it in an internal database. Traffic conditions and weather information are provided as output.

[0375] Step 5:

[0376] The server utilizes a generative AI model to integrate user sentiment data, destination information, and real-time external data. Once the input data is provided to the AI ​​model, it executes a predictive algorithm to calculate the optimal route. During this process, it receives a prompt instructing it to "suggest the optimal route based on the sentiment indicated by the user." The optimal route is then generated as output.

[0377] Step 6:

[0378] The server converts the generated optimal route into a visual information format. This conversion uses an image processing library to generate a map image. The output is a route display image that the user can intuitively understand.

[0379] Step 7:

[0380] The server sends the converted route image to the terminal. The communication method used here is a network protocol that enables real-time data transfer. Once the output data reaches the terminal, the user displays it to confirm the next traverse plan.

[0381] Step 8:

[0382] Users actually travel based on the proposed travel plan. After the trip, they input feedback via their device about how well the experienced route met their emotional needs. The input data is provided as subjective emotional evaluations and opinions on the proposed plan.

[0383] Step 9:

[0384] The server receives user feedback and performs a re-analysis using the sentiment engine. The received feedback is stored as system training data, contributing to the continuous improvement of the AI ​​model. To improve the quality of future route suggestions, the generative AI model is updated based on a new training cycle.

[0385] (Application Example 2)

[0386] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0387] Traditional travel experiences often provide a uniform route without considering the user's emotions, making it difficult to offer an optimal travel plan that meets the individual emotional needs of each user. Therefore, there is a growing demand for personalized routes that cater to users' emotions, such as whether they want to relax or be active.

[0388] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0389] In this invention, the server includes means for acquiring input data from the user and identifying their emotional state, means for acquiring real-time data through an external information source, and means for integrating the acquired data and generating multiple travel routes based on the user's emotional state. This makes it possible to provide a personalized travel plan that matches the user's emotions.

[0390] "Input data" refers to information collected from users and is used to identify their emotional state.

[0391] "Emotional state" refers to the user's psychological state, which can change in response to suggested travel routes.

[0392] "External information sources" refer to external data providers that offer real-time data such as traffic information and weather information.

[0393] A "visual format" is a graphical format displayed on a digital screen, provided in a form that is easily recognizable to the user.

[0394] An "information storage device" refers to a device or system that stores acquired data and keeps it in a state where it can be used for future analysis and reference.

[0395] An "emotional state profile" refers to a set of individual datasets that accumulate users' past emotional input data and are used in suggesting travel routes.

[0396] A "generative algorithm" refers to a procedure or method for calculating the optimal travel route based on input data.

[0397] The system that realizes this application primarily uses an emotion recognition engine, an external data acquisition module, an AI route generation engine, and a feedback learning module. The system is installed in an autonomous vehicle equipped with an onboard computer.

[0398] The server uses an emotion recognition engine to identify the user's emotional state based on the data they input. Specifically, it analyzes voice and text input using an emotion recognition API (e.g., Google's Speech Emotion Recognition API) to identify the emotions the user is currently experiencing.

[0399] Next, the server obtains real-time traffic and weather information from external sources via an external data acquisition module. This module utilizes information sources such as the Google Maps API and the Weather Data API.

[0400] The AI ​​route generation engine integrates acquired emotional state data with external information to generate multiple travel routes that best match the user's emotions. A custom model developed using TensorFlow is used for this process. The generated routes are converted into a visual format and displayed on the in-car display.

[0401] Users travel using the provided routes and provide feedback from their devices on how well the travel experience met their emotional needs. The server receives this feedback and stores it as data to improve the accuracy of the generation algorithm using a feedback learning module.

[0402] For example, if a user planning a family drive one afternoon inputs the emotion "I want to enjoy nature," the system can suggest a relaxing route. In this way, the system provides a travel experience tailored to the individual user's emotional needs.

[0403] An example of a prompt message is as follows:

[0404] Receive emotional input from the user such as "I want to relax," and use an AI model to generate the optimal route while considering available traffic and weather information, then display it on the in-car display.

[0405] This makes it possible to smoothly provide personalized travel plans that match the user's emotions.

[0406] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0407] Step 1:

[0408] The user inputs their emotions via voice or text using an in-vehicle terminal. This input is received by the emotion recognition engine. The input data is provided to the system as voice or text, and the emotional state is identified through the emotion recognition API. As a result, the user's emotions are identified, and analyzed emotional state data is generated.

[0409] Step 2:

[0410] The server obtains real-time traffic and weather information from external sources. This information is obtained using the Google Maps API and other weather data APIs. The server sends requests to the APIs and receives information about traffic conditions and weather in the response. It sends traffic and weather requests as input and receives the latest traffic and weather data as output.

[0411] Step 3:

[0412] The server integrates the data obtained in Step 1 and Step 2 and inputs it into the AI ​​route generation engine. The AI ​​route generation engine is a custom model using TensorFlow and generates multiple travel routes. Emotional state data and real-time data are passed to the AI ​​engine as input, and multiple route suggestions that best match the user's emotions are generated as output.

[0413] Step 4:

[0414] The server converts the generated route into a visual format and outputs it to the in-vehicle display. This process transforms the proposed route into a user-friendly graphical representation. It accepts route information in text or data format as input and provides the terminal with the corresponding image or map output.

[0415] Step 5:

[0416] Users experience a suggested travel route and then send feedback from their device to the server. The feedback evaluates satisfaction with the travel experience and how well it met their emotional needs, and the evaluation data is sent to the server as input.

[0417] Step 6:

[0418] The server analyzes the received feedback using a feedback learning module to improve the generation algorithm. The analysis extracts information that contributes to improving the accuracy of the AI ​​model, and this information is fed back into the AI ​​model. It receives user feedback data as input and produces an improved generation model as output.

[0419] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0420] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0421] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0422] [Third Embodiment]

[0423] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0424] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0425] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0426] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0427] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0428] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0429] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0430] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0431] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0432] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0433] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0434] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0435] This invention is a system that generates individually customized travel plans by having users input data specifying their destination, preferences, and mode of transportation, and combining that data with real-time data obtained from multiple external services. This system runs as a program on both the server and the terminal.

[0436] Users input their desired destinations and activities into the app using a smartphone or tablet. The entered data is immediately sent from the device to the server, which then stores this data in a database.

[0437] Next, the server communicates with external traffic information and weather forecast services via API to obtain real-time data. This data includes current traffic congestion, delay information for transportation services, the latest weather forecasts, and congestion levels at destination facilities.

[0438] The server processes the acquired data and analyzes it in combination with the user's input data. This analysis includes a process that uses an AI model to suggest the optimal travel route and travel plan. In particular, it generates multiple route options that take into account the user's preferences and conditions, and combines them to derive the most efficient and satisfying plan.

[0439] The server then converts these route suggestions into an image format that is easy to understand visually and sends it to the terminal. On the terminal, the route suggestions are visualized as a map or action plan, and the user can review the plan.

[0440] As a concrete example, let's consider a scenario where a user wants to visit a popular tourist destination on a holiday. The user enters the name of the tourist destination and the desired activities into the app. The server then considers traffic, weather, and facility congestion to suggest and provide the user with the optimal route combining various modes of transportation such as car, train, and walking.

[0441] This system allows users to easily obtain flexible travel plans tailored to their needs and conditions, resulting in a better travel experience.

[0442] The following describes the processing flow.

[0443] Step 1:

[0444] The user opens the app on their device and enters information such as their desired destination, activities, budget, and mode of transportation. The device then saves this input data locally as temporary data.

[0445] Step 2:

[0446] The terminal sends the entered data to the server. This data includes the user's pre-profile information and newly entered data each time. The server receives the data and stores it in the database.

[0447] Step 3:

[0448] The server obtains real-time data such as traffic information, weather forecasts, and destination congestion via external APIs. This aggregates the latest external information, which can then be used by users for planning.

[0449] Step 4:

[0450] The server integrates user input data with real-time data and generates route suggestions using an AI model. The calculated route suggestions are divided into multiple options based on evaluation criteria such as travel time, cost, and comfort.

[0451] Step 5:

[0452] The server selects the most optimal route and converts it into a visual image format. The visualized route plan is then prepared to be sent to the terminal as a map or schedule.

[0453] Step 6:

[0454] The device receives route suggestions sent from the server and presents them to the user. The user reviews these suggestions and makes adjustments as needed. At this point, the user can also receive alerts regarding congestion or delays.

[0455] Step 7:

[0456] After a trip or travel experience, users send feedback via their device. This feedback is then sent to a server, which uses it to improve the AI ​​algorithm. This results in more accurate plan generation for future trips.

[0457] (Example 1)

[0458] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0459] In modern travel, providing efficient routes tailored to individual needs quickly presents technical and infrastructure challenges. Furthermore, generating optimal travel plans based on user preferences and requests while considering fluctuating factors such as traffic conditions and weather is difficult.

[0460] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0461] In this invention, the server includes means for obtaining requests from users, means for obtaining time-series data through external information sources, means for combining the obtained data to generate multiple travel routes, means for converting the generated routes into visualization information, means for presenting the optimal route to the user's device, and means for optimizing the route according to the user's preferences using a generation AI model. This makes it possible to quickly provide users with the optimal itinerary, taking into account traffic conditions and weather conditions.

[0462] A "user" is an entity that inputs information and uses the system to obtain travel routes and travel plans.

[0463] A "request" is information that a user enters into an application to identify their destination, activity, and mode of transportation.

[0464] "External information sources" refer to third-party information systems that provide real-time data, such as traffic information services and weather forecast services.

[0465] "Time-series data" refers to data that changes over time, such as traffic conditions, weather information, and facility congestion levels.

[0466] "Combining acquired data" is the process of combining user request information with real-time data collected from external sources and performing a comprehensive analysis.

[0467] "Generating travel routes" refers to the process of formulating multiple possible travel plans based on user requests and real-time data.

[0468] "Visualized information" refers to information that converts generated travel routes and plans into formats that are easy for users to intuitively understand, such as maps and timelines.

[0469] "To present" means to display information on a device so that the user can review its contents and make a decision.

[0470] A "generative AI model" is an artificial intelligence technology used to learn user preferences and conditions and propose optimal travel routes and travel plans.

[0471] This invention relates to a system in which a user specifies their travel destination, preferences, and mode of transportation using a terminal, and a server generates an optimized travel plan based on this information. The system consists of a terminal and a server, and the specific hardware includes smartphones, tablets, and server computers. The software primarily consists of an application responsible for generating the travel plan and a server program.

[0472] First, the user uses a smartphone or tablet application to enter details such as their desired destination, activities, and mode of transportation. The user's request is then displayed as a prompt within the application. For example, the user might enter, "I'm planning a café hopping trip in Kyoto next weekend and would like to enjoy some traditional Japanese sweets there."

[0473] The terminal sends the entered data to the server. This server is responsible for obtaining time-series data from external sources via APIs. Real-time data, including traffic conditions, weather conditions, and facility congestion information, is sent to the server from external sources.

[0474] On the server, a generative AI model is used to analyze this acquired data and user input information to generate a travel plan. The generative AI model learns from the user's past data and preferences, enabling it to propose the most efficient and satisfying travel route.

[0475] For example, if a user expresses a desire to "enjoy surfing at Shonan Beach," the server can consider the congestion levels of surfing spots and transportation options to those locations, and then suggest recommended times and routes.

[0476] The plan generated by the server is converted into a visually easy-to-understand format and sent to the terminal. The plan received on the terminal is displayed as a map and a sequential action plan, allowing the user to immediately review the proposed plan and prepare to take action.

[0477] Thus, the present invention enables users to easily and accurately obtain travel plans and realize travel and transportation experiences optimized for them.

[0478] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0479] Step 1:

[0480] The user launches an application on their smartphone or tablet and enters their desired destination, activities, and mode of transportation. The application processes the entered information as text data and generates prompt statements. These prompt statements serve as foundational data for subsequent processing.

[0481] Step 2:

[0482] The terminal sends the generated prompt message to the server via internet communication. This transmission operation transfers the input data from the terminal to the server, preparing it for the next processing step. As a result, the user's request information is stored on the server.

[0483] Step 3:

[0484] The server, based on the user's request data received, collaborates with external information sources to collect time-series data. This process utilizes APIs from traffic and weather information services to obtain the latest traffic conditions, weather conditions, and facility congestion information. The input data is a prompt statement, and the output data is time-series data.

[0485] Step 4:

[0486] The server utilizes a generative AI model to integrate user input data with acquired time-series data to generate travel routes. In this process, the AI ​​model analyzes user preferences and conditions and generates multiple optimized travel plans. Specifically, it proposes several of the most efficient routes from the input data and prepares them for comparison and evaluation.

[0487] Step 5:

[0488] The server converts the generated travel plan into a format that is easy for the user to understand visually. Here, data processing is performed to visually represent the proposed route as a map or action plan. The input is the generated plan data, and the output is the visualized information.

[0489] Step 6:

[0490] The server sends the visualized information to the user's terminal. The user reviews the information received on the terminal and prepares to create a concrete action plan. The terminal displays prompts on the screen suggesting a specific travel plan, providing the foundation for proceeding to the next step.

[0491] (Application Example 1)

[0492] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0493] Traditional delivery systems struggled to provide optimal delivery routes in response to dynamic traffic conditions and weather changes. Furthermore, unexpected situations requiring route changes necessitated manual adjustments, leading to inefficiencies. This resulted in delivery personnel being unable to deliver smoothly, potentially lowering customer satisfaction.

[0494] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0495] In this invention, the server includes means for collecting input information from the user, means for acquiring dynamic data through an external information source, and means for integrating the acquired information and generating multiple routes. This enables the immediate output of the optimal route based on the dynamic data to the delivery person's terminal and the presentation of alternative routes in response to unexpected situations.

[0496] "User input information" refers to the details of the order received by the delivery person, and the information necessary for delivery.

[0497] "Dynamic data obtained through external sources" refers to communication methods for acquiring real-time, fluctuating external situational data, such as traffic conditions and weather information.

[0498] "Means for generating multiple routes" refers to algorithms that propose multiple efficient delivery routes for delivery personnel based on acquired dynamic data.

[0499] "Means of converting to a visual format" refers to methods of converting generated route information into maps or graphs that are easy for delivery personnel to understand.

[0500] A "delivery driver terminal" is an information processing device used by delivery drivers for their work, and includes smartphones and head-mounted displays.

[0501] "Means of suggesting alternative routes based on dynamic data" refers to a system that responds to real-time changes in traffic and weather information and proposes the most efficient alternative route.

[0502] "Response" refers to the reaction or feedback received from the user or delivery person.

[0503] "Means of improving the engine" refers to methods for improving the route generation algorithm based on the received response.

[0504] An "information storage system" is a data storage system used to store user characteristics and past data.

[0505] "Characteristic information" refers to data based on user preferences and behavioral history, and is used to customize services.

[0506] "Means for setting judgment criteria" refers to a method of determining criteria for selecting the optimal route based on user characteristic information, etc.

[0507] "Means for route re-evaluation" refers to a system that responds to unexpected traffic conditions or weather changes and immediately revises route selection.

[0508] To realize this invention, the delivery system uses a program consisting of a server and terminals. The server has the function of collecting input information from the user and, based on that, acquires real-time dynamic data from external information sources. For example, it uses the Google Maps API to obtain traffic information and the OpenWeatherMap API to obtain weather information. The server aggregates and analyzes this data and uses an AI model to generate and evaluate multiple routes.

[0509] This generated route information is converted into a visual format and transmitted to the delivery person's terminal. The terminal is implemented in various forms, such as a smartphone or head-mounted display, providing the delivery person with visualized information. This information includes the optimal delivery route based on real-time dynamic data, as well as alternative routes for unexpected situations.

[0510] For example, when a delivery person is making multiple deliveries in a busy area, alternative routes to avoid congestion are suggested. In case of a sudden change in weather, alternative routes to avoid rain are also considered.

[0511] An example of a prompt for the generating AI model is, "Provide the shortest and most efficient delivery route based on current traffic, weather, and destination information obtained from the API." Based on this prompt, the AI ​​model demonstrates its ability to generate the optimal delivery strategy in real time, tailored to the situation.

[0512] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0513] Step 1:

[0514] The user enters order information using the delivery app. Information such as order details, delivery address, and desired delivery time is entered and sent from the device to the server. Based on this input information, the server prepares the initial data for generating the delivery route.

[0515] Step 2:

[0516] The server obtains real-time dynamic data via external APIs. It uses the Google Maps API to collect traffic information and the OpenWeatherMap API to obtain weather information. This data is stored in a database as variables necessary for generating delivery routes.

[0517] Step 3:

[0518] The server integrates order information from users with acquired movement data and generates multiple delivery routes using a generative AI model. Given the prompt "Generate the optimal route based on current traffic and weather data," the AI ​​model proposes an efficient delivery plan. This results in the server outputting the route with the highest delivery efficiency.

[0519] Step 4:

[0520] The delivery routes generated by the server are visualized and sent to the terminal. The terminal overlays this information on a map, displaying it in a format that is easy for delivery personnel to understand. As a result, the optimal delivery route is clearly visualized.

[0521] Step 5:

[0522] During delivery, the user's device reacquires real-time movement data and sends updated information to the server. Based on this information, the server uses an AI model to re-evaluate the route as needed and generate alternative routes. At this time, the AI ​​model is given the prompt message "Re-evaluate the route based on the newly changed movement information," and the optimal route is immediately readjusted.

[0523] Step 6:

[0524] The terminal overlays the regenerated route information onto the map again, providing delivery personnel with the latest travel instructions. This ensures that the optimal route is always guaranteed in real time.

[0525] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0526] This invention is a system aimed at providing travel and transportation plans that take user emotions into consideration, and incorporates an emotion recognition engine. In addition to user input data and real-time data acquired from external sources, this system identifies the user's emotional state in real time and generates a route suggestion that is adjusted based on that.

[0527] When users input their desired destinations or activities using their device, they can also indicate their current emotional state through voice or text. The device uses an emotion recognition engine to extract emotional data from these inputs and send it to the server.

[0528] The server collects real-time traffic information, weather data, and other data from external services and integrates it with sentiment data sent by users. This allows it to utilize an AI model to generate the route that best matches the user's emotional state when sentiment is explicitly expressed.

[0529] For example, if a user expresses a desire to relax, the server can suggest quiet routes or travel plans surrounded by nature. Conversely, if a user expresses a desire to be active, the server will guide them to routes and destinations with plenty of activities.

[0530] The generated route is converted into a visually easy-to-understand image format and sent to the device. The user uses this information to navigate, and after the experience, can provide feedback via the device on how well the navigation met their emotional needs.

[0531] The server receives user feedback, reruns the emotion engine analysis, and stores it as training data for the entire system. This allows the system to continuously improve so that future suggestions are better aligned with the user's emotions.

[0532] Thus, the present invention is a system that makes it possible to provide an unprecedentedly customized travel experience by directly taking into account the user's emotions.

[0533] The following describes the processing flow.

[0534] Step 1:

[0535] Users use their devices to input desired destinations and activities into the app. To reflect emotions, the app also provides features for easily adding emotions via voice or text input. The device temporarily stores the entered data, and the emotional data is analyzed by an emotion recognition engine.

[0536] Step 2:

[0537] The terminal sends the analyzed emotion data and basic input information to the server. The server receives this information, records it in a database, and prepares route calculations that reflect the emotions in real time.

[0538] Step 3:

[0539] The server uses external APIs to obtain real-time data such as the latest traffic information, weather forecasts, and congestion levels at tourist attractions. This data is integrated with sentiment data and used as analytical base data to provide users with the best possible options.

[0540] Step 4:

[0541] The server uses an AI model to generate multiple route suggestions that take into account the user's emotional state. Specifically, if the user is seeking relaxation, a quiet and less crowded route will be considered; if they are seeking excitement, a route offering an active experience will be selected.

[0542] Step 5:

[0543] The server generates an image of the optimal route it has selected for visual display and sends this information to the terminal. The terminal receives this and presents it to the user as a map or plan. The user confirms the selected route.

[0544] Step 6:

[0545] If a user's emotions change during a trip or journey, the device can provide a function to re-enter those emotions and send them to the server. The server receives this change, recalculates the route if necessary, and sends the updated route to the device.

[0546] Step 7:

[0547] After a user enjoys their trip, they send feedback about their overall experience and emotional satisfaction to a server via their device. The server analyzes this information using an emotion engine and adjusts the AI ​​model so that it is reflected in future route generation. This allows the system to continuously improve and enable more accurate and personalized suggestions.

[0548] (Example 2)

[0549] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0550] Conventional travel and transportation planning systems propose uniform routes without considering user emotions, making it difficult to customize them to meet diverse emotional needs. Furthermore, there is little opportunity for improvement based on feedback, resulting in insufficient quality suggestions to enhance user satisfaction.

[0551] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0552] In this invention, the server includes means for acquiring user emotions and destination input, means for identifying the user's emotional state using an emotion recognition engine, and means for acquiring real-time environmental data from an external information provider. This makes it possible to generate an optimal travel and transportation plan that corresponds to the user's emotions and to improve it with each suggestion.

[0553] "Means for obtaining user sentiment and destination input" refers to a function that receives sentiment information and desired destination information entered by the user through a terminal.

[0554] An "emotion recognition engine" is a software architecture that analyzes a user's emotional state from voice and text data and identifies it in real time.

[0555] An "external information provider" is an external service that can acquire environmental data such as traffic conditions and weather information in real time via the internet.

[0556] A "generative artificial intelligence model" is an artificial intelligence algorithm that uses accumulated data to calculate the optimal route.

[0557] "Means of converting to visual information format" refers to the process of converting the generated route information into an image format that can be easily understood by the user.

[0558] An "information storage device" is a memory structure that holds user input information and feedback, and uses it for subsequent data processing and algorithm improvement.

[0559] "Means for selecting the optimal route" refers to a method for determining the best travel path for a user based on their emotional state and real-time data.

[0560] To implement this invention, the user uses a terminal to input information about desired destinations and activities in voice or text format, and to indicate their current emotional state. The terminal receives this input and extracts emotional data using an emotion recognition engine. This emotion recognition engine may also use PyTorch or other similar processing software and a high-performance computing device to analyze the voice data.

[0561] The device sends emotion data and destination data to the server. The server obtains real-time traffic and weather information from external services. This may include using the Google Maps API or other information-providing APIs. Based on the data thus collected, the server generates the optimal route using a generative AI model. This generative AI model is specifically tailored to calculate routes that are appropriate for the user's emotions and is implemented using deep learning libraries such as TensorFlow.

[0562] The generated route is converted into a visual information format using OpenCV or a similar image processing library. This image is designed to allow the user to intuitively understand the route. The converted visual information is sent to the user's terminal via the internet and displayed on the terminal. This allows the user to visually confirm a concrete travel plan.

[0563] After the journey, users can provide feedback on how well the experience met their emotional needs. This feedback is entered from the terminal and sent to the server. The server analyzes this feedback, performs a re-analysis using the emotion engine, and stores it as training data for the entire system, thereby continuously improving the generative AI model.

[0564] For example, if a user is feeling "I want to relax," a possible prompt might be, "Please suggest a travel plan that is best suited for when the user is feeling relaxed." Based on this prompt, the server can utilize a generative AI model to suggest a route that passes through a quiet natural environment.

[0565] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0566] Step 1:

[0567] Users input their desired destinations and activities via text or voice using their device, and also indicate their current emotional state. Examples of data users might input include "I want to relax" or "I want to be active." This input data is processed by the device's emotion recognition engine.

[0568] Step 2:

[0569] The device receives voice and text data input from the user. It then uses an emotion recognition engine to extract emotional data. Specifically, voice data is converted to text via a speech processing algorithm, and then emotional components are analyzed using natural language processing techniques. The output generates data indicating the user's emotional state.

[0570] Step 3:

[0571] The device sends the extracted sentiment data and destination data to the server. A secure and reliable data communication protocol is used for this transmission. The output data includes the user's sentiment data and destination information.

[0572] Step 4:

[0573] The server obtains real-time traffic and weather information from external information providers based on the received sentiment and destination data. Specifically, it collects data from external services using APIs and stores it in an internal database. Traffic conditions and weather information are provided as output.

[0574] Step 5:

[0575] The server utilizes a generative AI model to integrate user sentiment data, destination information, and real-time external data. Once the input data is provided to the AI ​​model, it executes a predictive algorithm to calculate the optimal route. During this process, it receives a prompt instructing it to "suggest the optimal route based on the sentiment expressed by the user." The optimal route is then generated as output.

[0576] Step 6:

[0577] The server converts the generated optimal route into a visual information format. This conversion uses an image processing library to generate a map image. The output is a route display image that the user can intuitively understand.

[0578] Step 7:

[0579] The server sends the converted route image to the terminal. The communication method used here is a network protocol that enables real-time data transfer. Once the output data reaches the terminal, the user displays it to confirm the next traverse plan.

[0580] Step 8:

[0581] Users actually travel based on the proposed travel plan. After the trip, they input feedback via their device about how well the experienced route met their emotional needs. The input data is provided as subjective emotional evaluations and opinions on the proposed plan.

[0582] Step 9:

[0583] The server receives user feedback and performs a re-analysis using the sentiment engine. The received feedback is stored as system training data, contributing to the continuous improvement of the AI ​​model. To improve the quality of future route suggestions, the generative AI model is updated based on a new training cycle.

[0584] (Application Example 2)

[0585] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0586] Traditional travel experiences often provide a uniform route without considering the user's emotions, making it difficult to offer an optimal travel plan that meets the individual emotional needs of each user. Therefore, there is a growing demand for personalized routes that cater to users' emotions, such as whether they want to relax or be active.

[0587] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0588] In this invention, the server includes means for acquiring input data from the user and identifying their emotional state, means for acquiring real-time data through an external information source, and means for integrating the acquired data and generating multiple travel routes based on the user's emotional state. This makes it possible to provide a personalized travel plan that matches the user's emotions.

[0589] "Input data" refers to information collected from users and is used to identify their emotional state.

[0590] "Emotional state" refers to the user's psychological state, which can change in response to suggested travel routes.

[0591] "External information sources" refer to external data providers that offer real-time data such as traffic information and weather information.

[0592] A "visual format" is a graphical format displayed on a digital screen, provided in a form that is easily recognizable to the user.

[0593] An "information storage device" refers to a device or system that stores acquired data and keeps it in a state where it can be used for future analysis and reference.

[0594] An "emotional state profile" refers to a set of individual datasets that accumulate users' past emotional input data and are used in suggesting travel routes.

[0595] A "generative algorithm" refers to a procedure or method for calculating the optimal travel route based on input data.

[0596] The system that realizes this application primarily uses an emotion recognition engine, an external data acquisition module, an AI route generation engine, and a feedback learning module. The system is installed in an autonomous vehicle equipped with an onboard computer.

[0597] The server uses an emotion recognition engine to identify the user's emotional state based on the data they input. Specifically, it analyzes voice and text input using an emotion recognition API (e.g., Google's Speech Emotion Recognition API) to identify the emotions the user is currently experiencing.

[0598] Next, the server obtains real-time traffic and weather information from external sources via an external data acquisition module. This module utilizes information sources such as the Google Maps API and the Weather Data API.

[0599] The AI ​​route generation engine integrates acquired emotional state data with external information to generate multiple travel routes that best match the user's emotions. A custom model developed using TensorFlow is used for this process. The generated routes are converted into a visual format and displayed on the in-car display.

[0600] Users travel using the provided routes and provide feedback from their devices on how well the travel experience met their emotional needs. The server receives this feedback and stores it as data to improve the accuracy of the generation algorithm using a feedback learning module.

[0601] For example, if a user planning a family drive one afternoon inputs the emotion "I want to enjoy nature," the system can suggest a relaxing route. In this way, the system provides a travel experience tailored to the individual user's emotional needs.

[0602] An example of a prompt message is as follows:

[0603] Receive emotional input from the user such as "I want to relax," and use an AI model to generate the optimal route while considering available traffic and weather information, then display it on the in-car display.

[0604] This makes it possible to smoothly provide personalized travel plans that match the user's emotions.

[0605] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0606] Step 1:

[0607] The user inputs their emotions via voice or text using an in-vehicle terminal. This input is received by the emotion recognition engine. The input data is provided to the system as voice or text, and the emotional state is identified through the emotion recognition API. As a result, the user's emotions are identified, and analyzed emotional state data is generated.

[0608] Step 2:

[0609] The server obtains real-time traffic and weather information from external sources. This information is obtained using the Google Maps API and other weather data APIs. The server sends requests to the APIs and receives information about traffic conditions and weather in the response. It sends traffic and weather requests as input and receives the latest traffic and weather data as output.

[0610] Step 3:

[0611] The server integrates the data obtained in Step 1 and Step 2 and inputs it into the AI ​​route generation engine. The AI ​​route generation engine is a custom model using TensorFlow and generates multiple travel routes. Emotional state data and real-time data are passed to the AI ​​engine as input, and multiple route suggestions that best match the user's emotions are generated as output.

[0612] Step 4:

[0613] The server converts the generated route into a visual format and outputs it to the in-vehicle display. This process transforms the proposed route into a user-friendly graphical representation. It accepts route information in text or data format as input and provides the terminal with the corresponding image or map output.

[0614] Step 5:

[0615] Users experience a suggested travel route and then send feedback from their device to the server. The feedback evaluates satisfaction with the travel experience and how well it met their emotional needs, and the evaluation data is sent to the server as input.

[0616] Step 6:

[0617] The server analyzes the received feedback using a feedback learning module to improve the generation algorithm. The analysis extracts information that contributes to improving the accuracy of the AI ​​model, and this information is fed back into the AI ​​model. It receives user feedback data as input and produces an improved generation model as output.

[0618] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0619] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0620] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0621] [Fourth Embodiment]

[0622] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0623] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0624] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0625] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0626] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0627] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0628] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0629] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0630] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0631] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0632] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0633] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0634] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0635] This invention is a system that generates individually customized travel plans by having users input data specifying their destination, preferences, and mode of transportation, and combining that data with real-time data obtained from multiple external services. This system runs as a program on both the server and the terminal.

[0636] Users input their desired destinations and activities into the app using a smartphone or tablet. The entered data is immediately sent from the device to the server, which then stores this data in a database.

[0637] Next, the server communicates with external traffic information and weather forecast services via API to obtain real-time data. This data includes current traffic congestion, delay information for transportation services, the latest weather forecasts, and congestion levels at destination facilities.

[0638] The server processes the acquired data and analyzes it in combination with the user's input data. This analysis includes a process that uses an AI model to suggest the optimal travel route and travel plan. In particular, it generates multiple route options that take into account the user's preferences and conditions, and combines them to derive the most efficient and satisfying plan.

[0639] The server then converts these route suggestions into an image format that is easy to understand visually and sends it to the terminal. On the terminal, the route suggestions are visualized as a map or action plan, and the user can review the plan.

[0640] As a concrete example, let's consider a scenario where a user wants to visit a popular tourist destination on a holiday. The user enters the name of the tourist destination and the desired activities into the app. The server then considers traffic, weather, and facility congestion to suggest and provide the user with the optimal route combining various modes of transportation such as car, train, and walking.

[0641] This system allows users to easily obtain flexible travel plans tailored to their needs and conditions, resulting in a better travel experience.

[0642] The following describes the processing flow.

[0643] Step 1:

[0644] The user opens the app on their device and enters information such as their desired destination, activities, budget, and mode of transportation. The device then saves this input data locally as temporary data.

[0645] Step 2:

[0646] The terminal sends the entered data to the server. This data includes the user's pre-profile information and newly entered data each time. The server receives the data and stores it in the database.

[0647] Step 3:

[0648] The server obtains real-time data such as traffic information, weather forecasts, and destination congestion via external APIs. This aggregates the latest external information, which can then be used by users for planning.

[0649] Step 4:

[0650] The server integrates user input data with real-time data and generates route suggestions using an AI model. The calculated route suggestions are divided into multiple options based on evaluation criteria such as travel time, cost, and comfort.

[0651] Step 5:

[0652] The server selects the most optimal route and converts it into a visual image format. The visualized route plan is then prepared to be sent to the terminal as a map or schedule.

[0653] Step 6:

[0654] The device receives route suggestions sent from the server and presents them to the user. The user reviews these suggestions and makes adjustments as needed. At this point, the user can also receive alerts regarding congestion or delays.

[0655] Step 7:

[0656] After a trip or travel experience, users send feedback via their device. This feedback is then sent to a server, which uses it to improve the AI ​​algorithm. This results in more accurate plan generation for future trips.

[0657] (Example 1)

[0658] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0659] In modern travel, providing efficient routes tailored to individual needs quickly presents technical and infrastructure challenges. Furthermore, generating optimal travel plans based on user preferences and requests while considering fluctuating factors such as traffic conditions and weather is difficult.

[0660] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0661] In this invention, the server includes means for obtaining requests from users, means for obtaining time-series data through external information sources, means for combining the obtained data to generate multiple travel routes, means for converting the generated routes into visualization information, means for presenting the optimal route to the user's device, and means for optimizing the route according to the user's preferences using a generation AI model. This makes it possible to quickly provide users with the optimal itinerary, taking into account traffic conditions and weather conditions.

[0662] A "user" is an entity that inputs information and uses the system to obtain travel routes and travel plans.

[0663] A "request" is information that a user enters into an application to identify their destination, activity, and mode of transportation.

[0664] "External information sources" refer to third-party information systems that provide real-time data, such as traffic information services and weather forecast services.

[0665] "Time-series data" refers to data that changes over time, such as traffic conditions, weather information, and facility congestion levels.

[0666] "Combining acquired data" is the process of combining user request information with real-time data collected from external sources and performing a comprehensive analysis.

[0667] "Generating travel routes" refers to the process of formulating multiple possible travel plans based on user requests and real-time data.

[0668] "Visualized information" refers to information that converts generated travel routes and plans into formats that are easy for users to intuitively understand, such as maps and timelines.

[0669] "To present" means to display information on a device so that the user can review its contents and make a decision.

[0670] A "generative AI model" is an artificial intelligence technology used to learn user preferences and conditions and propose optimal travel routes and travel plans.

[0671] This invention relates to a system in which a user specifies their travel destination, preferences, and mode of transportation using a terminal, and a server generates an optimized travel plan based on this information. The system consists of a terminal and a server, and the specific hardware includes smartphones, tablets, and server computers. The software primarily consists of an application responsible for generating the travel plan and a server program.

[0672] First, the user uses a smartphone or tablet application to enter details such as their desired destination, activities, and mode of transportation. The user's request is then displayed as a prompt within the application. For example, the user might enter, "I'm planning a café hopping trip in Kyoto next weekend and would like to enjoy some traditional Japanese sweets there."

[0673] The terminal sends the entered data to the server. This server is responsible for obtaining time-series data from external sources via APIs. Real-time data, including traffic conditions, weather conditions, and facility congestion information, is sent to the server from external sources.

[0674] On the server, a generative AI model is used to analyze this acquired data and user input information to generate a travel plan. The generative AI model learns from the user's past data and preferences, enabling it to propose the most efficient and satisfying travel route.

[0675] For example, if a user expresses a desire to "enjoy surfing at Shonan Beach," the server can consider the congestion levels of surfing spots and transportation options to those locations, and then suggest recommended times and routes.

[0676] The plan generated by the server is converted into a visually easy-to-understand format and sent to the terminal. The plan received on the terminal is displayed as a map and a sequential action plan, allowing the user to immediately review the proposed plan and prepare to take action.

[0677] Thus, the present invention enables users to easily and accurately obtain travel plans and realize travel and transportation experiences optimized for them.

[0678] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0679] Step 1:

[0680] The user launches an application on their smartphone or tablet and enters their desired destination, activities, and mode of transportation. The application processes the entered information as text data and generates prompt statements. These prompt statements serve as foundational data for subsequent processing.

[0681] Step 2:

[0682] The terminal sends the generated prompt message to the server via internet communication. This transmission operation transfers the input data from the terminal to the server, preparing it for the next processing step. As a result, the user's request information is stored on the server.

[0683] Step 3:

[0684] The server, based on the user's request data received, collaborates with external information sources to collect time-series data. This process utilizes APIs from traffic and weather information services to obtain the latest traffic conditions, weather conditions, and facility congestion information. The input data is a prompt statement, and the output data is time-series data.

[0685] Step 4:

[0686] The server utilizes a generative AI model to integrate user input data with acquired time-series data to generate travel routes. In this process, the AI ​​model analyzes user preferences and conditions and generates multiple optimized travel plans. Specifically, it proposes several of the most efficient routes from the input data and prepares them for comparison and evaluation.

[0687] Step 5:

[0688] The server converts the generated travel plan into a format that is easy for the user to understand visually. Here, data processing is performed to visually represent the proposed route as a map or action plan. The input is the generated plan data, and the output is the visualized information.

[0689] Step 6:

[0690] The server sends the visualized information to the user's terminal. The user reviews the information received on the terminal and prepares to create a concrete action plan. The terminal displays prompts on the screen suggesting a specific travel plan, providing the foundation for proceeding to the next step.

[0691] (Application Example 1)

[0692] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0693] Traditional delivery systems struggled to provide optimal delivery routes in response to dynamic traffic conditions and weather changes. Furthermore, unexpected situations requiring route changes necessitated manual adjustments, leading to inefficiencies. This resulted in delivery personnel being unable to deliver smoothly, potentially lowering customer satisfaction.

[0694] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0695] In this invention, the server includes means for collecting input information from the user, means for acquiring dynamic data through an external information source, and means for integrating the acquired information and generating multiple routes. This enables the immediate output of the optimal route based on the dynamic data to the delivery person's terminal and the presentation of alternative routes in response to unexpected situations.

[0696] "User input information" refers to the details of the order received by the delivery person, and the information necessary for delivery.

[0697] "Dynamic data obtained through external sources" refers to communication methods for acquiring real-time, fluctuating external situational data, such as traffic conditions and weather information.

[0698] "Means for generating multiple routes" refers to algorithms that propose multiple efficient delivery routes for delivery personnel based on acquired dynamic data.

[0699] "Means of converting to a visual format" refers to methods of converting generated route information into maps or graphs that are easy for delivery personnel to understand.

[0700] A "delivery driver terminal" is an information processing device used by delivery drivers for their work, and includes smartphones and head-mounted displays.

[0701] "Means of suggesting alternative routes based on dynamic data" refers to a system that responds to real-time changes in traffic and weather information and proposes the most efficient alternative route.

[0702] "Response" refers to the reaction or feedback received from the user or delivery person.

[0703] "Means of improving the engine" refers to methods for improving the route generation algorithm based on the received response.

[0704] An "information storage system" is a data storage system used to store user characteristics and past data.

[0705] "Characteristic information" refers to data based on user preferences and behavioral history, and is used to customize services.

[0706] "Means for setting judgment criteria" refers to a method of determining criteria for selecting the optimal route based on user characteristic information, etc.

[0707] "Means for route re-evaluation" refers to a system that responds to unexpected traffic conditions or weather changes and immediately revises route selection.

[0708] To realize this invention, the delivery system uses a program consisting of a server and terminals. The server has the function of collecting input information from the user and, based on that, acquires real-time dynamic data from external information sources. For example, it uses the Google Maps API to obtain traffic information and the OpenWeatherMap API to obtain weather information. The server aggregates and analyzes this data and uses an AI model to generate and evaluate multiple routes.

[0709] This generated route information is converted into a visual format and transmitted to the delivery person's terminal. The terminal is implemented in various forms, such as a smartphone or head-mounted display, providing the delivery person with visualized information. This information includes the optimal delivery route based on real-time dynamic data, as well as alternative routes for unexpected situations.

[0710] For example, when a delivery person is making multiple deliveries in a busy area, alternative routes to avoid congestion are suggested. In case of a sudden change in weather, alternative routes to avoid rain are also considered.

[0711] An example of a prompt for the generating AI model is, "Provide the shortest and most efficient delivery route based on current traffic, weather, and destination information obtained from the API." Based on this prompt, the AI ​​model demonstrates its ability to generate the optimal delivery strategy in real time, tailored to the situation.

[0712] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0713] Step 1:

[0714] The user enters order information using the delivery app. Information such as order details, delivery address, and desired delivery time is entered and sent from the device to the server. Based on this input information, the server prepares the initial data for generating the delivery route.

[0715] Step 2:

[0716] The server obtains real-time dynamic data via external APIs. It uses the Google Maps API to collect traffic information and the OpenWeatherMap API to obtain weather information. This data is stored in a database as variables necessary for generating delivery routes.

[0717] Step 3:

[0718] The server integrates order information from users with acquired movement data and generates multiple delivery routes using a generative AI model. Given the prompt "Generate the optimal route based on current traffic and weather data," the AI ​​model proposes an efficient delivery plan. This results in the server outputting the route with the highest delivery efficiency.

[0719] Step 4:

[0720] The delivery routes generated by the server are visualized and sent to the terminal. The terminal overlays this information on a map, displaying it in a format that is easy for delivery personnel to understand. As a result, the optimal delivery route is clearly visualized.

[0721] Step 5:

[0722] During delivery, the user's device reacquires real-time movement data and sends updated information to the server. Based on this information, the server uses an AI model to re-evaluate the route as needed and generate alternative routes. At this time, the AI ​​model is given the prompt message "Re-evaluate the route based on the newly changed movement information," and the optimal route is immediately readjusted.

[0723] Step 6:

[0724] The terminal overlays the regenerated route information onto the map again, providing delivery personnel with the latest travel instructions. This ensures that the optimal route is always guaranteed in real time.

[0725] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0726] This invention is a system aimed at providing travel and transportation plans that take user emotions into consideration, and incorporates an emotion recognition engine. In addition to user input data and real-time data acquired from external sources, this system identifies the user's emotional state in real time and generates a route suggestion that is adjusted based on that.

[0727] When users input their desired destinations or activities using their device, they can also indicate their current emotional state through voice or text. The device uses an emotion recognition engine to extract emotional data from these inputs and send it to the server.

[0728] The server collects real-time traffic information, weather data, and other data from external services and integrates it with sentiment data sent by users. This allows it to utilize an AI model to generate the route that best matches the user's emotional state when sentiment is explicitly expressed.

[0729] For example, if a user expresses a desire to relax, the server can suggest quiet routes or travel plans surrounded by nature. Conversely, if a user expresses a desire to be active, the server will guide them to routes and destinations with plenty of activities.

[0730] The generated route is converted into a visually easy-to-understand image format and sent to the device. The user uses this information to navigate, and after the experience, can provide feedback via the device on how well the navigation met their emotional needs.

[0731] The server receives user feedback, reruns the emotion engine analysis, and stores it as training data for the entire system. This allows the system to continuously improve so that future suggestions are better aligned with the user's emotions.

[0732] Thus, the present invention is a system that makes it possible to provide an unprecedentedly customized travel experience by directly taking into account the user's emotions.

[0733] The following describes the processing flow.

[0734] Step 1:

[0735] Users use their devices to input desired destinations and activities into the app. To reflect emotions, the app also provides features for easily adding emotions via voice or text input. The device temporarily stores the entered data, and the emotional data is analyzed by an emotion recognition engine.

[0736] Step 2:

[0737] The terminal sends the analyzed emotion data and basic input information to the server. The server receives this information, records it in a database, and prepares route calculations that reflect the emotions in real time.

[0738] Step 3:

[0739] The server uses external APIs to obtain real-time data such as the latest traffic information, weather forecasts, and congestion levels at tourist attractions. This data is integrated with sentiment data and used as analytical base data to provide users with the best possible options.

[0740] Step 4:

[0741] The server uses an AI model to generate multiple route suggestions that take into account the user's emotional state. Specifically, if the user is seeking relaxation, a quiet and less crowded route will be considered; if they are seeking excitement, a route offering an active experience will be selected.

[0742] Step 5:

[0743] The server generates an image of the optimal route it has selected for visual display and sends this information to the terminal. The terminal receives this and presents it to the user as a map or plan. The user confirms the selected route.

[0744] Step 6:

[0745] If a user's emotions change during a trip or journey, the device can provide a function to re-enter those emotions and send them to the server. The server receives this change, recalculates the route if necessary, and sends the updated route to the device.

[0746] Step 7:

[0747] After a user enjoys their trip, they send feedback about their overall experience and emotional satisfaction to a server via their device. The server analyzes this information using an emotion engine and adjusts the AI ​​model so that it is reflected in future route generation. This allows the system to continuously improve and enable more accurate and personalized suggestions.

[0748] (Example 2)

[0749] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0750] Conventional travel and transportation planning systems propose uniform routes without considering user emotions, making it difficult to customize them to meet diverse emotional needs. Furthermore, there is little opportunity for improvement based on feedback, resulting in insufficient quality suggestions to enhance user satisfaction.

[0751] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0752] In this invention, the server includes means for acquiring user emotions and destination input, means for identifying the user's emotional state using an emotion recognition engine, and means for acquiring real-time environmental data from an external information provider. This makes it possible to generate an optimal travel and transportation plan that corresponds to the user's emotions and to improve it with each suggestion.

[0753] "Means for obtaining user sentiment and destination input" refers to a function that receives sentiment information and desired destination information entered by the user through a terminal.

[0754] An "emotion recognition engine" is a software architecture that analyzes a user's emotional state from voice and text data and identifies it in real time.

[0755] An "external information provider" is an external service that can acquire environmental data such as traffic conditions and weather information in real time via the internet.

[0756] A "generative artificial intelligence model" is an artificial intelligence algorithm that uses accumulated data to calculate the optimal route.

[0757] "Means of converting to visual information format" refers to the process of converting the generated route information into an image format that can be easily understood by the user.

[0758] An "information storage device" is a memory structure that holds user input information and feedback, and uses it for subsequent data processing and algorithm improvement.

[0759] "Means for selecting the optimal route" refers to a method for determining the best travel path for a user based on their emotional state and real-time data.

[0760] To implement this invention, the user uses a terminal to input information about desired destinations and activities in voice or text format, and to indicate their current emotional state. The terminal receives this input and extracts emotional data using an emotion recognition engine. This emotion recognition engine may also use PyTorch or other similar processing software and a high-performance computing device to analyze the voice data.

[0761] The device sends emotion data and destination data to the server. The server obtains real-time traffic and weather information from external services. This may include using the Google Maps API or other information-providing APIs. Based on the data thus collected, the server generates the optimal route using a generative AI model. This generative AI model is specifically tailored to calculate routes that are appropriate for the user's emotions and is implemented using deep learning libraries such as TensorFlow.

[0762] The generated route is converted into a visual information format using OpenCV or a similar image processing library. This image is designed to allow the user to intuitively understand the route. The converted visual information is sent to the user's terminal via the internet and displayed on the terminal. This allows the user to visually confirm a concrete travel plan.

[0763] After the journey, users can provide feedback on how well the experience met their emotional needs. This feedback is entered from the terminal and sent to the server. The server analyzes this feedback, performs a re-analysis using the emotion engine, and stores it as training data for the entire system, thereby continuously improving the generative AI model.

[0764] For example, if a user is feeling "I want to relax," a possible prompt might be, "Please suggest a travel plan that is best suited for when the user is feeling relaxed." Based on this prompt, the server can utilize a generative AI model to suggest a route that passes through a quiet natural environment.

[0765] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0766] Step 1:

[0767] Users input their desired destinations and activities via text or voice using their device, and also indicate their current emotional state. Examples of data users might input include "I want to relax" or "I want to be active." This input data is processed by the device's emotion recognition engine.

[0768] Step 2:

[0769] The device receives voice and text data input from the user. It then uses an emotion recognition engine to extract emotional data. Specifically, voice data is converted to text via a speech processing algorithm, and then emotional components are analyzed using natural language processing techniques. The output generates data indicating the user's emotional state.

[0770] Step 3:

[0771] The device sends the extracted sentiment data and destination data to the server. A secure and reliable data communication protocol is used for this transmission. The output data includes the user's sentiment data and destination information.

[0772] Step 4:

[0773] The server obtains real-time traffic and weather information from external information providers based on the received sentiment and destination data. Specifically, it collects data from external services using APIs and stores it in an internal database. Traffic conditions and weather information are provided as output.

[0774] Step 5:

[0775] The server utilizes a generative AI model to integrate user sentiment data, destination information, and real-time external data. Once the input data is provided to the AI ​​model, it executes a predictive algorithm to calculate the optimal route. During this process, it receives a prompt instructing it to "suggest the optimal route based on the sentiment expressed by the user." The optimal route is then generated as output.

[0776] Step 6:

[0777] The server converts the generated optimal route into a visual information format. This conversion uses an image processing library to generate a map image. The output is a route display image that the user can intuitively understand.

[0778] Step 7:

[0779] The server sends the converted route image to the terminal. The communication method used here is a network protocol that enables real-time data transfer. Once the output data reaches the terminal, the user displays it to confirm the next traverse plan.

[0780] Step 8:

[0781] Users actually travel based on the proposed travel plan. After the trip, they input feedback via their device about how well the experienced route met their emotional needs. The input data is provided as subjective emotional evaluations and opinions on the proposed plan.

[0782] Step 9:

[0783] The server receives user feedback and performs a re-analysis using the sentiment engine. The received feedback is stored as system training data, contributing to the continuous improvement of the AI ​​model. To improve the quality of future route suggestions, the generative AI model is updated based on a new training cycle.

[0784] (Application Example 2)

[0785] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0786] Traditional travel experiences often provide a uniform route without considering the user's emotions, making it difficult to offer an optimal travel plan that meets the individual emotional needs of each user. Therefore, there is a growing demand for personalized routes that cater to users' emotions, such as whether they want to relax or be active.

[0787] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0788] In this invention, the server includes means for acquiring input data from the user and identifying their emotional state, means for acquiring real-time data through an external information source, and means for integrating the acquired data and generating multiple travel routes based on the user's emotional state. This makes it possible to provide a personalized travel plan that matches the user's emotions.

[0789] "Input data" refers to information collected from users and is used to identify their emotional state.

[0790] "Emotional state" refers to the user's psychological state, which can change in response to suggested travel routes.

[0791] "External information sources" refer to external data providers that offer real-time data such as traffic information and weather information.

[0792] A "visual format" is a graphical format displayed on a digital screen, provided in a form that is easily recognizable to the user.

[0793] An "information storage device" refers to a device or system that stores acquired data and keeps it in a state where it can be used for future analysis and reference.

[0794] An "emotional state profile" refers to a set of individual datasets that accumulate users' past emotional input data and are used in suggesting travel routes.

[0795] A "generative algorithm" refers to a procedure or method for calculating the optimal travel route based on input data.

[0796] The system that realizes this application primarily uses an emotion recognition engine, an external data acquisition module, an AI route generation engine, and a feedback learning module. The system is installed in an autonomous vehicle equipped with an onboard computer.

[0797] The server uses an emotion recognition engine to identify the user's emotional state based on the data they input. Specifically, it analyzes voice and text input using an emotion recognition API (e.g., Google's Speech Emotion Recognition API) to identify the emotions the user is currently experiencing.

[0798] Next, the server obtains real-time traffic and weather information from external sources via an external data acquisition module. This module utilizes information sources such as the Google Maps API and the Weather Data API.

[0799] The AI ​​route generation engine integrates acquired emotional state data with external information to generate multiple travel routes that best match the user's emotions. A custom model developed using TensorFlow is used for this process. The generated routes are converted into a visual format and displayed on the in-car display.

[0800] Users travel using the provided routes and provide feedback from their devices on how well the travel experience met their emotional needs. The server receives this feedback and stores it as data to improve the accuracy of the generation algorithm using a feedback learning module.

[0801] For example, if a user planning a family drive one afternoon inputs the emotion "I want to enjoy nature," the system can suggest a relaxing route. In this way, the system provides a travel experience tailored to the individual user's emotional needs.

[0802] An example of a prompt message is as follows:

[0803] Receive emotional input from the user such as "I want to relax," and use an AI model to generate the optimal route while considering available traffic and weather information, then display it on the in-car display.

[0804] This makes it possible to smoothly provide personalized travel plans that match the user's emotions.

[0805] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0806] Step 1:

[0807] The user inputs their emotions via voice or text using an in-vehicle terminal. This input is received by the emotion recognition engine. The input data is provided to the system as voice or text, and the emotional state is identified through the emotion recognition API. As a result, the user's emotions are identified, and analyzed emotional state data is generated.

[0808] Step 2:

[0809] The server obtains real-time traffic and weather information from external sources. This information is obtained using the Google Maps API and other weather data APIs. The server sends requests to the APIs and receives information about traffic conditions and weather in the response. It sends traffic and weather requests as input and receives the latest traffic and weather data as output.

[0810] Step 3:

[0811] The server integrates the data obtained in Step 1 and Step 2 and inputs it into the AI ​​route generation engine. The AI ​​route generation engine is a custom model using TensorFlow and generates multiple travel routes. Emotional state data and real-time data are passed to the AI ​​engine as input, and multiple route suggestions that best match the user's emotions are generated as output.

[0812] Step 4:

[0813] The server converts the generated route into a visual format and outputs it to the in-vehicle display. This process transforms the proposed route into a user-friendly graphical representation. It accepts route information in text or data format as input and provides the terminal with the corresponding image or map output.

[0814] Step 5:

[0815] Users experience a suggested travel route and then send feedback from their device to the server. The feedback evaluates satisfaction with the travel experience and how well it met their emotional needs, and the evaluation data is sent to the server as input.

[0816] Step 6:

[0817] The server analyzes the received feedback using a feedback learning module to improve the generation algorithm. The analysis extracts information that contributes to improving the accuracy of the AI ​​model, and this information is fed back into the AI ​​model. It receives user feedback data as input and produces an improved generation model as output.

[0818] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0819] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0820] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0821] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0822] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0823] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0824] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0825] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0826] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0827] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0828] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0829] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0830] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0831] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0832] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0833] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0834] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0835] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0836] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0837] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0838] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0839] The following is further disclosed regarding the embodiments described above.

[0840] (Claim 1)

[0841] Means for obtaining input data from users,

[0842] Means of obtaining real-time data through external services,

[0843] A means of integrating acquired data and generating multiple routes,

[0844] A means of converting the generated route into an image format,

[0845] A means of outputting the optimal route for the user terminal,

[0846] A system that includes this.

[0847] (Claim 2)

[0848] A means of receiving user feedback,

[0849] A means of analyzing the received feedback and improving the generation algorithm,

[0850] The system according to claim 1, comprising:

[0851] (Claim 3)

[0852] A means of saving user input data to a database and using it as profile information,

[0853] A means of setting route evaluation criteria using a generation algorithm,

[0854] A means of selecting the most appropriate route based on evaluation criteria,

[0855] The system according to claim 1, comprising:

[0856] "Example 1"

[0857] (Claim 1)

[0858] Means for obtaining requests from users,

[0859] Means for obtaining time-series data through external information sources,

[0860] A means for combining acquired data and generating multiple travel paths,

[0861] A means of converting the generated path into visualization information,

[0862] A means of presenting the optimal route for the user's equipment,

[0863] A means for optimizing routes according to user preferences using a generative AI model,

[0864] A system that includes this.

[0865] (Claim 2)

[0866] Means for receiving feedback from users,

[0867] A means to analyze the received opinions and improve the generation method,

[0868] The system according to claim 1, comprising:

[0869] (Claim 3)

[0870] A means of recording user request data in a storage device and utilizing it as user information,

[0871] A means for setting evaluation metrics for paths using a generation method,

[0872] A means of selecting the most suitable route based on evaluation indicators,

[0873] The system according to claim 1, comprising:

[0874] "Application Example 1"

[0875] (Claim 1)

[0876] Means for collecting user input information,

[0877] Means for acquiring dynamic data through external information sources,

[0878] A means for integrating acquired information and generating multiple routes,

[0879] A means for converting the generated path into a visual format,

[0880] A means of outputting the optimal route to the delivery person's terminal,

[0881] A means of suggesting alternative routes based on dynamic data,

[0882] A system that includes this.

[0883] (Claim 2)

[0884] A means of receiving responses from users,

[0885] A means of analyzing the received response and improving the generation engine,

[0886] The system according to claim 1, comprising:

[0887] (Claim 3)

[0888] A means for storing user input information in an information storage and using it as characteristic information,

[0889] A means for setting path determination criteria using a generation engine,

[0890] A means of selecting the most appropriate route based on the criteria,

[0891] A means of re-evaluating routes in response to unexpected circumstances,

[0892] The system according to claim 1, comprising:

[0893] "Example 2 of combining an emotion engine"

[0894] (Claim 1)

[0895] Means for obtaining sentiment and destination input from the user,

[0896] A means for identifying a user's emotional state using an emotion recognition engine,

[0897] A means of acquiring real-time environmental data from an external information provider,

[0898] A means of integrating user emotion data and environmental data and generating routes using a generative artificial intelligence model,

[0899] A means for converting the generated route into a visual information format,

[0900] A means for transmitting route information converted to a user terminal,

[0901] A system that includes this.

[0902] (Claim 2)

[0903] A means of receiving emotional feedback from users after they have moved,

[0904] A means of analyzing the received feedback, saving it as training data for the system, and improving the generation algorithm,

[0905] The system according to claim 1, comprising:

[0906] (Claim 3)

[0907] A means for storing user input information in an information storage device and using it as user profile information,

[0908] A means of setting route evaluation criteria using a generation algorithm and selecting the optimal route that takes into account the user's emotional state,

[0909] The system according to claim 1, comprising:

[0910] "Application example 2 when combining with an emotional engine"

[0911] (Claim 1)

[0912] A means of acquiring input data from the user and identifying their emotional state,

[0913] Means of obtaining real-time data through external information sources,

[0914] A means for integrating acquired data and generating multiple travel routes based on the user's emotional state,

[0915] A means for converting the generated route into a visual format and outputting it to a display device,

[0916] A system that includes this.

[0917] (Claim 2)

[0918] The system according to claim 1, comprising means for receiving user feedback and improving the generative model to improve the accuracy of route suggestions based on emotional state.

[0919] (Claim 3)

[0920] A means for storing user input data in an information storage device and using it as an emotional state profile,

[0921] A means of setting route evaluation criteria using a generation algorithm and selecting the most appropriate travel route based on sentiment data,

[0922] The system according to claim 1, comprising: [Explanation of Symbols]

[0923] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for obtaining input data from users, Means of obtaining real-time data through external services, A means of integrating acquired data and generating multiple routes, A means of converting the generated route into an image format, A means of outputting the optimal route for the user terminal, A system that includes this.

2. A means of receiving user feedback, A means of analyzing the received feedback and improving the generation algorithm, The system according to claim 1, comprising:

3. A means of saving user input data to a database and using it as profile information, A means of setting route evaluation criteria using a generation algorithm, A means of selecting the most appropriate route based on evaluation criteria, The system according to claim 1, comprising:

Images