system

Abstract

We provide the system. [Solution] A means of collecting information about the travel destination from people living there, Methods for obtaining weather information from external weather forecast APIs, A means for generating clothing recommendations based on acquired weather information, A means of presenting weather information and clothing recommendations to travelers, A means of providing live camera footage to check the situation on site, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When planning a trip, information about the destination, especially information regarding the weather and the clothing to be worn at that time, is important. However, in conventional systems, there are few means to easily collect the latest information from local residents, making it difficult to grasp the real - time situation of the travel destination. Also, the means to check the local situation using the video of a live camera are limited. Therefore, there is a need for a system that can comprehensively and timely provide the information necessary for travel planning.

Means for Solving the Problems

[0005] To solve this problem, the present invention provides the following means: means for collecting travel destination information from local residents; means for obtaining weather information from an external weather forecast API; means for generating clothing recommendations based on the obtained weather information; means for presenting weather information and clothing recommendations to travelers; and means for providing live camera footage to check local conditions. The system includes this series of means.

[0006] In particular, by including means of storing information collected from local residents in a database, means of displaying travel destination information, weather information, and clothing recommendations on the user interface, means of streaming live camera footage upon user request, and means of regularly obtaining updated weather forecasts and pushing notifications to users, it becomes possible to provide comprehensive and real-time information about travel destinations. This allows users to plan their trips with peace of mind.

[0007] "Destination information" refers to data that travelers want to know about their destination, such as tourist attractions, restaurants, transportation, culture, customs, and events.

[0008] "Local residents" refers to people who actually live or work in the region you are traveling to, and who can provide you with the latest information and real-world conditions of that region.

[0009] An "external weather forecast API" is a web service that provides weather forecast information for a specific region, and it is an interface for obtaining necessary weather data through the API.

[0010] "Clothing recommendations" refer to information that suggests appropriate clothing choices based on specific regions and weather conditions.

[0011] A "traveler" is an individual or group planning a trip to a specific region and who needs information or services related to that region.

[0012] "Live camera footage" refers to a video stream captured by a camera that provides real-time visual information for a specific area.

[0013] A "system" is a set of mechanisms consisting of multiple components or tools that integrate information gathering, data processing, and information provision.

[0014] A "database" is a collection of information that organizes, stores, and efficiently manages and accesses data related to a specific purpose.

[0015] A "user interface" is a mechanism that includes the means and screen displays that allow a user to interact with a system, and it is also a means of providing information to the user and accepting input.

[0016] "Streaming" is a technology that transmits audio and video data in real time over the internet and plays it back continuously on the receiving end.

[0017] "Push notifications" are a mechanism that automatically sends information updates and notifications from a server to a device, providing users with important information in real time. [Brief explanation of the drawing]

[0018] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6]It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Modes for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the language used in the following description will be explained.

[0021] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0022] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] This invention is a system that comprehensively provides information such as travel destination information, weather forecasts, clothing recommendations, and live camera footage. This system operates through the cooperation of a server and a terminal (user's device).

[0040] Forms of information gathering

[0041] The user enters the region they are planning to travel to via their device. For example, if the user is planning a trip to Kyoto City, they would specify Kyoto City as their destination.

[0042] The terminal sends the travel destination information entered by the user to the server. This request is formatted as an HTTP request. The server receives this request and retrieves local information for the specified region from its database.

[0043] The server retrieves the latest information from its database, including local tourist attractions, restaurant information, and information provided by local residents, and sends it back to the terminal. For example, this includes information about tourist attractions and festivals in Kyoto City.

[0044] Generation methods for weather forecasts and clothing recommendations

[0045] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[0046] The terminal forwards the user's request to the server, sending an HTTP request that includes geographical information about the travel destination. The server receives this request and retrieves weather information for the specified region using an external weather forecast API.

[0047] The server generates clothing recommendations for the user based on the acquired weather information. For example, if the forecast for Kyoto City is cold, it will respond to the device with a recommendation such as, "You will need a warm jacket."

[0048] How to view live camera footage

[0049] Users can request live camera footage to check the local situation in real time.

[0050] The device sends a request for live camera footage to the server. The server retrieves the URL of the appropriate live camera from its database and sends it back to the device.

[0051] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface. This allows the user to check the local situation in real time.

[0052] Update and notification methods

[0053] The server periodically checks the weather forecast API and retrieves any updated weather information. If an important update is detected, it sends a push notification to the relevant users.

[0054] The device receives push notifications and displays them on the user interface. This allows users to receive real-time updates on weather conditions and other important information, which they can then incorporate into their travel plans.

[0055] As a concrete example, a user is planning a trip to Kyoto City and will use the following functions during that process.

[0056] The user enters "Kyoto City" and retrieves local information.

[0057] The server retrieves information from the database and sends a response to the terminal.

[0058] The user requests a weather forecast and clothing recommendations, and the server retrieves weather information from an external API and sends appropriate clothing advice to the user's device.

[0059] Users can view live camera footage and get real-time information about the location.

[0060] The server checks for weather forecast updates and sends push notifications to the device, providing users with the latest information.

[0061] This allows users to obtain all the necessary information in one place, making travel planning much smoother.

[0062] The following describes the processing flow.

[0063] Step 1:

[0064] The user starts up their device and enters the destination they want to go to (for example, "Kyoto City").

[0065] Step 2:

[0066] The terminal receives user input and formats that information as a request to the server.

[0067] Step 3:

[0068] The device sends a formatted request to the server. This request asks for local information about the travel destination.

[0069] Step 4:

[0070] The server receives the request and executes a query against the database to retrieve local information about "Kyoto City".

[0071] Step 5:

[0072] The server retrieves travel destination information related to "Kyoto City" from the database (such as tourist spots, restaurant information, and the latest information from local residents).

[0073] Step 6:

[0074] The server formats the acquired information into JSON format and sends it back to the terminal.

[0075] Step 7:

[0076] The terminal receives information returned from the server and displays it on the user interface. This allows the user to understand basic information about their travel destination.

[0077] Step 8:

[0078] If a user wants to know the weather in Kyoto City on a specific day, they send that request to the server via their device.

[0079] Step 9:

[0080] The device forwards the user's weather request to the server. This request includes local information and the date.

[0081] Step 10:

[0082] Based on the request received by the server, it queries an external weather forecast API for weather information.

[0083] Step 11:

[0084] The server receives weather information returned from an external weather forecast API and generates appropriate clothing recommendations based on the acquired weather information. For example, if the weather forecast is for cold weather, it will generate a recommendation such as "You need a warm jacket."

[0085] Step 12:

[0086] The server formats the weather information and generated clothing recommendations into JSON format and sends them back to the terminal.

[0087] Step 13:

[0088] The device displays weather information and clothing recommendations received by the user interface. The user can use this information to prepare for their trip.

[0089] Step 14:

[0090] If a user wants to view live camera footage from the location, they send a request to the server via their device.

[0091] Step 15:

[0092] The device forwards the live camera request to the server.

[0093] Step 16:

[0094] The server retrieves the URL of the live camera feed for "Kyoto City" from the database and sends it back to the terminal.

[0095] Step 17:

[0096] The device retrieves the video stream based on the live camera's URL and displays it on the user interface. This allows users to check the local situation in real time.

[0097] Step 18:

[0098] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[0099] Step 19:

[0100] When the server detects important weather updates, it sends that information as a push notification to the relevant users.

[0101] Step 20:

[0102] The device receives push notifications and displays them on the user interface. This allows users to receive the latest weather information in real time and respond quickly.

[0103] (Example 1)

[0104] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0105] Modern travelers have a growing need for detailed information and real-time updates on their destinations. However, obtaining the right information in one place often requires using multiple different applications and websites, which is inconvenient. Furthermore, there is no information system that allows travelers to check weather forecasts and local conditions in real time and respond quickly. As a result, it is difficult for travelers to plan efficient and comfortable trips.

[0106] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0107] In this invention, the server includes means for collecting information about the travel destination from local residents, means for obtaining weather information from an external weather forecast API, means for generating clothing recommendations based on the obtained weather information, means for receiving regional information about the travel destination entered by the traveler, means for presenting weather information and clothing recommendations to the traveler, means for providing live camera footage to check local conditions, and means for obtaining and providing traveler information on tourist spots and restaurants related to the travel destination from a database. This enables travelers to comprehensively obtain information about their travel destination within a single system and plan their trip efficiently and comfortably.

[0108] "Travel destination information" refers to a general term for tourist attractions, restaurants, events, and other region-specific information related to the area you plan to visit.

[0109] "Local residents" refers to people who permanently live in the travel destination region and can provide up-to-date and specific information about that area.

[0110] An "external weather forecast API" refers to a programmatic interface for obtaining weather forecast data provided by a third party.

[0111] "Clothing recommendations" refer to information that suggests appropriate clothing for travelers based on acquired weather data.

[0112] A "traveler" refers to an individual or group planning a trip to a specific region.

[0113] "Local information" refers collectively to geographical, social, and economic information concerning a specific region.

[0114] A "live camera" refers to a device or system that broadcasts real-time video from a specific location over the internet.

[0115] A "database" refers to a collection of data organized for the purpose of efficiently searching, retrieving, and managing information.

[0116] "Push notification" refers to a communication method in which a server sends information to a user in real time.

[0117] A "user interface" is an interface through which a user interacts with a system, and includes visual or functional elements.

[0118] This invention is a system that comprehensively provides travelers with information such as destination information, weather forecasts, clothing recommendations, and live camera footage. This system operates through the cooperation of a server and a terminal (user's device).

[0119] The user enters the region they are planning to travel to via their device. For example, if the user is planning a trip to Kyoto City, they would enter "Kyoto City" as the destination. This input information becomes the starting point for the system.

[0120] The terminal formats the travel destination information entered by the user as an HTTP request and sends it to the server. This is done over a standard internet connection. When the server receives the request, it accesses its internal database and external weather forecast APIs (e.g., OpenWeatherMap API) to collect the necessary information.

[0121] The server first accesses an internal database to retrieve information on tourist spots, restaurants, and events in a specified area (e.g., Kyoto City). A database called GLDB (General Location Database) can be used for this retrieval. Next, the server accesses an external weather forecast API to obtain the latest weather information. This API access can be done using, for example, an OpenWeatherMap API key.

[0122] Based on the acquired weather information, the server executes a process to generate clothing recommendations suitable for the user. For example, if the forecast for Kyoto City is cold, it will generate a specific recommendation such as "You will need a warm jacket."

[0123] The terminal receives information retrieved from the server and displays it in the user interface. This user interface is built using web technologies such as HTML / CSS and JavaScript (registered trademark). Here, users can check information on tourist attractions and restaurants in their travel destination, as well as weather forecasts and clothing recommendations.

[0124] Users can also request live camera footage to check real-time information from the location. The device sends this request to the server, which retrieves the URL of the corresponding live camera from its database and returns it to the device. The device uses the retrieved live camera URL to obtain the video stream and displays it on the user interface.

[0125] Furthermore, the server periodically checks external weather forecast APIs and retrieves any important weather forecast updates. When an important update is detected, the server also has a function to send push notifications to the relevant users. The device receives this push notification and displays it in the user interface, allowing users to obtain the latest information about their travel destination in real time.

[0126] As a concrete example, the following series of operations can be considered.

[0127] 1. The user enters "Kyoto City" and retrieves local information.

[0128] 2. The server retrieves information on tourist spots and restaurants in Kyoto City from the database and sends it back to the terminal.

[0129] 3. The user requests a weather forecast and clothing recommendations.

[0130] 4. The server retrieves weather information from an external API, generates appropriate clothing recommendations, and sends them to the device.

[0131] 5. Users can view live camera footage and understand the real-time situation on site.

[0132] 6. The server checks for weather forecast updates and sends important update information to the device via push notification.

[0133] Examples of prompt statements include:

[0134] "Travel destination information: Kyoto City, weather forecast, clothing recommendations, live cameras"

[0135] It can be done this way.

[0136] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0137] Step 1:

[0138] The user uses their device to enter information about their travel destination (e.g., "Kyoto City").

[0139] (Input) Travel destination information entered by the user.

[0140] (Processing) The terminal formats the user's input as an HTTP request.

[0141] (Output) A formatted HTTP request is generated.

[0142] Step 2:

[0143] The device sends an HTTP request to the server containing the travel destination information entered by the user.

[0144] (Input) A formatted HTTP request.

[0145] (Processing) The terminal sends a request to the server via the internet.

[0146] (Output) The server receives an HTTP request.

[0147] Step 3:

[0148] Based on the received request, the server retrieves tourist spots and restaurant information for the travel destination (e.g., "Kyoto City") from the database.

[0149] (Input) A request containing the user's travel destination information.

[0150] (Processing) The server queries the database to retrieve relevant information.

[0151] (Output) Acquired information on tourist spots and restaurants.

[0152] Step 4:

[0153] The server returns the acquired tourist spot and restaurant information to the terminal.

[0154] (Input) Information retrieved by the server from the database.

[0155] (Processing) The server formats the information and sends it to the terminal as an HTTP response.

[0156] (Output) Formatted HTTP response.

[0157] Step 5:

[0158] The terminal displays information received from the server on the user interface.

[0159] (Input) HTTP response received from the server.

[0160] (Processing) The terminal analyzes the information and displays it on the user interface.

[0161] (Output) Information displayed in a format that the user can view.

[0162] Step 6:

[0163] The user enters information into the device to request weather forecasts and clothing recommendations.

[0164] (Input) Geographical information of your travel destination.

[0165] (Processing) The user enters the necessary information into the terminal.

[0166] (Output) Input weather forecast and clothing request information.

[0167] Step 7:

[0168] The device sends an HTTP request to the server that includes weather forecasts and clothing recommendations.

[0169] (Input) Weather forecast and clothing request information entered by the user.

[0170] (Processing) The terminal formats the information as an HTTP request and sends it to the server.

[0171] (Output) HTTP requests received by the server.

[0172] Step 8:

[0173] The server accesses an external weather forecast API to obtain weather information for a specified area (e.g., "Kyoto City").

[0174] (Input) Regional information included in the HTTP request.

[0175] (Processing) The server calls an external weather forecast API to obtain weather information.

[0176] (Output) Acquired weather information.

[0177] Step 9:

[0178] The server generates clothing recommendations based on the acquired weather information.

[0179] (Input) Obtained weather information.

[0180] (Processing) The server analyzes weather information and executes an algorithm to recommend appropriate clothing.

[0181] (Output) Generated clothing recommendations.

[0182] Step 10:

[0183] The server sends the generated clothing recommendations to the device.

[0184] (Input) Clothing recommendation.

[0185] (Processing) The server formats the clothing recommendation into an HTTP response and sends it to the terminal.

[0186] (Output) Formatted HTTP response.

[0187] Step 11:

[0188] The device displays the received clothing recommendations in the user interface.

[0189] (Input) HTTP response received from the server.

[0190] (Processing) The terminal analyzes the information and displays it on the user interface.

[0191] (Output) Clothing recommendations displayed in a format viewable by the user.

[0192] Step 12:

[0193] Users request live camera footage to check real-time information from the location.

[0194] (Input) Request for live camera footage.

[0195] (Processing) The user enters a request on the terminal.

[0196] (Output) Input live camera request information.

[0197] Step 13:

[0198] The terminal sends a request for live camera footage to the server.

[0199] (Input) Live camera request information.

[0200] (Processing) The terminal formats the information as an HTTP request and sends it to the server.

[0201] (Output) HTTP requests received by the server.

[0202] Step 14:

[0203] The server retrieves the URL of the corresponding live camera from the database and sends it back to the terminal.

[0204] (Input) Regional information included in the request.

[0205] (Processing) The server queries the database to obtain the URL of the live camera.

[0206] (Output) The retrieved live camera URL.

[0207] Step 15:

[0208] The device uses the received URL of the live camera to retrieve the video stream and displays it in the user interface.

[0209] (Input) The URL of the live camera received from the server.

[0210] (Processing) The terminal acquires the video stream from the live camera and displays it on the user interface.

[0211] (Output) Live camera footage displayed in a format that can be viewed by the user.

[0212] Step 16:

[0213] The server periodically checks the weather forecast API to obtain the latest weather information.

[0214] (Input) Periodic calls to a weather forecast API.

[0215] (Processing) The server periodically calls the weather forecast API to obtain new weather information.

[0216] (Output) The latest weather information obtained.

[0217] Step 17:

[0218] When there are important updates, the server creates and sends push notifications to the relevant users.

[0219] (Input) The latest weather information obtained.

[0220] (Processing) The server generates a push notification based on important update information and sends it to the relevant users.

[0221] (Output) Sent push notification.

[0222] Step 18:

[0223] The device receives push notifications and displays them in the user interface.

[0224] (Input) Push notification sent from the server.

[0225] (Processing) The terminal analyzes the notification and displays it on the user interface.

[0226] (Output) Notification information displayed in a format that the user can view.

[0227] (Application Example 1)

[0228] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0229] A challenge for travelers is that they often need to individually check numerous information sources to obtain necessary information in a timely manner at their destination. Furthermore, when using autonomous vehicles, it is essential to be able to easily and appropriately access real-time information that changes during travel. In this situation, a system is needed that centrally provides tourist information, weather forecasts, and traffic information to improve traveler convenience. Additionally, a function that provides timely notifications of real-time information such as traffic conditions and event information is also crucial.

[0230] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0231] In this invention, the server includes means for collecting information about the travel destination from local residents, means for obtaining weather information from an external weather forecast API, means for generating clothing recommendations based on the obtained weather information, means for presenting the weather information and clothing recommendations to the traveler, means for providing live camera footage to check local conditions, means for displaying local information about the travel destination and transit points on an in-vehicle interface, and means for providing push notifications that provide real-time traffic conditions and event information. As a result, travelers can obtain diverse information about their travel destination in one place and acquire the latest information in real time, even while traveling in an autonomous vehicle.

[0232] "Travel destination information" refers to detailed local information about the region the traveler will be visiting, such as tourist attractions, restaurants, and festivals.

[0233] "Local residents" are people who permanently live in the travel destination area and provide the latest local information and recommended spots.

[0234] An "external weather forecast API" is an external application programming interface that provides weather forecasts through a web service.

[0235] "Weather information" refers to meteorological data such as temperature, rainfall, and wind speed for a specified area.

[0236] "Clothing recommendations" are suggestions based on weather information to help users choose appropriate clothing.

[0237] "Live camera footage" refers to video streaming of real-time conditions in a designated area through cameras installed within that area.

[0238] An "in-vehicle interface" refers to devices such as displays and touchscreens installed inside an autonomous vehicle.

[0239] "Push notifications" are a system that automatically sends notifications to the user's device when there is new information or an important update.

[0240] "Real-time traffic conditions" refers to highly timely information about traffic, such as current road congestion and accident information.

[0241] "Event information" refers to information about festivals, exhibitions, special displays, and other events held at your travel destination.

[0242] The system for implementing this invention is realized as follows: The system is an information provision device for autonomous vehicles equipped with features such as providing travel destination information, generating weather forecasts and clothing recommendations, providing live camera footage, and a push notification function. The system operates in cooperation with a server, an in-vehicle interface, and user input information.

[0243] Program Overview

[0244] Each function of the system operates as follows:

[0245] 1. Input and collection of travel destination information

[0246] The user enters their travel destination using the in-car interface. This involves entering GPS information and the name of the destination city.

[0247] The in-vehicle interface sends the specified travel destination information to the server in HTTP request format.

[0248] The server retrieves tourist information, restaurant information, and other data collected from local residents from a database and displays it on the in-car interface.

[0249] 2. Provision of weather forecasts and clothing recommendations.

[0250] Users request weather information for their travel destination and their current location.

[0251] Upon receiving a request from the in-vehicle interface, the server uses an external weather forecast API to retrieve weather data.

[0252] The server generates recommendation information indicating appropriate clothing based on the acquired weather information and sends it back to the in-vehicle interface.

[0253] For example, if the temperature is below 10 degrees Celsius, recommendations such as "You need a warm jacket" will be provided, and if it's above 20 degrees Celsius, "Light clothing is fine" will be offered.

[0254] 3. Provision of live camera footage

[0255] Users can request live camera footage from their travel destination or transit point.

[0256] The server retrieves the URL of the live camera for the relevant region from the database and provides it to the in-vehicle interface.

[0257] The in-vehicle interface uses the URL of the live camera to retrieve the video stream and display it to the user.

[0258] 4. Push notifications for real-time information

[0259] The server periodically checks the weather forecast API to see if there is any new information.

[0260] When there are updates to important weather forecasts, traffic conditions, or event information, the server sends the information to the in-vehicle interface via push notifications.

[0261] The in-car interface receives push notifications and displays them on the user interface, providing users with important information in real time while they are traveling.

[0262] Hardware / software to use

[0263] Hardware:

[0264] In-vehicle interface display: Installed inside the vehicle, it facilitates information exchange between the user and the system.

[0265] GPS module: Provides location information.

[0266] Internet connection module: Enables communication for sending requests to external APIs and servers.

[0267] software:

[0268] Node.js: A JavaScript runtime for running server-side programs.

[0269] Express: Used as a web application framework to handle HTTP requests.

[0270] External weather forecast API: An external service that provides weather information for a specified region.

[0271] Database: Stores and manages travel information and live camera URLs collected from local residents.

[0272] Examples of specific cases and prompt statements

[0273] Specific example: When a user enters "Kyoto City" into the in-car interface, the system provides tourist attractions, weather forecasts, and live camera footage of Kyoto City. If the weather is cold, a clothing recommendation such as "You need a warm jacket" will also be displayed.

[0274] Prompt messages: "Please provide the weather forecast and clothing recommendations for Kyoto City.", "Display live camera footage of Kyoto City."

[0275] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0276] Step 1:

[0277] The user inputs travel destination information into the in-vehicle interface.

[0278] Input: The user inputs the "name of the travel destination" into the interface.

[0279] Action: The in-vehicle interface receives the input travel destination information and sends it to the server in the form of an HTTP request.

[0280] Output: The name of the travel destination is sent as a request to the server.

[0281] Step 2:

[0282] The server obtains local information.

[0283] Input: The travel destination information sent by the user reaches the server.

[0284] Action: Based on the travel destination information, the server obtains the corresponding local information (tourist attractions, restaurant information, etc.) from the database.

[0285] Output: The obtained local information is returned to the in-vehicle interface.

[0286] Step 3:

[0287] The in-vehicle interface displays the local information.

[0288] Input: Receive the local information returned from the server.

[0289] Action: The in-vehicle interface displays the received local information on the user interface.

[0290] Output: Local information (tourist attractions, restaurant information, etc.) is displayed to the user.

[0291] Step 4:

[0292] The user requests weather information and clothing recommendations.

[0293] Input: The user interacts with the interface to enter a request for weather information.

[0294] Operation: The in-vehicle interface sends weather information requests to the server in HTTP request format.

[0295] Output: Geographic information of the travel destination is sent to the server as a weather information request.

[0296] Step 5:

[0297] The server retrieves weather information and generates clothing recommendations.

[0298] Input: Receive weather information requests sent by users.

[0299] Operation: The server uses an external weather forecast API to obtain weather information and generates clothing recommendations based on that information. For example, if the temperature is low, it will generate a recommendation such as "You need a warm jacket."

[0300] Output: Weather information and clothing recommendations are sent back to the in-vehicle interface.

[0301] Step 6:

[0302] The in-car interface displays weather information and clothing recommendations.

[0303] Input: Receive weather information and clothing recommendations sent from the server.

[0304] Operation: The in-vehicle interface displays received weather information and clothing recommendations on the user interface.

[0305] Output: Weather information and clothing recommendations are displayed to the user.

[0306] Step 7:

[0307] The user requests a live camera video.

[0308] Input: The user operates the interface to input a request for a live camera video.

[0309] Action: The in-vehicle interface sends a request for a live camera video to the server in the form of an HTTP request.

[0310] Output: Geographical information of the travel destination is sent to the server as a request for a live camera video.

[0311] Step 8:

[0312] The server obtains the URL of the live camera and provides the video.

[0313] Input: Receive a request for a live camera video sent from the user.

[0314] Action: The server obtains the URL of the live camera in the local area from the database and provides it to the in-vehicle interface.

[0315] Output: The URL of the live camera is returned to the in-vehicle interface.

[0316] Step 9:

[0317] The in-vehicle interface displays the live camera video.

[0318] Input: Receive the URL of the live camera sent from the server.

[0319] Operation: The in-vehicle interface uses the live camera's URL to retrieve the video stream and displays it on the user interface.

[0320] Output: The user will see the live camera feed.

[0321] Step 10:

[0322] The server automatically checks weather forecasts and traffic information and sends push notifications.

[0323] Input: Requests to weather forecast APIs and traffic information APIs that are executed automatically at regular intervals.

[0324] Operation: The server periodically checks external APIs and retrieves any important updates. If important information is retrieved, it is sent to the in-vehicle interface in the form of a push notification.

[0325] Output: Updated weather forecasts and traffic information are pushed to the in-car interface.

[0326] Step 11:

[0327] The in-car interface displays push notifications.

[0328] Input: Receive push notifications sent from the server.

[0329] Operation: The in-vehicle interface displays push notifications on the user interface to inform the user of important information.

[0330] Output: The user will be shown important updates to weather forecasts and traffic information.

[0331] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0332] This invention provides travel destination information, weather forecasts, clothing recommendations, and live camera footage, as well as a system that recognizes user emotions and customizes information based on those emotions. This system is implemented using a server, terminals, and an emotion engine.

[0333] Forms of information gathering

[0334] The user enters the region they are planning to travel to via their device. For example, if the user is planning a trip to Kyoto City, they would specify Kyoto City as their destination.

[0335] The terminal sends the travel destination information entered by the user to the server. This request is formatted as an HTTP request. The server receives this request and retrieves local information for the specified region from its database.

[0336] The server retrieves the latest information from its database, including local tourist attractions, restaurant information, and information provided by local residents, and sends it back to the terminal. For example, this includes information about tourist attractions and festivals in Kyoto City.

[0337] Generation methods for weather forecasts and clothing recommendations

[0338] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[0339] The terminal forwards the user's request to the server, sending an HTTP request that includes geographical information about the travel destination. The server receives this request and retrieves weather information for the specified region using an external weather forecast API.

[0340] The server generates clothing recommendations for the user based on the acquired weather information. For example, if the forecast for Kyoto City is cold, it will respond to the device with a recommendation such as, "You will need a warm jacket."

[0341] Forms of emotional engines

[0342] To recognize the user's emotions, the device is equipped with a camera and microphone, which are used to analyze the user's facial expressions and tone of voice. The emotion engine analyzes this data to recognize the user's emotional state.

[0343] The device sends recognized emotion information to the server. The server uses this information to customize weather information and clothing recommendations. For example, if the user is feeling stressed, it provides information on tourist spots suitable for relaxation and clothing recommendations accordingly.

[0344] How to view live camera footage

[0345] Users can request live camera footage to check the local situation in real time.

[0346] The device sends a request for live camera footage to the server. The server retrieves the URL of the appropriate live camera from its database and sends it back to the device.

[0347] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface. This allows the user to check the local situation in real time.

[0348] Update and notification methods

[0349] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[0350] When an important update is detected, the server will send a push notification to the relevant users.

[0351] The device receives push notifications and displays them on the user interface. This allows users to receive real-time updates on weather conditions and other important information, which they can then incorporate into their travel plans.

[0352] As a concrete example, a user is planning a trip to Kyoto City and will use the following functions during that process.

[0353] The user enters "Kyoto City" and retrieves local information.

[0354] The server retrieves information from the database and sends a response to the terminal.

[0355] The user requests a weather forecast and clothing recommendations, and the server retrieves weather information from an external API and sends appropriate clothing advice to the user's device.

[0356] Users can view live camera footage and get real-time information about the location.

[0357] The server checks for weather forecast updates and sends push notifications to the device, providing users with the latest information.

[0358] The emotion engine recognizes the user's emotions, and the server provides customized information based on that data.

[0359] This allows users to obtain all the necessary information in one place, making travel planning smoother. Furthermore, it enables personalized information delivery tailored to the user's emotional state, resulting in a more satisfying travel experience.

[0360] The following describes the processing flow.

[0361] Step 1:

[0362] The user starts up their device and enters the destination they want to go to (for example, "Kyoto City").

[0363] Step 2:

[0364] The terminal receives user input and formats that information as a request to the server.

[0365] Step 3:

[0366] The device sends a formatted request to the server. This request asks for local information about the travel destination.

[0367] Step 4:

[0368] The server receives the request and executes a query against the database to retrieve local information about "Kyoto City".

[0369] Step 5:

[0370] The server retrieves travel destination information related to "Kyoto City" from the database (such as tourist spots, restaurant information, and the latest information from local residents).

[0371] Step 6:

[0372] The server formats the acquired information into JSON format and sends it back to the terminal.

[0373] Step 7:

[0374] The terminal receives information returned from the server and displays it on the user interface. This allows the user to understand basic information about their travel destination.

[0375] Step 8:

[0376] If a user wants to know the weather in Kyoto City on a specific day, they send that request to the server via their device.

[0377] Step 9:

[0378] The device forwards the user's weather request to the server. This request includes local information and the date.

[0379] Step 10:

[0380] Based on the request received by the server, it queries an external weather forecast API for weather information.

[0381] Step 11:

[0382] The server receives weather information returned from an external weather forecast API and generates appropriate clothing recommendations based on the acquired weather information. For example, if the weather forecast is for cold weather, it will generate a recommendation such as "You need a warm jacket."

[0383] Step 12:

[0384] The server formats the weather information and generated clothing recommendations into JSON format and sends them back to the terminal.

[0385] Step 13:

[0386] The device displays weather information and clothing recommendations received by the user interface. The user can use this information to prepare for their trip.

[0387] Step 14:

[0388] If a user wants to view live camera footage from the location, they send a request to the server via their device.

[0389] Step 15:

[0390] The device forwards the live camera request to the server.

[0391] Step 16:

[0392] The server retrieves the URL of the live camera feed for "Kyoto City" from the database and sends it back to the terminal.

[0393] Step 17:

[0394] The device retrieves the video stream based on the live camera's URL and displays it on the user interface. This allows users to check the local situation in real time.

[0395] Step 18:

[0396] The system collects emotional data (such as facial expressions and voice tone) from the user through the camera and microphone built into their device.

[0397] Step 19:

[0398] The device sends collected emotional data to an emotion engine, which then recognizes the user's emotional state.

[0399] Step 20:

[0400] The emotion engine analyzes the user's emotions and sends the results back to the device.

[0401] Step 21:

[0402] The device sends recognized emotion information to the server.

[0403] Step 22:

[0404] The server customizes weather information, clothing recommendations, and other information based on emotional data. For example, if a user is feeling stressed, it will provide information on tourist spots and cafes suitable for relaxation.

[0405] Step 23:

[0406] The server formats the customized information into JSON format and sends it back to the terminal.

[0407] Step 24:

[0408] The device displays the customized information it receives in the user interface. The user can use this information to optimize their travel plan.

[0409] Step 25:

[0410] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[0411] Step 26:

[0412] When the server detects important weather updates, it sends that information as a push notification to the relevant users.

[0413] Step 27:

[0414] The device receives push notifications and displays them on the user interface. This allows users to receive the latest weather information in real time and respond quickly.

[0415] (Example 2)

[0416] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0417] There is a need for a system that allows users planning trips to efficiently obtain information about their destination and receive recommendations for appropriate clothing based on weather information. However, in conventional systems, the information is scattered and not provided in a centralized manner. Furthermore, there is a problem in that information is not customized based on the user's emotional state, making it difficult to provide personalized information. In addition, the provision of live camera footage to check real-time conditions at the destination is not integrated, so users face the inconvenience of gathering information from multiple sources.

[0418] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0419] In this invention, the server includes means for collecting travel destination information from local residents, means for obtaining weather information from an external weather forecast API, and means for generating clothing recommendations based on the acquired weather information. This allows users to centrally obtain travel destination information and receive appropriate clothing recommendations based on weather information. The invention also includes means for providing live camera footage to check local conditions, and an emotion engine that recognizes and analyzes the user's emotions using the terminal's camera and microphone, and customizes weather information and clothing recommendations based on the recognized user emotion information. This enables personalized information provision according to the user's emotional state, not only allowing travel planning to proceed more smoothly but also providing a more satisfying travel experience.

[0420] "Travel destination information" refers to detailed information about tourist attractions, restaurants, events, festivals, and other information related to the region where the user is planning to travel.

[0421] "Local residents" refer to people who actually live in the area where the user is planning to travel, and the information they provide is the most up-to-date local information.

[0422] An "external weather forecast API" is an application programming interface for a service that provides weather forecast information via the internet.

[0423] "Weather information" refers to information about current and future weather conditions (temperature, precipitation, wind speed, etc.) for a specific region.

[0424] "Clothing recommendations" refer to information that suggests appropriate clothing to the user based on acquired weather information.

[0425] A "traveler" is a user who is planning a trip.

[0426] "Live camera footage" refers to a video stream that captures real-time footage of the local situation in a specific area and distributes it via the internet.

[0427] A "device" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[0428] A "database" is a collection of data used to systematically store and manage information, and it is accessible from a server.

[0429] "User interface" refers to the screens and operating methods that serve as the point of contact between a system and a user when inputting or outputting information.

[0430] An "HTTP request" is a communication protocol used to request the transmission or reception of information from a server over the internet.

[0431] An "emotion engine" is software that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

[0432] "Push notifications" are a technology that allows a server to send information to a user's device in real time.

[0433] This invention provides travel destination information, weather forecasts, clothing recommendations, and live camera footage, as well as a system that recognizes user emotions and customizes information based on those emotions. This system is implemented using a server, terminals, and an emotion engine.

[0434] Forms of information gathering

[0435] The user enters the region they are planning to travel to via their device. For example, they might enter "Kyoto City".

[0436] The terminal sends the travel destination information entered by the user to the server as an HTTP request. The server receives this request and retrieves local information for the specified region from its database.

[0437] The server retrieves the latest information from its database, including local tourist attractions, restaurant information, and information provided by local residents, and sends it back to the terminal. For example, this includes information about tourist attractions and festivals in Kyoto City.

[0438] Example of a prompt:

[0439] Please enter "Kyoto City" as your travel destination.

[0440] Generation methods for weather forecasts and clothing recommendations

[0441] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[0442] The terminal forwards the user's request to the server, sending an HTTP request that includes geographical information about the travel destination.

[0443] The server receives this request and uses an external weather forecast API to retrieve weather information for the specified area. Based on the retrieved weather information, it generates clothing recommendations suitable for the user. For example, if the forecast for Kyoto City is cold, it will return a recommendation to the device such as, "You will need a warm jacket."

[0444] Example of a prompt:

[0445] Could you please tell me the weather forecast for Kyoto City?

[0446] Forms of emotional engines

[0447] To recognize the user's emotions, the device is equipped with a camera and microphone, which are used to analyze the user's facial expressions and tone of voice.

[0448] The emotion engine analyzes this data to recognize the user's emotional state.

[0449] The device sends the recognized emotion information to the server.

[0450] The server uses this information to customize weather forecasts and clothing recommendations. For example, if a user is feeling stressed, it will provide information on relaxing tourist spots and appropriate clothing recommendations.

[0451] Example of a prompt:

[0452] Please recommend some places based on your current emotional state.

[0453] How to view live camera footage

[0454] Users request live camera footage to check the local situation in real time.

[0455] The device sends a request for live camera footage to the server.

[0456] The server retrieves the URL of the appropriate live camera from the database and sends it back to the terminal.

[0457] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface. This allows the user to check the local situation in real time.

[0458] Example of a prompt:

[0459] Please show me the live camera footage of Kyoto City.

[0460] Update and notification methods

[0461] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[0462] When an important update is detected, the server will send a push notification to the relevant users.

[0463] The device receives push notifications and displays them on the user interface. This allows users to receive real-time updates on weather conditions and other important information, which they can then incorporate into their travel plans.

[0464] Example of a prompt:

[0465] Please provide real-time notifications of the latest weather forecast for Kyoto City.

[0466] This system allows users to efficiently receive a series of pieces of information and plan a comfortable trip. Furthermore, it enables personalized information delivery tailored to the user's emotional state, resulting in a highly satisfying travel experience.

[0467] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0468] Forms of information gathering

[0469] Step 1:

[0470] The user enters the region they are planning to travel to via their device. For example, they might enter "Kyoto City".

[0471] Input: Travel destination (e.g., "Kyoto City")

[0472] Output: Entered travel destination information

[0473] Step 2:

[0474] The terminal formats the travel destination information entered by the user as an HTTP request.

[0475] Input: Entered travel destination information

[0476] Output: HTTP request containing travel destination information

[0477] Step 3:

[0478] The device sends travel destination information to the server as an HTTP request.

[0479] Input: HTTP request containing travel destination information

[0480] Output: Sending a request to the server

[0481] Step 4:

[0482] The server receives this request and retrieves local information for the specified region from the database.

[0483] Input: HTTP request containing travel destination information

[0484] Output: Local information (e.g., tourist spots, restaurant information, etc.)

[0485] Step 5:

[0486] The server returns the retrieved information to the terminal in JSON format.

[0487] Input: Local information

[0488] Output: HTTP response for the terminal

[0489] Step 6:

[0490] The terminal parses the received JSON data and displays it in the user interface.

[0491] Input: Response from server

[0492] Output: Local information displayed in the user interface

[0493] Generation methods for weather forecasts and clothing recommendations

[0494] Step 1:

[0495] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[0496] Input: Weather information request

[0497] Output: Request for terminal

[0498] Step 2:

[0499] The device generates an HTTP request containing geographical information about the travel destination and sends it to the server.

[0500] Input: Weather information request

[0501] Output: HTTP request containing geographical information

[0502] Step 3:

[0503] The server receives this request and uses an external weather forecast API to retrieve weather information for the specified area.

[0504] Input: HTTP request containing geographical information

[0505] Output: Weather information

[0506] Step 4:

[0507] The server generates clothing recommendations suitable for the user based on the acquired weather information.

[0508] Input: Weather information

[0509] Output: Clothing recommendation (e.g., "You need a warm jacket")

[0510] Step 5:

[0511] The server returns the generated recommendations to the device.

[0512] Input: Clothing Recommendation

[0513] Output: HTTP response for the terminal

[0514] Step 6:

[0515] The device displays the received clothing recommendations in the user interface.

[0516] Input: Response from server

[0517] Output: Clothing recommendations displayed in the user interface

[0518] Forms of emotional engines

[0519] Step 1:

[0520] The device is equipped with a camera and microphone to collect the user's facial expressions and voice tone.

[0521] Input: User's facial expressions and tone of voice

[0522] Output: Collected data

[0523] Step 2:

[0524] The emotion engine analyzes this data to recognize the user's emotional state.

[0525] Input: Collected data

[0526] Output: Recognized emotional state

[0527] Step 3:

[0528] The device sends the recognized emotion information to the server.

[0529] Input: Recognized emotional state

[0530] Output: HTTP request to the server

[0531] Step 4:

[0532] The server uses this information to customize weather forecasts and clothing recommendations.

[0533] Input: Emotional information

[0534] Output: Customized weather information and clothing recommendations

[0535] Step 5:

[0536] The server returns customized information to the terminal.

[0537] Input: Customized information

[0538] Output: HTTP response for the terminal

[0539] How to view live camera footage

[0540] Step 1:

[0541] Users request live camera footage to check the local situation in real time.

[0542] Input: Live camera video request

[0543] Output: Request for terminal

[0544] Step 2:

[0545] The device sends a request for live camera footage to the server.

[0546] Input: Live camera video request

[0547] Output: HTTP request to the server

[0548] Step 3:

[0549] The server retrieves the URL of the appropriate live camera from the database and sends it back to the terminal.

[0550] Input: Live camera video request

[0551] Output: URL of the live camera

[0552] Step 4:

[0553] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface.

[0554] Input: URL of the live camera

[0555] Output: Video stream displayed on the user interface

[0556] Update and notification methods

[0557] Step 1:

[0558] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[0559] Input: Periodic check request

[0560] Output: Latest weather information

[0561] Step 2:

[0562] When an important update is detected, the server will send a push notification to the relevant users.

[0563] Input: Latest weather information

[0564] Output: Push notification

[0565] Step 3:

[0566] The device receives push notifications and displays them on the user interface.

[0567] Input: Push notification

[0568] Output: Notification displayed in the user interface

[0569] Through the above processing steps, users can efficiently obtain information about their travel destination and receive appropriate advice based on the latest information in real time. Furthermore, by providing customized information that responds to the user's emotions, a highly satisfying travel experience can be achieved.

[0570] (Application Example 2)

[0571] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0572] Traditional food delivery systems simply deliver the food ordered by the user, but they do not adequately provide information such as travel destination details, weather, live camera footage, or personalized information based on the user's emotions. Furthermore, features such as delivery time predictions based on weather information in the delivery area and real-time delivery status checks are lacking. This has resulted in a lack of an environment where users can utilize a more comfortable and efficient service, which is a significant challenge.

[0573] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for collecting travel destination information from local residents, means for obtaining weather information from an external weather forecast API, means for generating clothing recommendations based on the acquired weather information, means for presenting weather information and clothing recommendations to travelers, means for recognizing the user's emotions and customizing information based on those emotions, means for providing live camera footage to check local conditions, means for recommending appropriate food based on the user's emotions, means for predicting delivery times based on weather information in the delivery area and notifying the user, and means for checking the delivery status in real time. This makes it possible to provide personalized information, improve delivery efficiency, and enhance the user experience, which were challenges in the past.

[0574] "Travel destination information" refers to local information about the travel destination the user is planning, such as tourist spots, restaurants, and festivals.

[0575] "Local residents" are people who live in the destination city and have the role of providing local information.

[0576] An "external weather forecast API" is an API (Application Programming Interface) for an external service that provides weather information via the internet.

[0577] "Clothing recommendations" are systems that suggest the most suitable clothing for the user based on acquired weather information.

[0578] "User emotions" refer to the mental state perceived from the user's facial expressions, tone of voice, and other similar factors.

[0579] "Means of recognizing emotions" refers to technologies that use cameras and microphones to analyze a user's emotional state.

[0580] "Customizing information" means appropriately changing the information provided according to the user's emotions and circumstances.

[0581] "Local conditions" refers to the real-time environment and events at the travel destination.

[0582] "Live camera footage" refers to video that streams the situation at the location in real time.

[0583] "Food recommendations" refer to recommending appropriate food items based on the user's emotional state and local information.

[0584] "Delivery area" refers to the region or area where food delivery services are provided.

[0585] "Predicting delivery time" means estimating the time required for delivery based on factors such as weather and traffic conditions.

[0586] "Checking delivery status in real time" means being able to instantly monitor the current location and progress of the delivery driver.

[0587] A "server" is a central computer system that processes data and provides information to users.

[0588] This document describes a system designed to provide personalized information and streamline delivery for travelers using food delivery services. This system primarily consists of a server, terminals, and an emotion engine.

[0589] Hardware and software configuration

[0590] The following hardware and software are used to implement this system:

[0591] Hardware:

[0592] User devices (smartphones, tablets)

[0593] server

[0594] Delivery driver's device (smartphone)

[0595] Camera (for live camera)

[0596] software:

[0597] Mobile application (iOS / ANDROID® registered trademark)

[0598] Web server (Node.js)

[0599] Database (MySQL(registered trademark))

[0600] Weather forecast APIs (such as OpenWeatherMap)

[0601] Emotion recognition engines (Amazon Rekognition, Microsoft® Azure® Face API, etc.)

[0602] Program processing

[0603] 1. Gathering information about your travel destination:

[0604] The user enters travel destination information from their device. For example, they might specify "Kyoto City" as their travel destination.

[0605] The device sends information about the specified travel destination to the server.

[0606] The server retrieves information collected from local residents from a database and responds to users with information such as tourist attractions and restaurants in their travel destination.

[0607] 2. Weather information and clothing recommendations:

[0608] The user sends a request from their device to the server to get weather information for their travel destination.

[0609] The server uses an external weather forecast API to obtain weather information for the specified region.

[0610] Based on the acquired weather information, the server generates a recommendation for appropriate clothing and sends it back to the device.

[0611] 3. Emotion Recognition and Personalization:

[0612] The device is equipped with a camera and microphone, which are used to analyze the user's facial expressions and voice tone.

[0613] The emotion engine analyzes this data to recognize the user's emotional state.

[0614] Based on the recognized emotional information, the server customizes the information to provide appropriate food recommendations and special messages.

[0615] 4. Weather and delivery time forecast for the delivery area:

[0616] The server uses geographical information of the delivery area to obtain weather information from a weather forecast API.

[0617] Based on weather information, the server predicts the delivery time and notifies the terminal.

[0618] 5. Provision of live camera footage:

[0619] Users request live camera footage to check the situation on site.

[0620] The server retrieves the appropriate live camera URL from the database and sends it back to the terminal.

[0621] The device displays live camera footage to the user in real time.

[0622] 6. Real-time delivery status check:

[0623] Obtain the delivery driver's current location information.

[0624] The server uses this information to notify the user of the delivery status in real time.

[0625] Examples of specific cases and prompt statements

[0626] As a concrete example, consider a case where the user is tired. When the user orders food delivery using their device, the emotion engine recognizes the user's fatigue level and recommends an "energy drink to give them a boost." Furthermore, if it's raining in the delivery area, it recommends a "warm soup."

[0627] Example of a prompt:

[0628] "When a user is tired, the app should recommend a nutritious energy drink and allow them to check the delivery driver's location via live video. Also, on rainy days, it should recommend a warm soup."

[0629] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0630] Step 1:

[0631] Gathering information about your travel destination

[0632] Input: The user enters travel destination information into the terminal.

[0633] Specific operation: The user opens an application on their device and enters a location, such as "Kyoto City," as their travel destination. The device then sends this information to the server.

[0634] Data processing: The server retrieves and searches information collected from local residents in the database.

[0635] Output: The server retrieves information on tourist spots and restaurants in Kyoto City and sends it to the terminal.

[0636] Step 2:

[0637] Obtaining weather information and clothing recommendations

[0638] Input: The user requests weather information.

[0639] Specific operation: The user sends a request from their device to the server to get weather information for their travel destination. The server uses an external weather forecast API to retrieve weather information for the specified area.

[0640] Data processing: The server analyzes the acquired weather information and generates clothing recommendations based on the weather.

[0641] Output: The server returns the generated clothing recommendations to the terminal.

[0642] Step 3:

[0643] emotion recognition

[0644] Input: User's camera video and audio data.

[0645] Specific operation: The device is equipped with a camera and microphone, which are used to capture the user's facial expressions and voice tone. The captured data is then sent to an emotion recognition engine.

[0646] Data processing: The emotion recognition engine analyzes facial expressions and voice tone to determine the user's emotional state.

[0647] Output: Sends recognized emotion data to the server.

[0648] Step 4:

[0649] Emotion-based information customization

[0650] Input: Emotion recognition data.

[0651] Specific operation: The server receives emotion recognition data and generates information (such as special food recommendations or messages) that corresponds to the user's emotional state.

[0652] Data processing: The server selects and customizes appropriate information from the database based on the emotional state.

[0653] Output: Sends customized information to the device.

[0654] Step 5:

[0655] Weather information and delivery time forecast for the delivery area.

[0656] Input: Geographical information of the delivery area.

[0657] Specific operation: The server uses geographical information of the delivery area to obtain weather information from an external weather forecast API. Based on this, the server predicts the delivery time.

[0658] Data processing: Using an algorithm based on weather information, delivery times are predicted.

[0659] Output: Notifies the device of the estimated delivery time.

[0660] Step 6:

[0661] Live camera footage provided

[0662] Input: User request for live camera footage.

[0663] Specific operation: The user requests live camera footage through their device. The server retrieves the appropriate live camera URL from the database and sends it back to the device.

[0664] Data processing: The server selects the appropriate live camera URL and returns it.

[0665] Output: The terminal displays live camera footage to the user in real time.

[0666] Step 7:

[0667] Real-time delivery status check

[0668] Input: Delivery driver's current location information.

[0669] Specific operation: The server obtains location information from the delivery driver's terminal. The server uses this information to notify the user of the delivery status in real time.

[0670] Data processing: Analyze delivery progress based on current location information.

[0671] Output: Notifies the terminal of the real-time delivery status.

[0672] Through these steps, users can receive information about their travel destination, weather, live camera footage, and personalized information tailored to their emotions, allowing them to enjoy a convenient food delivery service.

[0673] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0674] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0675] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0676] [Second Embodiment]

[0677] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0678] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0679] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0680] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0681] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0682] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0683] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0684] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0685] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0686] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0687] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0688] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0689] This invention is a system that comprehensively provides information such as travel destination information, weather forecasts, clothing recommendations, and live camera footage. This system operates through the cooperation of a server and a terminal (user's device).

[0690] Forms of information gathering

[0691] The user enters the region they are planning to travel to via their device. For example, if the user is planning a trip to Kyoto City, they would specify Kyoto City as their destination.

[0692] The terminal sends the travel destination information entered by the user to the server. This request is formatted as an HTTP request. The server receives this request and retrieves local information for the specified region from its database.

[0693] The server retrieves the latest information from its database, including local tourist attractions, restaurant information, and information provided by local residents, and sends it back to the terminal. For example, this includes information about tourist attractions and festivals in Kyoto City.

[0694] Generation methods for weather forecasts and clothing recommendations

[0695] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[0696] The terminal forwards the user's request to the server, sending an HTTP request that includes geographical information about the travel destination. The server receives this request and retrieves weather information for the specified region using an external weather forecast API.

[0697] The server generates clothing recommendations for the user based on the acquired weather information. For example, if the forecast for Kyoto City is cold, it will respond to the device with a recommendation such as, "You will need a warm jacket."

[0698] How to view live camera footage

[0699] Users can request live camera footage to check the local situation in real time.

[0700] The device sends a request for live camera footage to the server. The server retrieves the URL of the appropriate live camera from its database and sends it back to the device.

[0701] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface. This allows the user to check the local situation in real time.

[0702] Update and notification methods

[0703] The server periodically checks the weather forecast API and retrieves any updated weather information. If an important update is detected, it sends a push notification to the relevant users.

[0704] The device receives push notifications and displays them on the user interface. This allows users to receive real-time updates on weather conditions and other important information, which they can then incorporate into their travel plans.

[0705] As a concrete example, a user is planning a trip to Kyoto City and will use the following functions during that process.

[0706] The user enters "Kyoto City" and retrieves local information.

[0707] The server retrieves information from the database and sends a response to the terminal.

[0708] The user requests a weather forecast and clothing recommendations, and the server retrieves weather information from an external API and sends appropriate clothing advice to the user's device.

[0709] Users can view live camera footage and get real-time information about the location.

[0710] The server checks for weather forecast updates and sends push notifications to the device, providing users with the latest information.

[0711] This allows users to obtain all the necessary information in one place, making travel planning much smoother.

[0712] The following describes the processing flow.

[0713] Step 1:

[0714] The user starts up their device and enters the destination they want to go to (for example, "Kyoto City").

[0715] Step 2:

[0716] The terminal receives user input and formats that information as a request to the server.

[0717] Step 3:

[0718] The device sends a formatted request to the server. This request asks for local information about the travel destination.

[0719] Step 4:

[0720] The server receives the request and executes a query against the database to retrieve local information about "Kyoto City".

[0721] Step 5:

[0722] The server retrieves travel destination information related to "Kyoto City" from the database (such as tourist spots, restaurant information, and the latest information from local residents).

[0723] Step 6:

[0724] The server formats the acquired information into JSON format and sends it back to the terminal.

[0725] Step 7:

[0726] The terminal receives information returned from the server and displays it on the user interface. This allows the user to understand basic information about their travel destination.

[0727] Step 8:

[0728] If a user wants to know the weather in Kyoto City on a specific day, they send that request to the server via their device.

[0729] Step 9:

[0730] The device forwards the user's weather request to the server. This request includes local information and the date.

[0731] Step 10:

[0732] Based on the request received by the server, it queries an external weather forecast API for weather information.

[0733] Step 11:

[0734] The server receives weather information returned from an external weather forecast API and generates appropriate clothing recommendations based on the acquired weather information. For example, if the weather forecast is for cold weather, it will generate a recommendation such as "You need a warm jacket."

[0735] Step 12:

[0736] The server formats the weather information and generated clothing recommendations into JSON format and sends them back to the terminal.

[0737] Step 13:

[0738] The device displays weather information and clothing recommendations received by the user interface. The user can use this information to prepare for their trip.

[0739] Step 14:

[0740] If a user wants to view live camera footage from the location, they send a request to the server via their device.

[0741] Step 15:

[0742] The device forwards the live camera request to the server.

[0743] Step 16:

[0744] The server retrieves the URL of the live camera feed for "Kyoto City" from the database and sends it back to the terminal.

[0745] Step 17:

[0746] The device retrieves the video stream based on the live camera's URL and displays it on the user interface. This allows users to check the local situation in real time.

[0747] Step 18:

[0748] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[0749] Step 19:

[0750] When the server detects important weather updates, it sends that information as a push notification to the relevant users.

[0751] Step 20:

[0752] The device receives push notifications and displays them on the user interface. This allows users to receive the latest weather information in real time and respond quickly.

[0753] (Example 1)

[0754] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0755] Modern travelers have a growing need for detailed information and real-time updates on their destinations. However, obtaining the right information in one place often requires using multiple different applications and websites, which is inconvenient. Furthermore, there is no information system that allows travelers to check weather forecasts and local conditions in real time and respond quickly. As a result, it is difficult for travelers to plan efficient and comfortable trips.

[0756] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0757] In this invention, the server includes means for collecting information about the travel destination from local residents, means for obtaining weather information from an external weather forecast API, means for generating clothing recommendations based on the obtained weather information, means for receiving regional information about the travel destination entered by the traveler, means for presenting weather information and clothing recommendations to the traveler, means for providing live camera footage to check local conditions, and means for obtaining and providing traveler information on tourist spots and restaurants related to the travel destination from a database. This enables travelers to comprehensively obtain information about their travel destination within a single system and plan their trip efficiently and comfortably.

[0758] "Travel destination information" refers to a general term for tourist attractions, restaurants, events, and other region-specific information related to the area you plan to visit.

[0759] "Local residents" refers to people who permanently live in the travel destination region and can provide up-to-date and specific information about that area.

[0760] An "external weather forecast API" refers to a programmatic interface for obtaining weather forecast data provided by a third party.

[0761] "Clothing recommendations" refer to information that suggests appropriate clothing for travelers based on acquired weather data.

[0762] A "traveler" refers to an individual or group planning a trip to a specific region.

[0763] "Local information" refers collectively to geographical, social, and economic information concerning a specific region.

[0764] A "live camera" refers to a device or system that broadcasts real-time video from a specific location over the internet.

[0765] A "database" refers to a collection of data organized for the purpose of efficiently searching, retrieving, and managing information.

[0766] "Push notification" refers to a communication method in which a server sends information to a user in real time.

[0767] A "user interface" is an interface through which a user interacts with a system, and includes visual or functional elements.

[0768] This invention is a system that comprehensively provides travelers with information such as destination information, weather forecasts, clothing recommendations, and live camera footage. This system operates through the cooperation of a server and a terminal (user's device).

[0769] The user enters the region they are planning to travel to via their device. For example, if the user is planning a trip to Kyoto City, they would enter "Kyoto City" as the destination. This input information becomes the starting point for the system.

[0770] The terminal formats the travel destination information entered by the user as an HTTP request and sends it to the server. This is done over a standard internet connection. When the server receives the request, it accesses its internal database and external weather forecast APIs (e.g., OpenWeatherMap API) to collect the necessary information.

[0771] The server first accesses an internal database to retrieve information on tourist spots, restaurants, and events in a specified area (e.g., Kyoto City). A database called GLDB (General Location Database) can be used for this retrieval. Next, the server accesses an external weather forecast API to obtain the latest weather information. This API access can be done using, for example, an OpenWeatherMap API key.

[0772] Based on the acquired weather information, the server executes a process to generate clothing recommendations suitable for the user. For example, if the forecast for Kyoto City is cold, it will generate a specific recommendation such as "You will need a warm jacket."

[0773] The terminal receives information from the server and displays it in the user interface. This user interface is built using web technologies such as HTML / CSS and JavaScript. Here, users can check information on tourist attractions and restaurants in their travel destination, as well as weather forecasts and clothing recommendations.

[0774] Users can also request live camera footage to check real-time information from the location. The device sends this request to the server, which retrieves the URL of the corresponding live camera from its database and returns it to the device. The device uses the retrieved live camera URL to obtain the video stream and displays it on the user interface.

[0775] Furthermore, the server periodically checks external weather forecast APIs and retrieves any important weather forecast updates. When an important update is detected, the server also has a function to send push notifications to the relevant users. The device receives this push notification and displays it in the user interface, allowing users to obtain the latest information about their travel destination in real time.

[0776] As a concrete example, the following series of operations can be considered.

[0777] 1. The user enters "Kyoto City" and retrieves local information.

[0778] 2. The server retrieves information on tourist spots and restaurants in Kyoto City from the database and sends it back to the terminal.

[0779] 3. The user requests a weather forecast and clothing recommendations.

[0780] 4. The server retrieves weather information from an external API, generates appropriate clothing recommendations, and sends them to the device.

[0781] 5. Users can view live camera footage and understand the real-time situation on site.

[0782] 6. The server checks for weather forecast updates and sends important update information to the device via push notification.

[0783] Examples of prompt statements include:

[0784] "Travel destination information: Kyoto City, weather forecast, clothing recommendations, live cameras"

[0785] It can be done this way.

[0786] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0787] Step 1:

[0788] The user uses their device to enter information about their travel destination (e.g., "Kyoto City").

[0789] (Input) Travel destination information entered by the user.

[0790] (Processing) The terminal formats the user's input as an HTTP request.

[0791] (Output) A formatted HTTP request is generated.

[0792] Step 2:

[0793] The device sends an HTTP request to the server containing the travel destination information entered by the user.

[0794] (Input) A formatted HTTP request.

[0795] (Processing) The terminal sends a request to the server via the internet.

[0796] (Output) The server receives an HTTP request.

[0797] Step 3:

[0798] Based on the received request, the server retrieves tourist spots and restaurant information for the travel destination (e.g., "Kyoto City") from the database.

[0799] (Input) A request containing the user's travel destination information.

[0800] (Processing) The server queries the database to retrieve relevant information.

[0801] (Output) Acquired information on tourist spots and restaurants.

[0802] Step 4:

[0803] The server returns the acquired tourist spot and restaurant information to the terminal.

[0804] (Input) Information retrieved by the server from the database.

[0805] (Processing) The server formats the information and sends it to the terminal as an HTTP response.

[0806] (Output) Formatted HTTP response.

[0807] Step 5:

[0808] The terminal displays information received from the server on the user interface.

[0809] (Input) HTTP response received from the server.

[0810] (Processing) The terminal analyzes the information and displays it on the user interface.

[0811] (Output) Information displayed in a format that the user can view.

[0812] Step 6:

[0813] The user enters information into the device to request weather forecasts and clothing recommendations.

[0814] (Input) Geographical information of your travel destination.

[0815] (Processing) The user enters the necessary information into the terminal.

[0816] (Output) Input weather forecast and clothing request information.

[0817] Step 7:

[0818] The device sends an HTTP request to the server that includes weather forecasts and clothing recommendations.

[0819] (Input) Weather forecast and clothing request information entered by the user.

[0820] (Processing) The terminal formats the information as an HTTP request and sends it to the server.

[0821] (Output) HTTP requests received by the server.

[0822] Step 8:

[0823] The server accesses an external weather forecast API to obtain weather information for a specified area (e.g., "Kyoto City").

[0824] (Input) Regional information included in the HTTP request.

[0825] (Processing) The server calls an external weather forecast API to obtain weather information.

[0826] (Output) Acquired weather information.

[0827] Step 9:

[0828] The server generates clothing recommendations based on the acquired weather information.

[0829] (Input) Obtained weather information.

[0830] (Processing) The server analyzes weather information and executes an algorithm to recommend appropriate clothing.

[0831] (Output) Generated clothing recommendations.

[0832] Step 10:

[0833] The server sends the generated clothing recommendations to the device.

[0834] (Input) Clothing recommendation.

[0835] (Processing) The server formats the clothing recommendation into an HTTP response and sends it to the terminal.

[0836] (Output) Formatted HTTP response.

[0837] Step 11:

[0838] The device displays the received clothing recommendations in the user interface.

[0839] (Input) HTTP response received from the server.

[0840] (Processing) The terminal analyzes the information and displays it on the user interface.

[0841] (Output) Clothing recommendations displayed in a format viewable by the user.

[0842] Step 12:

[0843] Users request live camera footage to check real-time information from the location.

[0844] (Input) Request for live camera footage.

[0845] (Processing) The user enters a request on the terminal.

[0846] (Output) Input live camera request information.

[0847] Step 13:

[0848] The terminal sends a request for live camera footage to the server.

[0849] (Input) Live camera request information.

[0850] (Processing) The terminal formats the information as an HTTP request and sends it to the server.

[0851] (Output) HTTP requests received by the server.

[0852] Step 14:

[0853] The server retrieves the URL of the corresponding live camera from the database and sends it back to the terminal.

[0854] (Input) Regional information included in the request.

[0855] (Processing) The server queries the database to obtain the URL of the live camera.

[0856] (Output) The retrieved live camera URL.

[0857] Step 15:

[0858] The device uses the received URL of the live camera to retrieve the video stream and displays it in the user interface.

[0859] (Input) The URL of the live camera received from the server.

[0860] (Processing) The terminal acquires the video stream from the live camera and displays it on the user interface.

[0861] (Output) Live camera footage displayed in a format that can be viewed by the user.

[0862] Step 16:

[0863] The server periodically checks the weather forecast API to obtain the latest weather information.

[0864] (Input) Periodic calls to a weather forecast API.

[0865] (Processing) The server periodically calls the weather forecast API to obtain new weather information.

[0866] (Output) The latest weather information obtained.

[0867] Step 17:

[0868] When there are important updates, the server creates and sends push notifications to the relevant users.

[0869] (Input) The latest weather information obtained.

[0870] (Processing) The server generates a push notification based on important update information and sends it to the relevant users.

[0871] (Output) Sent push notification.

[0872] Step 18:

[0873] The device receives push notifications and displays them in the user interface.

[0874] (Input) Push notification sent from the server.

[0875] (Processing) The terminal analyzes the notification and displays it on the user interface.

[0876] (Output) Notification information displayed in a format that the user can view.

[0877] (Application Example 1)

[0878] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0879] A challenge for travelers is that they often need to individually check numerous information sources to obtain necessary information in a timely manner at their destination. Furthermore, when using autonomous vehicles, it is essential to be able to easily and appropriately access real-time information that changes during travel. In this situation, a system is needed that centrally provides tourist information, weather forecasts, and traffic information to improve traveler convenience. Additionally, a function that provides timely notifications of real-time information such as traffic conditions and event information is also crucial.

[0880] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0881] In this invention, the server includes means for collecting information about the travel destination from local residents, means for obtaining weather information from an external weather forecast API, means for generating clothing recommendations based on the obtained weather information, means for presenting the weather information and clothing recommendations to the traveler, means for providing live camera footage to check local conditions, means for displaying local information about the travel destination and transit points on an in-vehicle interface, and means for providing push notifications that provide real-time traffic conditions and event information. As a result, travelers can obtain diverse information about their travel destination in one place and acquire the latest information in real time, even while traveling in an autonomous vehicle.

[0882] "Travel destination information" refers to detailed local information about the region the traveler will be visiting, such as tourist attractions, restaurants, and festivals.

[0883] "Local residents" are people who permanently live in the travel destination area and provide the latest local information and recommended spots.

[0884] An "external weather forecast API" is an external application programming interface that provides weather forecasts through a web service.

[0885] "Weather information" refers to meteorological data such as temperature, rainfall, and wind speed for a specified area.

[0886] "Clothing recommendations" are suggestions based on weather information to help users choose appropriate clothing.

[0887] "Live camera footage" refers to video streaming of real-time conditions in a designated area through cameras installed within that area.

[0888] An "in-vehicle interface" refers to devices such as displays and touchscreens installed inside an autonomous vehicle.

[0889] "Push notifications" are a system that automatically sends notifications to the user's device when there is new information or an important update.

[0890] "Real-time traffic conditions" refers to highly timely information about traffic, such as current road congestion and accident information.

[0891] "Event information" refers to information about festivals, exhibitions, special displays, and other events held at your travel destination.

[0892] The system for implementing this invention is realized as follows: The system is an information provision device for autonomous vehicles equipped with features such as providing travel destination information, generating weather forecasts and clothing recommendations, providing live camera footage, and a push notification function. The system operates in cooperation with a server, an in-vehicle interface, and user input information.

[0893] Program Overview

[0894] Each function of the system operates as follows:

[0895] 1. Input and collection of travel destination information

[0896] The user enters their travel destination using the in-car interface. This involves entering GPS information and the name of the destination city.

[0897] The in-vehicle interface sends the specified travel destination information to the server in HTTP request format.

[0898] The server retrieves tourist information, restaurant information, and other data collected from local residents from a database and displays it on the in-car interface.

[0899] 2. Provision of weather forecasts and clothing recommendations.

[0900] Users request weather information for their travel destination and their current location.

[0901] Upon receiving a request from the in-vehicle interface, the server uses an external weather forecast API to retrieve weather data.

[0902] The server generates recommendation information indicating appropriate clothing based on the acquired weather information and sends it back to the in-vehicle interface.

[0903] For example, if the temperature is below 10 degrees Celsius, recommendations such as "You need a warm jacket" will be provided, and if it's above 20 degrees Celsius, "Light clothing is fine" will be offered.

[0904] 3. Provision of live camera footage

[0905] Users can request live camera footage from their travel destination or transit point.

[0906] The server retrieves the URL of the live camera for the relevant region from the database and provides it to the in-vehicle interface.

[0907] The in-vehicle interface uses the URL of the live camera to retrieve the video stream and display it to the user.

[0908] 4. Push notifications for real-time information

[0909] The server periodically checks the weather forecast API to see if there is any new information.

[0910] When there are updates to important weather forecasts, traffic conditions, or event information, the server sends the information to the in-vehicle interface via push notifications.

[0911] The in-car interface receives push notifications and displays them on the user interface, providing users with important information in real time while they are traveling.

[0912] Hardware / software to use

[0913] Hardware:

[0914] In-vehicle interface display: Installed inside the vehicle, it facilitates information exchange between the user and the system.

[0915] GPS module: Provides location information.

[0916] Internet connection module: Enables communication for sending requests to external APIs and servers.

[0917] software:

[0918] Node.js: A JavaScript runtime for running server-side programs.

[0919] Express: Used as a web application framework to handle HTTP requests.

[0920] External weather forecast API: An external service that provides weather information for a specified region.

[0921] Database: Stores and manages travel information and live camera URLs collected from local residents.

[0922] Examples of specific cases and prompt statements

[0923] Specific example: When a user enters "Kyoto City" into the in-car interface, the system provides tourist attractions, weather forecasts, and live camera footage of Kyoto City. If the weather is cold, a clothing recommendation such as "You need a warm jacket" will also be displayed.

[0924] Prompt messages: "Please provide the weather forecast and clothing recommendations for Kyoto City.", "Display live camera footage of Kyoto City."

[0925] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0926] Step 1:

[0927] The user enters information about their travel destination into the in-car interface.

[0928] Input: The user enters the name of their travel destination into the interface.

[0929] Operation: The in-vehicle interface receives the entered travel destination information and sends it to the server in the form of an HTTP request.

[0930] Output: The name of the travel destination is sent as a request to the server.

[0931] Step 2:

[0932] The server retrieves local information.

[0933] Input: Travel destination information sent by the user arrives at the server.

[0934] Operation: Based on the travel destination information, the server retrieves relevant local information (tourist attractions, restaurant information, etc.) from the database.

[0935] Output: The acquired local information is sent back to the in-vehicle interface.

[0936] Step 3:

[0937] The in-vehicle interface displays local information.

[0938] Input: Receive local information returned from the server.

[0939] Operation: The in-vehicle interface displays the received local information on the user interface.

[0940] Output: Local information (tourist attractions, restaurant information, etc.) is displayed to the user.

[0941] Step 4:

[0942] The user requests weather information and clothing recommendations.

[0943] Input: The user interacts with the interface to enter a request for weather information.

[0944] Operation: The in-vehicle interface sends weather information requests to the server in HTTP request format.

[0945] Output: Geographic information of the travel destination is sent to the server as a weather information request.

[0946] Step 5:

[0947] The server retrieves weather information and generates clothing recommendations.

[0948] Input: Receive weather information requests sent by users.

[0949] Operation: The server uses an external weather forecast API to obtain weather information and generates clothing recommendations based on that information. For example, if the temperature is low, it will generate a recommendation such as "You need a warm jacket."

[0950] Output: Weather information and clothing recommendations are sent back to the in-vehicle interface.

[0951] Step 6:

[0952] The in-car interface displays weather information and clothing recommendations.

[0953] Input: Receive weather information and clothing recommendations sent from the server.

[0954] Operation: The in-vehicle interface displays received weather information and clothing recommendations on the user interface.

[0955] Output: The user is shown weather information and clothing recommendations.

[0956] Step 7:

[0957] A user requests live camera footage.

[0958] Input: The user interacts with the interface to submit a request for live camera footage.

[0959] Operation: The in-vehicle interface sends live camera video requests to the server in HTTP request format.

[0960] Output: Geographic information of the travel destination is sent to the server as a request for live camera footage.

[0961] Step 8:

[0962] The server retrieves the URL of the live camera and provides the video feed.

[0963] Input: Receive requests for live camera footage sent by users.

[0964] Operation: The server retrieves the URL of the live camera in the relevant area from the database and provides it to the in-vehicle interface.

[0965] Output: The URL of the live camera is sent back to the in-vehicle interface.

[0966] Step 9:

[0967] The in-vehicle interface displays live camera footage.

[0968] Input: Receive the URL of the live camera sent from the server.

[0969] Operation: The in-vehicle interface uses the live camera's URL to retrieve the video stream and displays it on the user interface.

[0970] Output: The user will see the live camera feed.

[0971] Step 10:

[0972] The server automatically checks weather forecasts and traffic information and sends push notifications.

[0973] Input: Requests to weather forecast APIs and traffic information APIs that are executed automatically at regular intervals.

[0974] Operation: The server periodically checks external APIs and retrieves any important updates. If important information is retrieved, it is sent to the in-vehicle interface in the form of a push notification.

[0975] Output: Updated weather forecasts and traffic information are pushed to the in-car interface.

[0976] Step 11:

[0977] The in-car interface displays push notifications.

[0978] Input: Receive push notifications sent from the server.

[0979] Operation: The in-vehicle interface displays push notifications on the user interface to inform the user of important information.

[0980] Output: The user will be shown important updates to weather forecasts and traffic information.

[0981] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0982] This invention provides travel destination information, weather forecasts, clothing recommendations, and live camera footage, as well as a system that recognizes user emotions and customizes information based on those emotions. This system is implemented using a server, terminals, and an emotion engine.

[0983] Forms of information gathering

[0984] The user enters the region they are planning to travel to via their device. For example, if the user is planning a trip to Kyoto City, they would specify Kyoto City as their destination.

[0985] The terminal sends the travel destination information entered by the user to the server. This request is formatted as an HTTP request. The server receives this request and retrieves local information for the specified region from its database.

[0986] The server retrieves the latest information from its database, including local tourist attractions, restaurant information, and information provided by local residents, and sends it back to the terminal. For example, this includes information about tourist attractions and festivals in Kyoto City.

[0987] Generation methods for weather forecasts and clothing recommendations

[0988] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[0989] The terminal forwards the user's request to the server, sending an HTTP request that includes geographical information about the travel destination. The server receives this request and retrieves weather information for the specified region using an external weather forecast API.

[0990] The server generates clothing recommendations for the user based on the acquired weather information. For example, if the forecast for Kyoto City is cold, it will respond to the device with a recommendation such as, "You will need a warm jacket."

[0991] Forms of emotional engines

[0992] To recognize the user's emotions, the device is equipped with a camera and microphone, which are used to analyze the user's facial expressions and tone of voice. The emotion engine analyzes this data to recognize the user's emotional state.

[0993] The device sends recognized emotion information to the server. The server uses this information to customize weather information and clothing recommendations. For example, if the user is feeling stressed, it provides information on tourist spots suitable for relaxation and clothing recommendations accordingly.

[0994] How to view live camera footage

[0995] Users can request live camera footage to check the local situation in real time.

[0996] The device sends a request for live camera footage to the server. The server retrieves the URL of the appropriate live camera from its database and sends it back to the device.

[0997] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface. This allows the user to check the local situation in real time.

[0998] Update and notification methods

[0999] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[1000] When an important update is detected, the server will send a push notification to the relevant users.

[1001] The device receives push notifications and displays them on the user interface. This allows users to receive real-time updates on weather conditions and other important information, which they can then incorporate into their travel plans.

[1002] As a concrete example, a user is planning a trip to Kyoto City and will use the following functions during that process.

[1003] The user enters "Kyoto City" and retrieves local information.

[1004] The server retrieves information from the database and sends a response to the terminal.

[1005] The user requests a weather forecast and clothing recommendations, and the server retrieves weather information from an external API and sends appropriate clothing advice to the user's device.

[1006] Users can view live camera footage and get real-time information about the location.

[1007] The server checks for weather forecast updates and sends push notifications to the device, providing users with the latest information.

[1008] The emotion engine recognizes the user's emotions, and the server provides customized information based on that data.

[1009] This allows users to obtain all the necessary information in one place, making travel planning smoother. Furthermore, it enables personalized information delivery tailored to the user's emotional state, resulting in a more satisfying travel experience.

[1010] The following describes the processing flow.

[1011] Step 1:

[1012] The user starts up their device and enters the destination they want to go to (for example, "Kyoto City").

[1013] Step 2:

[1014] The terminal receives user input and formats that information as a request to the server.

[1015] Step 3:

[1016] The device sends a formatted request to the server. This request asks for local information about the travel destination.

[1017] Step 4:

[1018] The server receives the request and executes a query against the database to retrieve local information about "Kyoto City".

[1019] Step 5:

[1020] The server retrieves travel destination information related to "Kyoto City" from the database (such as tourist spots, restaurant information, and the latest information from local residents).

[1021] Step 6:

[1022] The server formats the acquired information into JSON format and sends it back to the terminal.

[1023] Step 7:

[1024] The terminal receives information returned from the server and displays it on the user interface. This allows the user to understand basic information about their travel destination.

[1025] Step 8:

[1026] If a user wants to know the weather in Kyoto City on a specific day, they send that request to the server via their device.

[1027] Step 9:

[1028] The device forwards the user's weather request to the server. This request includes local information and the date.

[1029] Step 10:

[1030] Based on the request received by the server, it queries an external weather forecast API for weather information.

[1031] Step 11:

[1032] The server receives weather information returned from an external weather forecast API and generates appropriate clothing recommendations based on the acquired weather information. For example, if the weather forecast is for cold weather, it will generate a recommendation such as "You need a warm jacket."

[1033] Step 12:

[1034] The server formats the weather information and generated clothing recommendations into JSON format and sends them back to the terminal.

[1035] Step 13:

[1036] The device displays weather information and clothing recommendations received by the user interface. The user can use this information to prepare for their trip.

[1037] Step 14:

[1038] If a user wants to view live camera footage from the location, they send a request to the server via their device.

[1039] Step 15:

[1040] The device forwards the live camera request to the server.

[1041] Step 16:

[1042] The server retrieves the URL of the live camera feed for "Kyoto City" from the database and sends it back to the terminal.

[1043] Step 17:

[1044] The device retrieves the video stream based on the live camera's URL and displays it on the user interface. This allows users to check the local situation in real time.

[1045] Step 18:

[1046] The system collects emotional data (such as facial expressions and voice tone) from the user through the camera and microphone built into their device.

[1047] Step 19:

[1048] The device sends collected emotional data to an emotion engine, which then recognizes the user's emotional state.

[1049] Step 20:

[1050] The emotion engine analyzes the user's emotions and sends the results back to the device.

[1051] Step 21:

[1052] The device sends recognized emotion information to the server.

[1053] Step 22:

[1054] The server customizes weather information, clothing recommendations, and other information based on emotional data. For example, if a user is feeling stressed, it will provide information on tourist spots and cafes suitable for relaxation.

[1055] Step 23:

[1056] The server formats the customized information into JSON format and sends it back to the terminal.

[1057] Step 24:

[1058] The device displays the customized information it receives in the user interface. The user can use this information to optimize their travel plan.

[1059] Step 25:

[1060] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[1061] Step 26:

[1062] When the server detects important weather updates, it sends that information as a push notification to the relevant users.

[1063] Step 27:

[1064] The device receives push notifications and displays them on the user interface. This allows users to receive the latest weather information in real time and respond quickly.

[1065] (Example 2)

[1066] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1067] There is a need for a system that allows users planning trips to efficiently obtain information about their destination and receive recommendations for appropriate clothing based on weather information. However, in conventional systems, the information is scattered and not provided in a centralized manner. Furthermore, there is a problem in that information is not customized based on the user's emotional state, making it difficult to provide personalized information. In addition, the provision of live camera footage to check real-time conditions at the destination is not integrated, so users face the inconvenience of gathering information from multiple sources.

[1068] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1069] In this invention, the server includes means for collecting travel destination information from local residents, means for obtaining weather information from an external weather forecast API, and means for generating clothing recommendations based on the acquired weather information. This allows users to centrally obtain travel destination information and receive appropriate clothing recommendations based on weather information. The invention also includes means for providing live camera footage to check local conditions, and an emotion engine that recognizes and analyzes the user's emotions using the terminal's camera and microphone, and customizes weather information and clothing recommendations based on the recognized user emotion information. This enables personalized information provision according to the user's emotional state, not only allowing travel planning to proceed more smoothly but also providing a more satisfying travel experience.

[1070] "Travel destination information" refers to detailed information about tourist attractions, restaurants, events, festivals, and other information related to the region where the user is planning to travel.

[1071] "Local residents" refer to people who actually live in the area where the user is planning to travel, and the information they provide is the most up-to-date local information.

[1072] An "external weather forecast API" is an application programming interface for a service that provides weather forecast information via the internet.

[1073] "Weather information" refers to information about current and future weather conditions (temperature, precipitation, wind speed, etc.) for a specific region.

[1074] "Clothing recommendations" refer to information that suggests appropriate clothing to the user based on acquired weather information.

[1075] A "traveler" is a user who is planning a trip.

[1076] "Live camera footage" refers to a video stream that captures real-time footage of the local situation in a specific area and distributes it via the internet.

[1077] A "device" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[1078] A "database" is a collection of data used to systematically store and manage information, and it is accessible from a server.

[1079] "User interface" refers to the screens and operating methods that serve as the point of contact between a system and a user when inputting or outputting information.

[1080] An "HTTP request" is a communication protocol used to request the transmission or reception of information from a server over the internet.

[1081] An "emotion engine" is software that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

[1082] "Push notifications" are a technology that allows a server to send information to a user's device in real time.

[1083] This invention provides travel destination information, weather forecasts, clothing recommendations, and live camera footage, as well as a system that recognizes user emotions and customizes information based on those emotions. This system is implemented using a server, terminals, and an emotion engine.

[1084] Forms of information gathering

[1085] The user enters the region they are planning to travel to via their device. For example, they might enter "Kyoto City".

[1086] The terminal sends the travel destination information entered by the user to the server as an HTTP request. The server receives this request and retrieves local information for the specified region from its database.

[1087] The server retrieves the latest information from its database, including local tourist attractions, restaurant information, and information provided by local residents, and sends it back to the terminal. For example, this includes information about tourist attractions and festivals in Kyoto City.

[1088] Example of a prompt:

[1089] Please enter "Kyoto City" as your travel destination.

[1090] Generation methods for weather forecasts and clothing recommendations

[1091] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[1092] The terminal forwards the user's request to the server, sending an HTTP request that includes geographical information about the travel destination.

[1093] The server receives this request and uses an external weather forecast API to retrieve weather information for the specified area. Based on the retrieved weather information, it generates clothing recommendations suitable for the user. For example, if the forecast for Kyoto City is cold, it will return a recommendation to the device such as, "You will need a warm jacket."

[1094] Example of a prompt:

[1095] Could you please tell me the weather forecast for Kyoto City?

[1096] Forms of emotional engines

[1097] To recognize the user's emotions, the device is equipped with a camera and microphone, which are used to analyze the user's facial expressions and tone of voice.

[1098] The emotion engine analyzes this data to recognize the user's emotional state.

[1099] The device sends the recognized emotion information to the server.

[1100] The server uses this information to customize weather forecasts and clothing recommendations. For example, if a user is feeling stressed, it will provide information on relaxing tourist spots and appropriate clothing recommendations.

[1101] Example of a prompt:

[1102] Please recommend some places based on your current emotional state.

[1103] How to view live camera footage

[1104] Users request live camera footage to check the local situation in real time.

[1105] The device sends a request for live camera footage to the server.

[1106] The server retrieves the URL of the appropriate live camera from the database and sends it back to the terminal.

[1107] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface. This allows the user to check the local situation in real time.

[1108] Example of a prompt:

[1109] Please show me the live camera footage of Kyoto City.

[1110] Update and notification methods

[1111] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[1112] When an important update is detected, the server will send a push notification to the relevant users.

[1113] The device receives push notifications and displays them on the user interface. This allows users to receive real-time updates on weather conditions and other important information, which they can then incorporate into their travel plans.

[1114] Example of a prompt:

[1115] Please provide real-time notifications of the latest weather forecast for Kyoto City.

[1116] This system allows users to efficiently receive a series of pieces of information and plan a comfortable trip. Furthermore, it enables personalized information delivery tailored to the user's emotional state, resulting in a highly satisfying travel experience.

[1117] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1118] Forms of information gathering

[1119] Step 1:

[1120] The user enters the region they are planning to travel to via their device. For example, they might enter "Kyoto City".

[1121] Input: Travel destination (e.g., "Kyoto City")

[1122] Output: Entered travel destination information

[1123] Step 2:

[1124] The terminal formats the travel destination information entered by the user as an HTTP request.

[1125] Input: Entered travel destination information

[1126] Output: HTTP request containing travel destination information

[1127] Step 3:

[1128] The device sends travel destination information to the server as an HTTP request.

[1129] Input: HTTP request containing travel destination information

[1130] Output: Sending a request to the server

[1131] Step 4:

[1132] The server receives this request and retrieves local information for the specified region from the database.

[1133] Input: HTTP request containing travel destination information

[1134] Output: Local information (e.g., tourist spots, restaurant information, etc.)

[1135] Step 5:

[1136] The server returns the retrieved information to the terminal in JSON format.

[1137] Input: Local information

[1138] Output: HTTP response for the terminal

[1139] Step 6:

[1140] The terminal parses the received JSON data and displays it in the user interface.

[1141] Input: Response from server

[1142] Output: Local information displayed in the user interface

[1143] Generation methods for weather forecasts and clothing recommendations

[1144] Step 1:

[1145] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[1146] Input: Weather information request

[1147] Output: Request for terminal

[1148] Step 2:

[1149] The device generates an HTTP request containing geographical information about the travel destination and sends it to the server.

[1150] Input: Weather information request

[1151] Output: HTTP request containing geographical information

[1152] Step 3:

[1153] The server receives this request and uses an external weather forecast API to retrieve weather information for the specified area.

[1154] Input: HTTP request containing geographical information

[1155] Output: Weather information

[1156] Step 4:

[1157] The server generates clothing recommendations suitable for the user based on the acquired weather information.

[1158] Input: Weather information

[1159] Output: Clothing recommendation (e.g., "You need a warm jacket")

[1160] Step 5:

[1161] The server returns the generated recommendations to the device.

[1162] Input: Clothing Recommendation

[1163] Output: HTTP response for the terminal

[1164] Step 6:

[1165] The device displays the received clothing recommendations in the user interface.

[1166] Input: Response from server

[1167] Output: Clothing recommendations displayed in the user interface

[1168] Forms of emotional engines

[1169] Step 1:

[1170] The device is equipped with a camera and microphone to collect the user's facial expressions and voice tone.

[1171] Input: User's facial expressions and tone of voice

[1172] Output: Collected data

[1173] Step 2:

[1174] The emotion engine analyzes this data to recognize the user's emotional state.

[1175] Input: Collected data

[1176] Output: Recognized emotional state

[1177] Step 3:

[1178] The device sends the recognized emotion information to the server.

[1179] Input: Recognized emotional state

[1180] Output: HTTP request to the server

[1181] Step 4:

[1182] The server uses this information to customize weather forecasts and clothing recommendations.

[1183] Input: Emotional information

[1184] Output: Customized weather information and clothing recommendations

[1185] Step 5:

[1186] The server returns customized information to the terminal.

[1187] Input: Customized information

[1188] Output: HTTP response for the terminal

[1189] How to view live camera footage

[1190] Step 1:

[1191] Users request live camera footage to check the local situation in real time.

[1192] Input: Live camera video request

[1193] Output: Request for terminal

[1194] Step 2:

[1195] The device sends a request for live camera footage to the server.

[1196] Input: Live camera video request

[1197] Output: HTTP request to the server

[1198] Step 3:

[1199] The server retrieves the URL of the appropriate live camera from the database and sends it back to the terminal.

[1200] Input: Live camera video request

[1201] Output: URL of the live camera

[1202] Step 4:

[1203] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface.

[1204] Input: URL of the live camera

[1205] Output: Video stream displayed on the user interface

[1206] Update and notification methods

[1207] Step 1:

[1208] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[1209] Input: Periodic check request

[1210] Output: Latest weather information

[1211] Step 2:

[1212] When an important update is detected, the server will send a push notification to the relevant users.

[1213] Input: Latest weather information

[1214] Output: Push notification

[1215] Step 3:

[1216] The device receives push notifications and displays them on the user interface.

[1217] Input: Push notification

[1218] Output: Notification displayed in the user interface

[1219] Through the above processing steps, users can efficiently obtain information about their travel destination and receive appropriate advice based on the latest information in real time. Furthermore, by providing customized information that responds to the user's emotions, a highly satisfying travel experience can be achieved.

[1220] (Application Example 2)

[1221] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1222] Traditional food delivery systems simply deliver the food ordered by the user, but they do not adequately provide information such as travel destination details, weather, live camera footage, or personalized information based on the user's emotions. Furthermore, features such as delivery time predictions based on weather information in the delivery area and real-time delivery status checks are lacking. This has resulted in a lack of an environment where users can utilize a more comfortable and efficient service, which is a significant challenge.

[1223] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for collecting travel destination information from local residents, means for obtaining weather information from an external weather forecast API, means for generating clothing recommendations based on the acquired weather information, means for presenting weather information and clothing recommendations to travelers, means for recognizing the user's emotions and customizing information based on those emotions, means for providing live camera footage to check local conditions, means for recommending appropriate food based on the user's emotions, means for predicting delivery times based on weather information in the delivery area and notifying the user, and means for checking the delivery status in real time. This makes it possible to provide personalized information, improve delivery efficiency, and enhance the user experience, which were challenges in the past.

[1224] "Travel destination information" refers to local information about the travel destination the user is planning, such as tourist spots, restaurants, and festivals.

[1225] "Local residents" are people who live in the destination city and have the role of providing local information.

[1226] An "external weather forecast API" is an API (Application Programming Interface) for an external service that provides weather information via the internet.

[1227] "Clothing recommendations" are systems that suggest the most suitable clothing for the user based on acquired weather information.

[1228] "User emotions" refer to the mental state perceived from the user's facial expressions, tone of voice, and other similar factors.

[1229] "Means of recognizing emotions" refers to technologies that use cameras and microphones to analyze a user's emotional state.

[1230] "Customizing information" means appropriately changing the information provided according to the user's emotions and circumstances.

[1231] "Local conditions" refers to the real-time environment and events at the travel destination.

[1232] "Live camera footage" refers to video that streams the situation at the location in real time.

[1233] "Food recommendations" refer to recommending appropriate food items based on the user's emotional state and local information.

[1234] "Delivery area" refers to the region or area where food delivery services are provided.

[1235] "Predicting delivery time" means estimating the time required for delivery based on factors such as weather and traffic conditions.

[1236] "Checking delivery status in real time" means being able to instantly monitor the current location and progress of the delivery driver.

[1237] A "server" is a central computer system that processes data and provides information to users.

[1238] This document describes a system designed to provide personalized information and streamline delivery for travelers using food delivery services. This system primarily consists of a server, terminals, and an emotion engine.

[1239] Hardware and software configuration

[1240] The following hardware and software are used to implement this system:

[1241] Hardware:

[1242] User devices (smartphones, tablets)

[1243] server

[1244] Delivery driver's device (smartphone)

[1245] Camera (for live camera)

[1246] software:

[1247] Mobile applications (iOS / Android)

[1248] Web server (Node.js)

[1249] Database (MySQL)

[1250] Weather forecast APIs (such as OpenWeatherMap)

[1251] Emotion recognition engines (Amazon Rekognition, Microsoft Azure Face API, etc.)

[1252] Program processing

[1253] 1. Gathering information about your travel destination:

[1254] The user enters travel destination information from their device. For example, they might specify "Kyoto City" as their travel destination.

[1255] The device sends information about the specified travel destination to the server.

[1256] The server retrieves information collected from local residents from a database and responds to users with information such as tourist attractions and restaurants in their travel destination.

[1257] 2. Weather information and clothing recommendations:

[1258] The user sends a request from their device to the server to get weather information for their travel destination.

[1259] The server uses an external weather forecast API to obtain weather information for the specified region.

[1260] Based on the acquired weather information, the server generates a recommendation for appropriate clothing and sends it back to the device.

[1261] 3. Emotion Recognition and Personalization:

[1262] The device is equipped with a camera and microphone, which are used to analyze the user's facial expressions and voice tone.

[1263] The emotion engine analyzes this data to recognize the user's emotional state.

[1264] Based on the recognized emotional information, the server customizes the information to provide appropriate food recommendations and special messages.

[1265] 4. Weather and delivery time forecast for the delivery area:

[1266] The server uses geographical information of the delivery area to obtain weather information from a weather forecast API.

[1267] Based on weather information, the server predicts the delivery time and notifies the terminal.

[1268] 5. Provision of live camera footage:

[1269] Users request live camera footage to check the situation on site.

[1270] The server retrieves the appropriate live camera URL from the database and sends it back to the terminal.

[1271] The device displays live camera footage to the user in real time.

[1272] 6. Real-time delivery status check:

[1273] Obtain the delivery driver's current location information.

[1274] The server uses this information to notify the user of the delivery status in real time.

[1275] Examples of specific cases and prompt statements

[1276] As a concrete example, consider a case where the user is tired. When the user orders food delivery using their device, the emotion engine recognizes the user's fatigue level and recommends an "energy drink to give them a boost." Furthermore, if it's raining in the delivery area, it recommends a "warm soup."

[1277] Example of a prompt:

[1278] "When a user is tired, the app should recommend a nutritious energy drink and allow them to check the delivery driver's location via live video. Also, on rainy days, it should recommend a warm soup."

[1279] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1280] Step 1:

[1281] Gathering information about your travel destination

[1282] Input: The user enters travel destination information into the terminal.

[1283] Specific operation: The user opens an application on their device and enters a location, such as "Kyoto City," as their travel destination. The device then sends this information to the server.

[1284] Data processing: The server retrieves and searches information collected from local residents in the database.

[1285] Output: The server retrieves information on tourist spots and restaurants in Kyoto City and sends it to the terminal.

[1286] Step 2:

[1287] Obtaining weather information and clothing recommendations

[1288] Input: The user requests weather information.

[1289] Specific operation: The user sends a request from their device to the server to get weather information for their travel destination. The server uses an external weather forecast API to retrieve weather information for the specified area.

[1290] Data processing: The server analyzes the acquired weather information and generates clothing recommendations based on the weather.

[1291] Output: The server returns the generated clothing recommendations to the terminal.

[1292] Step 3:

[1293] emotion recognition

[1294] Input: User's camera video and audio data.

[1295] Specific operation: The device is equipped with a camera and microphone, which are used to capture the user's facial expressions and voice tone. The captured data is then sent to an emotion recognition engine.

[1296] Data processing: The emotion recognition engine analyzes facial expressions and voice tone to determine the user's emotional state.

[1297] Output: Sends recognized emotion data to the server.

[1298] Step 4:

[1299] Emotion-based information customization

[1300] Input: Emotion recognition data.

[1301] Specific operation: The server receives emotion recognition data and generates information (such as special food recommendations or messages) that corresponds to the user's emotional state.

[1302] Data processing: The server selects and customizes appropriate information from the database based on the emotional state.

[1303] Output: Sends customized information to the device.

[1304] Step 5:

[1305] Weather information and delivery time forecast for the delivery area.

[1306] Input: Geographical information of the delivery area.

[1307] Specific operation: The server uses geographical information of the delivery area to obtain weather information from an external weather forecast API. Based on this, the server predicts the delivery time.

[1308] Data processing: Using an algorithm based on weather information, delivery times are predicted.

[1309] Output: Notifies the device of the estimated delivery time.

[1310] Step 6:

[1311] Live camera footage provided

[1312] Input: User request for live camera footage.

[1313] Specific operation: The user requests live camera footage through their device. The server retrieves the appropriate live camera URL from the database and sends it back to the device.

[1314] Data processing: The server selects the appropriate live camera URL and returns it.

[1315] Output: The terminal displays live camera footage to the user in real time.

[1316] Step 7:

[1317] Real-time delivery status check

[1318] Input: Delivery driver's current location information.

[1319] Specific operation: The server obtains location information from the delivery driver's terminal. The server uses this information to notify the user of the delivery status in real time.

[1320] Data processing: Analyze delivery progress based on current location information.

[1321] Output: Notifies the terminal of the real-time delivery status.

[1322] Through these steps, users can receive information about their travel destination, weather, live camera footage, and personalized information tailored to their emotions, allowing them to enjoy a convenient food delivery service.

[1323] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1324] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1325] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1326] [Third Embodiment]

[1327] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1328] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1329] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1330] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1331] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1332] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1333] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1334] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1335] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1336] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1337] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1338] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1339] This invention is a system that comprehensively provides information such as travel destination information, weather forecasts, clothing recommendations, and live camera footage. This system operates through the cooperation of a server and a terminal (user's device).

[1340] Forms of information gathering

[1341] The user enters the region they are planning to travel to via their device. For example, if the user is planning a trip to Kyoto City, they would specify Kyoto City as their destination.

[1342] The terminal sends the travel destination information entered by the user to the server. This request is formatted as an HTTP request. The server receives this request and retrieves local information for the specified region from its database.

[1343] The server retrieves the latest information from its database, including local tourist attractions, restaurant information, and information provided by local residents, and sends it back to the terminal. For example, this includes information about tourist attractions and festivals in Kyoto City.

[1344] Generation methods for weather forecasts and clothing recommendations

[1345] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[1346] The terminal forwards the user's request to the server, sending an HTTP request that includes geographical information about the travel destination. The server receives this request and retrieves weather information for the specified region using an external weather forecast API.

[1347] The server generates clothing recommendations for the user based on the acquired weather information. For example, if the forecast for Kyoto City is cold, it will respond to the device with a recommendation such as, "You will need a warm jacket."

[1348] How to view live camera footage

[1349] Users can request live camera footage to check the local situation in real time.

[1350] The device sends a request for live camera footage to the server. The server retrieves the URL of the appropriate live camera from its database and sends it back to the device.

[1351] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface. This allows the user to check the local situation in real time.

[1352] Update and notification methods

[1353] The server periodically checks the weather forecast API and retrieves any updated weather information. If an important update is detected, it sends a push notification to the relevant users.

[1354] The device receives push notifications and displays them on the user interface. This allows users to receive real-time updates on weather conditions and other important information, which they can then incorporate into their travel plans.

[1355] As a concrete example, a user is planning a trip to Kyoto City and will use the following functions during that process.

[1356] The user enters "Kyoto City" and retrieves local information.

[1357] The server retrieves information from the database and sends a response to the terminal.

[1358] The user requests a weather forecast and clothing recommendations, and the server retrieves weather information from an external API and sends appropriate clothing advice to the user's device.

[1359] Users can view live camera footage and get real-time information about the location.

[1360] The server checks for weather forecast updates and sends push notifications to the device, providing users with the latest information.

[1361] This allows users to obtain all the necessary information in one place, making travel planning much smoother.

[1362] The following describes the processing flow.

[1363] Step 1:

[1364] The user starts up their device and enters the destination they want to go to (for example, "Kyoto City").

[1365] Step 2:

[1366] The terminal receives user input and formats that information as a request to the server.

[1367] Step 3:

[1368] The device sends a formatted request to the server. This request asks for local information about the travel destination.

[1369] Step 4:

[1370] The server receives the request and executes a query against the database to retrieve local information about "Kyoto City".

[1371] Step 5:

[1372] The server retrieves travel destination information related to "Kyoto City" from the database (such as tourist spots, restaurant information, and the latest information from local residents).

[1373] Step 6:

[1374] The server formats the acquired information into JSON format and sends it back to the terminal.

[1375] Step 7:

[1376] The terminal receives information returned from the server and displays it on the user interface. This allows the user to understand basic information about their travel destination.

[1377] Step 8:

[1378] If a user wants to know the weather in Kyoto City on a specific day, they send that request to the server via their device.

[1379] Step 9:

[1380] The device forwards the user's weather request to the server. This request includes local information and the date.

[1381] Step 10:

[1382] Based on the request received by the server, it queries an external weather forecast API for weather information.

[1383] Step 11:

[1384] The server receives weather information returned from an external weather forecast API and generates appropriate clothing recommendations based on the acquired weather information. For example, if the weather forecast is for cold weather, it will generate a recommendation such as "You need a warm jacket."

[1385] Step 12:

[1386] The server formats the weather information and generated clothing recommendations into JSON format and sends them back to the terminal.

[1387] Step 13:

[1388] The device displays weather information and clothing recommendations received by the user interface. The user can use this information to prepare for their trip.

[1389] Step 14:

[1390] If a user wants to view live camera footage from the location, they send a request to the server via their device.

[1391] Step 15:

[1392] The device forwards the live camera request to the server.

[1393] Step 16:

[1394] The server retrieves the URL of the live camera feed for "Kyoto City" from the database and sends it back to the terminal.

[1395] Step 17:

[1396] The device retrieves the video stream based on the live camera's URL and displays it on the user interface. This allows users to check the local situation in real time.

[1397] Step 18:

[1398] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[1399] Step 19:

[1400] When the server detects important weather updates, it sends that information as a push notification to the relevant users.

[1401] Step 20:

[1402] The device receives push notifications and displays them on the user interface. This allows users to receive the latest weather information in real time and respond quickly.

[1403] (Example 1)

[1404] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1405] Modern travelers have a growing need for detailed information and real-time updates on their destinations. However, obtaining the right information in one place often requires using multiple different applications and websites, which is inconvenient. Furthermore, there is no information system that allows travelers to check weather forecasts and local conditions in real time and respond quickly. As a result, it is difficult for travelers to plan efficient and comfortable trips.

[1406] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1407] In this invention, the server includes means for collecting information about the travel destination from local residents, means for obtaining weather information from an external weather forecast API, means for generating clothing recommendations based on the obtained weather information, means for receiving regional information about the travel destination entered by the traveler, means for presenting weather information and clothing recommendations to the traveler, means for providing live camera footage to check local conditions, and means for obtaining and providing traveler information on tourist spots and restaurants related to the travel destination from a database. This enables travelers to comprehensively obtain information about their travel destination within a single system and plan their trip efficiently and comfortably.

[1408] "Travel destination information" refers to a general term for tourist attractions, restaurants, events, and other region-specific information related to the area you plan to visit.

[1409] "Local residents" refers to people who permanently live in the travel destination region and can provide up-to-date and specific information about that area.

[1410] An "external weather forecast API" refers to a programmatic interface for obtaining weather forecast data provided by a third party.

[1411] "Clothing recommendations" refer to information that suggests appropriate clothing for travelers based on acquired weather data.

[1412] A "traveler" refers to an individual or group planning a trip to a specific region.

[1413] "Local information" refers collectively to geographical, social, and economic information concerning a specific region.

[1414] A "live camera" refers to a device or system that broadcasts real-time video from a specific location over the internet.

[1415] A "database" refers to a collection of data organized for the purpose of efficiently searching, retrieving, and managing information.

[1416] "Push notification" refers to a communication method in which a server sends information to a user in real time.

[1417] A "user interface" is an interface through which a user interacts with a system, and includes visual or functional elements.

[1418] This invention is a system that comprehensively provides travelers with information such as destination information, weather forecasts, clothing recommendations, and live camera footage. This system operates through the cooperation of a server and a terminal (user's device).

[1419] The user enters the region they are planning to travel to via their device. For example, if the user is planning a trip to Kyoto City, they would enter "Kyoto City" as the destination. This input information becomes the starting point for the system.

[1420] The terminal formats the travel destination information entered by the user as an HTTP request and sends it to the server. This is done over a standard internet connection. When the server receives the request, it accesses its internal database and external weather forecast APIs (e.g., OpenWeatherMap API) to collect the necessary information.

[1421] The server first accesses an internal database to retrieve information on tourist spots, restaurants, and events in a specified area (e.g., Kyoto City). A database called GLDB (General Location Database) can be used for this retrieval. Next, the server accesses an external weather forecast API to obtain the latest weather information. This API access can be done using, for example, an OpenWeatherMap API key.

[1422] Based on the acquired weather information, the server executes a process to generate clothing recommendations suitable for the user. For example, if the forecast for Kyoto City is cold, it will generate a specific recommendation such as "You will need a warm jacket."

[1423] The terminal receives information from the server and displays it in the user interface. This user interface is built using web technologies such as HTML / CSS and JavaScript. Here, users can check information on tourist attractions and restaurants in their travel destination, as well as weather forecasts and clothing recommendations.

[1424] Users can also request live camera footage to check real-time information from the location. The device sends this request to the server, which retrieves the URL of the corresponding live camera from its database and returns it to the device. The device uses the retrieved live camera URL to obtain the video stream and displays it on the user interface.

[1425] Furthermore, the server periodically checks external weather forecast APIs and retrieves any important weather forecast updates. When an important update is detected, the server also has a function to send push notifications to the relevant users. The device receives this push notification and displays it in the user interface, allowing users to obtain the latest information about their travel destination in real time.

[1426] As a concrete example, the following series of operations can be considered.

[1427] 1. The user enters "Kyoto City" and retrieves local information.

[1428] 2. The server retrieves information on tourist spots and restaurants in Kyoto City from the database and sends it back to the terminal.

[1429] 3. The user requests a weather forecast and clothing recommendations.

[1430] 4. The server retrieves weather information from an external API, generates appropriate clothing recommendations, and sends them to the device.

[1431] 5. Users can view live camera footage and understand the real-time situation on site.

[1432] 6. The server checks for weather forecast updates and sends important update information to the device via push notification.

[1433] Examples of prompt statements include:

[1434] "Travel destination information: Kyoto City, weather forecast, clothing recommendations, live cameras"

[1435] It can be done this way.

[1436] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1437] Step 1:

[1438] The user uses their device to enter information about their travel destination (e.g., "Kyoto City").

[1439] (Input) Travel destination information entered by the user.

[1440] (Processing) The terminal formats the user's input as an HTTP request.

[1441] (Output) A formatted HTTP request is generated.

[1442] Step 2:

[1443] The device sends an HTTP request to the server containing the travel destination information entered by the user.

[1444] (Input) A formatted HTTP request.

[1445] (Processing) The terminal sends a request to the server via the internet.

[1446] (Output) The server receives an HTTP request.

[1447] Step 3:

[1448] Based on the received request, the server retrieves tourist spots and restaurant information for the travel destination (e.g., "Kyoto City") from the database.

[1449] (Input) A request containing the user's travel destination information.

[1450] (Processing) The server queries the database to retrieve relevant information.

[1451] (Output) Acquired information on tourist spots and restaurants.

[1452] Step 4:

[1453] The server returns the acquired tourist spot and restaurant information to the terminal.

[1454] (Input) Information retrieved by the server from the database.

[1455] (Processing) The server formats the information and sends it to the terminal as an HTTP response.

[1456] (Output) Formatted HTTP response.

[1457] Step 5:

[1458] The terminal displays information received from the server on the user interface.

[1459] (Input) HTTP response received from the server.

[1460] (Processing) The terminal analyzes the information and displays it on the user interface.

[1461] (Output) Information displayed in a format that the user can view.

[1462] Step 6:

[1463] The user enters information into the device to request weather forecasts and clothing recommendations.

[1464] (Input) Geographical information of your travel destination.

[1465] (Processing) The user enters the necessary information into the terminal.

[1466] (Output) Input weather forecast and clothing request information.

[1467] Step 7:

[1468] The device sends an HTTP request to the server that includes weather forecasts and clothing recommendations.

[1469] (Input) Weather forecast and clothing request information entered by the user.

[1470] (Processing) The terminal formats the information as an HTTP request and sends it to the server.

[1471] (Output) HTTP requests received by the server.

[1472] Step 8:

[1473] The server accesses an external weather forecast API to obtain weather information for a specified area (e.g., "Kyoto City").

[1474] (Input) Regional information included in the HTTP request.

[1475] (Processing) The server calls an external weather forecast API to obtain weather information.

[1476] (Output) Acquired weather information.

[1477] Step 9:

[1478] The server generates clothing recommendations based on the acquired weather information.

[1479] (Input) Obtained weather information.

[1480] (Processing) The server analyzes weather information and executes an algorithm to recommend appropriate clothing.

[1481] (Output) Generated clothing recommendations.

[1482] Step 10:

[1483] The server sends the generated clothing recommendations to the device.

[1484] (Input) Clothing recommendation.

[1485] (Processing) The server formats the clothing recommendation into an HTTP response and sends it to the terminal.

[1486] (Output) Formatted HTTP response.

[1487] Step 11:

[1488] The device displays the received clothing recommendations in the user interface.

[1489] (Input) HTTP response received from the server.

[1490] (Processing) The terminal analyzes the information and displays it on the user interface.

[1491] (Output) Clothing recommendations displayed in a format viewable by the user.

[1492] Step 12:

[1493] Users request live camera footage to check real-time information from the location.

[1494] (Input) Request for live camera footage.

[1495] (Processing) The user enters a request on the terminal.

[1496] (Output) Input live camera request information.

[1497] Step 13:

[1498] The terminal sends a request for live camera footage to the server.

[1499] (Input) Live camera request information.

[1500] (Processing) The terminal formats the information as an HTTP request and sends it to the server.

[1501] (Output) HTTP requests received by the server.

[1502] Step 14:

[1503] The server retrieves the URL of the corresponding live camera from the database and sends it back to the terminal.

[1504] (Input) Regional information included in the request.

[1505] (Processing) The server queries the database to obtain the URL of the live camera.

[1506] (Output) The retrieved live camera URL.

[1507] Step 15:

[1508] The device uses the received URL of the live camera to retrieve the video stream and displays it in the user interface.

[1509] (Input) The URL of the live camera received from the server.

[1510] (Processing) The terminal acquires the video stream from the live camera and displays it on the user interface.

[1511] (Output) Live camera footage displayed in a format that can be viewed by the user.

[1512] Step 16:

[1513] The server periodically checks the weather forecast API to obtain the latest weather information.

[1514] (Input) Periodic calls to a weather forecast API.

[1515] (Processing) The server periodically calls the weather forecast API to obtain new weather information.

[1516] (Output) The latest weather information obtained.

[1517] Step 17:

[1518] When there are important updates, the server creates and sends push notifications to the relevant users.

[1519] (Input) The latest weather information obtained.

[1520] (Processing) The server generates a push notification based on important update information and sends it to the relevant users.

[1521] (Output) Sent push notification.

[1522] Step 18:

[1523] The device receives push notifications and displays them in the user interface.

[1524] (Input) Push notification sent from the server.

[1525] (Processing) The terminal analyzes the notification and displays it on the user interface.

[1526] (Output) Notification information displayed in a format that the user can view.

[1527] (Application Example 1)

[1528] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1529] A challenge for travelers is that they often need to individually check numerous information sources to obtain necessary information in a timely manner at their destination. Furthermore, when using autonomous vehicles, it is essential to be able to easily and appropriately access real-time information that changes during travel. In this situation, a system is needed that centrally provides tourist information, weather forecasts, and traffic information to improve traveler convenience. Additionally, a function that provides timely notifications of real-time information such as traffic conditions and event information is also crucial.

[1530] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1531] In this invention, the server includes means for collecting information about the travel destination from local residents, means for obtaining weather information from an external weather forecast API, means for generating clothing recommendations based on the obtained weather information, means for presenting the weather information and clothing recommendations to the traveler, means for providing live camera footage to check local conditions, means for displaying local information about the travel destination and transit points on an in-vehicle interface, and means for providing push notifications that provide real-time traffic conditions and event information. As a result, travelers can obtain diverse information about their travel destination in one place and acquire the latest information in real time, even while traveling in an autonomous vehicle.

[1532] "Travel destination information" refers to detailed local information about the region the traveler will be visiting, such as tourist attractions, restaurants, and festivals.

[1533] "Local residents" are people who permanently live in the travel destination area and provide the latest local information and recommended spots.

[1534] An "external weather forecast API" is an external application programming interface that provides weather forecasts through a web service.

[1535] "Weather information" refers to meteorological data such as temperature, rainfall, and wind speed for a specified area.

[1536] "Clothing recommendations" are suggestions based on weather information to help users choose appropriate clothing.

[1537] "Live camera footage" refers to video streaming of real-time conditions in a designated area through cameras installed within that area.

[1538] An "in-vehicle interface" refers to devices such as displays and touchscreens installed inside an autonomous vehicle.

[1539] "Push notifications" are a system that automatically sends notifications to the user's device when there is new information or an important update.

[1540] "Real-time traffic conditions" refers to highly timely information about traffic, such as current road congestion and accident information.

[1541] "Event information" refers to information about festivals, exhibitions, special displays, and other events held at your travel destination.

[1542] The system for implementing this invention is realized as follows: The system is an information provision device for autonomous vehicles equipped with features such as providing travel destination information, generating weather forecasts and clothing recommendations, providing live camera footage, and a push notification function. The system operates in cooperation with a server, an in-vehicle interface, and user input information.

[1543] Program Overview

[1544] Each function of the system operates as follows:

[1545] 1. Input and collection of travel destination information

[1546] The user enters their travel destination using the in-car interface. This involves entering GPS information and the name of the destination city.

[1547] The in-vehicle interface sends the specified travel destination information to the server in HTTP request format.

[1548] The server retrieves tourist information, restaurant information, and other data collected from local residents from a database and displays it on the in-car interface.

[1549] 2. Provision of weather forecasts and clothing recommendations.

[1550] Users request weather information for their travel destination and their current location.

[1551] Upon receiving a request from the in-vehicle interface, the server uses an external weather forecast API to retrieve weather data.

[1552] The server generates recommendation information indicating appropriate clothing based on the acquired weather information and sends it back to the in-vehicle interface.

[1553] For example, if the temperature is below 10 degrees Celsius, recommendations such as "You need a warm jacket" will be provided, and if it's above 20 degrees Celsius, "Light clothing is fine" will be offered.

[1554] 3. Provision of live camera footage

[1555] Users can request live camera footage from their travel destination or transit point.

[1556] The server retrieves the URL of the live camera for the relevant region from the database and provides it to the in-vehicle interface.

[1557] The in-vehicle interface uses the URL of the live camera to retrieve the video stream and display it to the user.

[1558] 4. Push notifications for real-time information

[1559] The server periodically checks the weather forecast API to see if there is any new information.

[1560] When there are updates to important weather forecasts, traffic conditions, or event information, the server sends the information to the in-vehicle interface via push notifications.

[1561] The in-car interface receives push notifications and displays them on the user interface, providing users with important information in real time while they are traveling.

[1562] Hardware / software to use

[1563] Hardware:

[1564] In-vehicle interface display: Installed inside the vehicle, it facilitates information exchange between the user and the system.

[1565] GPS module: Provides location information.

[1566] Internet connection module: Enables communication for sending requests to external APIs and servers.

[1567] software:

[1568] Node.js: A JavaScript runtime for running server-side programs.

[1569] Express: Used as a web application framework to handle HTTP requests.

[1570] External weather forecast API: An external service that provides weather information for a specified region.

[1571] Database: Stores and manages travel information and live camera URLs collected from local residents.

[1572] Examples of specific cases and prompt statements

[1573] Specific example: When a user enters "Kyoto City" into the in-car interface, the system provides tourist attractions, weather forecasts, and live camera footage of Kyoto City. If the weather is cold, a clothing recommendation such as "You need a warm jacket" will also be displayed.

[1574] Prompt messages: "Please provide the weather forecast and clothing recommendations for Kyoto City.", "Display live camera footage of Kyoto City."

[1575] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1576] Step 1:

[1577] The user enters information about their travel destination into the in-car interface.

[1578] Input: The user enters the name of their travel destination into the interface.

[1579] Operation: The in-vehicle interface receives the entered travel destination information and sends it to the server in the form of an HTTP request.

[1580] Output: The name of the travel destination is sent as a request to the server.

[1581] Step 2:

[1582] The server retrieves local information.

[1583] Input: Travel destination information sent by the user arrives at the server.

[1584] Operation: Based on the travel destination information, the server retrieves relevant local information (tourist attractions, restaurant information, etc.) from the database.

[1585] Output: The acquired local information is sent back to the in-vehicle interface.

[1586] Step 3:

[1587] The in-vehicle interface displays local information.

[1588] Input: Receive local information returned from the server.

[1589] Operation: The in-vehicle interface displays the received local information on the user interface.

[1590] Output: Local information (tourist attractions, restaurant information, etc.) is displayed to the user.

[1591] Step 4:

[1592] The user requests weather information and clothing recommendations.

[1593] Input: The user interacts with the interface to enter a request for weather information.

[1594] Operation: The in-vehicle interface sends weather information requests to the server in HTTP request format.

[1595] Output: Geographic information of the travel destination is sent to the server as a weather information request.

[1596] Step 5:

[1597] The server retrieves weather information and generates clothing recommendations.

[1598] Input: Receive weather information requests sent by users.

[1599] Operation: The server uses an external weather forecast API to obtain weather information and generates clothing recommendations based on that information. For example, if the temperature is low, it will generate a recommendation such as "You need a warm jacket."

[1600] Output: Weather information and clothing recommendations are sent back to the in-vehicle interface.

[1601] Step 6:

[1602] The in-car interface displays weather information and clothing recommendations.

[1603] Input: Receive weather information and clothing recommendations sent from the server.

[1604] Operation: The in-vehicle interface displays received weather information and clothing recommendations on the user interface.

[1605] Output: The user is shown weather information and clothing recommendations.

[1606] Step 7:

[1607] A user requests live camera footage.

[1608] Input: The user interacts with the interface to submit a request for live camera footage.

[1609] Operation: The in-vehicle interface sends live camera video requests to the server in HTTP request format.

[1610] Output: Geographic information of the travel destination is sent to the server as a request for live camera footage.

[1611] Step 8:

[1612] The server retrieves the URL of the live camera and provides the video feed.

[1613] Input: Receive requests for live camera footage sent by users.

[1614] Operation: The server retrieves the URL of the live camera in the relevant area from the database and provides it to the in-vehicle interface.

[1615] Output: The URL of the live camera is sent back to the in-vehicle interface.

[1616] Step 9:

[1617] The in-vehicle interface displays live camera footage.

[1618] Input: Receive the URL of the live camera sent from the server.

[1619] Operation: The in-vehicle interface uses the live camera's URL to retrieve the video stream and displays it on the user interface.

[1620] Output: The user will see the live camera feed.

[1621] Step 10:

[1622] The server automatically checks weather forecasts and traffic information and sends push notifications.

[1623] Input: Requests to weather forecast APIs and traffic information APIs that are executed automatically at regular intervals.

[1624] Operation: The server periodically checks external APIs and retrieves any important updates. If important information is retrieved, it is sent to the in-vehicle interface in the form of a push notification.

[1625] Output: Updated weather forecasts and traffic information are pushed to the in-car interface.

[1626] Step 11:

[1627] The in-car interface displays push notifications.

[1628] Input: Receive push notifications sent from the server.

[1629] Operation: The in-vehicle interface displays push notifications on the user interface to inform the user of important information.

[1630] Output: The user will be shown important updates to weather forecasts and traffic information.

[1631] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1632] This invention provides travel destination information, weather forecasts, clothing recommendations, and live camera footage, as well as a system that recognizes user emotions and customizes information based on those emotions. This system is implemented using a server, terminals, and an emotion engine.

[1633] Forms of information gathering

[1634] The user enters the region they are planning to travel to via their device. For example, if the user is planning a trip to Kyoto City, they would specify Kyoto City as their destination.

[1635] The terminal sends the travel destination information entered by the user to the server. This request is formatted as an HTTP request. The server receives this request and retrieves local information for the specified region from its database.

[1636] The server retrieves the latest information from its database, including local tourist attractions, restaurant information, and information provided by local residents, and sends it back to the terminal. For example, this includes information about tourist attractions and festivals in Kyoto City.

[1637] Generation methods for weather forecasts and clothing recommendations

[1638] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[1639] The terminal forwards the user's request to the server, sending an HTTP request that includes geographical information about the travel destination. The server receives this request and retrieves weather information for the specified region using an external weather forecast API.

[1640] The server generates clothing recommendations for the user based on the acquired weather information. For example, if the forecast for Kyoto City is cold, it will respond to the device with a recommendation such as, "You will need a warm jacket."

[1641] Forms of emotional engines

[1642] To recognize the user's emotions, the device is equipped with a camera and microphone, which are used to analyze the user's facial expressions and tone of voice. The emotion engine analyzes this data to recognize the user's emotional state.

[1643] The device sends recognized emotion information to the server. The server uses this information to customize weather information and clothing recommendations. For example, if the user is feeling stressed, it provides information on tourist spots suitable for relaxation and clothing recommendations accordingly.

[1644] How to view live camera footage

[1645] Users can request live camera footage to check the local situation in real time.

[1646] The device sends a request for live camera footage to the server. The server retrieves the URL of the appropriate live camera from its database and sends it back to the device.

[1647] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface. This allows the user to check the local situation in real time.

[1648] Update and notification methods

[1649] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[1650] When an important update is detected, the server will send a push notification to the relevant users.

[1651] The device receives push notifications and displays them on the user interface. This allows users to receive real-time updates on weather conditions and other important information, which they can then incorporate into their travel plans.

[1652] As a concrete example, a user is planning a trip to Kyoto City and will use the following functions during that process.

[1653] The user enters "Kyoto City" and retrieves local information.

[1654] The server retrieves information from the database and sends a response to the terminal.

[1655] The user requests a weather forecast and clothing recommendations, and the server retrieves weather information from an external API and sends appropriate clothing advice to the user's device.

[1656] Users can view live camera footage and get real-time information about the location.

[1657] The server checks for weather forecast updates and sends push notifications to the device, providing users with the latest information.

[1658] The emotion engine recognizes the user's emotions, and the server provides customized information based on that data.

[1659] This allows users to obtain all the necessary information in one place, making travel planning smoother. Furthermore, it enables personalized information delivery tailored to the user's emotional state, resulting in a more satisfying travel experience.

[1660] The following describes the processing flow.

[1661] Step 1:

[1662] The user starts up their device and enters the destination they want to go to (for example, "Kyoto City").

[1663] Step 2:

[1664] The terminal receives user input and formats that information as a request to the server.

[1665] Step 3:

[1666] The device sends a formatted request to the server. This request asks for local information about the travel destination.

[1667] Step 4:

[1668] The server receives the request and executes a query against the database to retrieve local information about "Kyoto City".

[1669] Step 5:

[1670] The server retrieves travel destination information related to "Kyoto City" from the database (such as tourist spots, restaurant information, and the latest information from local residents).

[1671] Step 6:

[1672] The server formats the acquired information into JSON format and sends it back to the terminal.

[1673] Step 7:

[1674] The terminal receives information returned from the server and displays it on the user interface. This allows the user to understand basic information about their travel destination.

[1675] Step 8:

[1676] If a user wants to know the weather in Kyoto City on a specific day, they send that request to the server via their device.

[1677] Step 9:

[1678] The device forwards the user's weather request to the server. This request includes local information and the date.

[1679] Step 10:

[1680] Based on the request received by the server, it queries an external weather forecast API for weather information.

[1681] Step 11:

[1682] The server receives weather information returned from an external weather forecast API and generates appropriate clothing recommendations based on the acquired weather information. For example, if the weather forecast is for cold weather, it will generate a recommendation such as "You need a warm jacket."

[1683] Step 12:

[1684] The server formats the weather information and generated clothing recommendations into JSON format and sends them back to the terminal.

[1685] Step 13:

[1686] The device displays weather information and clothing recommendations received by the user interface. The user can use this information to prepare for their trip.

[1687] Step 14:

[1688] If a user wants to view live camera footage from the location, they send a request to the server via their device.

[1689] Step 15:

[1690] The device forwards the live camera request to the server.

[1691] Step 16:

[1692] The server retrieves the URL of the live camera feed for "Kyoto City" from the database and sends it back to the terminal.

[1693] Step 17:

[1694] The device retrieves the video stream based on the live camera's URL and displays it on the user interface. This allows users to check the local situation in real time.

[1695] Step 18:

[1696] The system collects emotional data (such as facial expressions and voice tone) from the user through the camera and microphone built into their device.

[1697] Step 19:

[1698] The device sends collected emotional data to an emotion engine, which then recognizes the user's emotional state.

[1699] Step 20:

[1700] The emotion engine analyzes the user's emotions and sends the results back to the device.

[1701] Step 21:

[1702] The device sends recognized emotion information to the server.

[1703] Step 22:

[1704] The server customizes weather information, clothing recommendations, and other information based on emotional data. For example, if a user is feeling stressed, it will provide information on tourist spots and cafes suitable for relaxation.

[1705] Step 23:

[1706] The server formats the customized information into JSON format and sends it back to the terminal.

[1707] Step 24:

[1708] The device displays the customized information it receives in the user interface. The user can use this information to optimize their travel plan.

[1709] Step 25:

[1710] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[1711] Step 26:

[1712] When the server detects important weather updates, it sends that information as a push notification to the relevant users.

[1713] Step 27:

[1714] The device receives push notifications and displays them on the user interface. This allows users to receive the latest weather information in real time and respond quickly.

[1715] (Example 2)

[1716] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1717] There is a need for a system that allows users planning trips to efficiently obtain information about their destination and receive recommendations for appropriate clothing based on weather information. However, in conventional systems, the information is scattered and not provided in a centralized manner. Furthermore, there is a problem in that information is not customized based on the user's emotional state, making it difficult to provide personalized information. In addition, the provision of live camera footage to check real-time conditions at the destination is not integrated, so users face the inconvenience of gathering information from multiple sources.

[1718] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1719] In this invention, the server includes means for collecting travel destination information from local residents, means for obtaining weather information from an external weather forecast API, and means for generating clothing recommendations based on the acquired weather information. This allows users to centrally obtain travel destination information and receive appropriate clothing recommendations based on weather information. The invention also includes means for providing live camera footage to check local conditions, and an emotion engine that recognizes and analyzes the user's emotions using the terminal's camera and microphone, and customizes weather information and clothing recommendations based on the recognized user emotion information. This enables personalized information provision according to the user's emotional state, not only allowing travel planning to proceed more smoothly but also providing a more satisfying travel experience.

[1720] "Travel destination information" refers to detailed information about tourist attractions, restaurants, events, festivals, and other information related to the region where the user is planning to travel.

[1721] "Local residents" refer to people who actually live in the area where the user is planning to travel, and the information they provide is the most up-to-date local information.

[1722] An "external weather forecast API" is an application programming interface for a service that provides weather forecast information via the internet.

[1723] "Weather information" refers to information about current and future weather conditions (temperature, precipitation, wind speed, etc.) for a specific region.

[1724] "Clothing recommendations" refer to information that suggests appropriate clothing to the user based on acquired weather information.

[1725] A "traveler" is a user who is planning a trip.

[1726] "Live camera footage" refers to a video stream that captures real-time footage of the local situation in a specific area and distributes it via the internet.

[1727] A "device" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[1728] A "database" is a collection of data used to systematically store and manage information, and it is accessible from a server.

[1729] "User interface" refers to the screens and operating methods that serve as the point of contact between a system and a user when inputting or outputting information.

[1730] An "HTTP request" is a communication protocol used to request the transmission or reception of information from a server over the internet.

[1731] An "emotion engine" is software that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

[1732] "Push notifications" are a technology that allows a server to send information to a user's device in real time.

[1733] This invention provides travel destination information, weather forecasts, clothing recommendations, and live camera footage, as well as a system that recognizes user emotions and customizes information based on those emotions. This system is implemented using a server, terminals, and an emotion engine.

[1734] Forms of information gathering

[1735] The user enters the region they are planning to travel to via their device. For example, they might enter "Kyoto City".

[1736] The terminal sends the travel destination information entered by the user to the server as an HTTP request. The server receives this request and retrieves local information for the specified region from its database.

[1737] The server retrieves the latest information from its database, including local tourist attractions, restaurant information, and information provided by local residents, and sends it back to the terminal. For example, this includes information about tourist attractions and festivals in Kyoto City.

[1738] Example of a prompt:

[1739] Please enter "Kyoto City" as your travel destination.

[1740] Generation methods for weather forecasts and clothing recommendations

[1741] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[1742] The terminal forwards the user's request to the server, sending an HTTP request that includes geographical information about the travel destination.

[1743] The server receives this request and uses an external weather forecast API to retrieve weather information for the specified area. Based on the retrieved weather information, it generates clothing recommendations suitable for the user. For example, if the forecast for Kyoto City is cold, it will return a recommendation to the device such as, "You will need a warm jacket."

[1744] Example of a prompt:

[1745] Could you please tell me the weather forecast for Kyoto City?

[1746] Forms of emotional engines

[1747] To recognize the user's emotions, the device is equipped with a camera and microphone, which are used to analyze the user's facial expressions and tone of voice.

[1748] The emotion engine analyzes this data to recognize the user's emotional state.

[1749] The device sends the recognized emotion information to the server.

[1750] The server uses this information to customize weather forecasts and clothing recommendations. For example, if a user is feeling stressed, it will provide information on relaxing tourist spots and appropriate clothing recommendations.

[1751] Example of a prompt:

[1752] Please recommend some places based on your current emotional state.

[1753] How to view live camera footage

[1754] Users request live camera footage to check the local situation in real time.

[1755] The device sends a request for live camera footage to the server.

[1756] The server retrieves the URL of the appropriate live camera from the database and sends it back to the terminal.

[1757] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface. This allows the user to check the local situation in real time.

[1758] Example of a prompt:

[1759] Please show me the live camera footage of Kyoto City.

[1760] Update and notification methods

[1761] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[1762] When an important update is detected, the server will send a push notification to the relevant users.

[1763] The device receives push notifications and displays them on the user interface. This allows users to receive real-time updates on weather conditions and other important information, which they can then incorporate into their travel plans.

[1764] Example of a prompt:

[1765] Please provide real-time notifications of the latest weather forecast for Kyoto City.

[1766] This system allows users to efficiently receive a series of pieces of information and plan a comfortable trip. Furthermore, it enables personalized information delivery tailored to the user's emotional state, resulting in a highly satisfying travel experience.

[1767] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1768] Forms of information gathering

[1769] Step 1:

[1770] The user enters the region they are planning to travel to via their device. For example, they might enter "Kyoto City".

[1771] Input: Travel destination (e.g., "Kyoto City")

[1772] Output: Entered travel destination information

[1773] Step 2:

[1774] The terminal formats the travel destination information entered by the user as an HTTP request.

[1775] Input: Entered travel destination information

[1776] Output: HTTP request containing travel destination information

[1777] Step 3:

[1778] The device sends travel destination information to the server as an HTTP request.

[1779] Input: HTTP request containing travel destination information

[1780] Output: Sending a request to the server

[1781] Step 4:

[1782] The server receives this request and retrieves local information for the specified region from the database.

[1783] Input: HTTP request containing travel destination information

[1784] Output: Local information (e.g., tourist spots, restaurant information, etc.)

[1785] Step 5:

[1786] The server returns the retrieved information to the terminal in JSON format.

[1787] Input: Local information

[1788] Output: HTTP response for the terminal

[1789] Step 6:

[1790] The terminal parses the received JSON data and displays it in the user interface.

[1791] Input: Response from server

[1792] Output: Local information displayed in the user interface

[1793] Generation methods for weather forecasts and clothing recommendations

[1794] Step 1:

[1795] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[1796] Input: Weather information request

[1797] Output: Request for terminal

[1798] Step 2:

[1799] The device generates an HTTP request containing geographical information about the travel destination and sends it to the server.

[1800] Input: Weather information request

[1801] Output: HTTP request containing geographical information

[1802] Step 3:

[1803] The server receives this request and uses an external weather forecast API to retrieve weather information for the specified area.

[1804] Input: HTTP request containing geographical information

[1805] Output: Weather information

[1806] Step 4:

[1807] The server generates clothing recommendations suitable for the user based on the acquired weather information.

[1808] Input: Weather information

[1809] Output: Clothing recommendation (e.g., "You need a warm jacket")

[1810] Step 5:

[1811] The server returns the generated recommendations to the device.

[1812] Input: Clothing Recommendation

[1813] Output: HTTP response for the terminal

[1814] Step 6:

[1815] The device displays the received clothing recommendations in the user interface.

[1816] Input: Response from server

[1817] Output: Clothing recommendations displayed in the user interface

[1818] Forms of emotional engines

[1819] Step 1:

[1820] The device is equipped with a camera and microphone to collect the user's facial expressions and voice tone.

[1821] Input: User's facial expressions and tone of voice

[1822] Output: Collected data

[1823] Step 2:

[1824] The emotion engine analyzes this data to recognize the user's emotional state.

[1825] Input: Collected data

[1826] Output: Recognized emotional state

[1827] Step 3:

[1828] The device sends the recognized emotion information to the server.

[1829] Input: Recognized emotional state

[1830] Output: HTTP request to the server

[1831] Step 4:

[1832] The server uses this information to customize weather forecasts and clothing recommendations.

[1833] Input: Emotional information

[1834] Output: Customized weather information and clothing recommendations

[1835] Step 5:

[1836] The server returns customized information to the terminal.

[1837] Input: Customized information

[1838] Output: HTTP response for the terminal

[1839] How to view live camera footage

[1840] Step 1:

[1841] Users request live camera footage to check the local situation in real time.

[1842] Input: Live camera video request

[1843] Output: Request for terminal

[1844] Step 2:

[1845] The device sends a request for live camera footage to the server.

[1846] Input: Live camera video request

[1847] Output: HTTP request to the server

[1848] Step 3:

[1849] The server retrieves the URL of the appropriate live camera from the database and sends it back to the terminal.

[1850] Input: Live camera video request

[1851] Output: URL of the live camera

[1852] Step 4:

[1853] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface.

[1854] Input: URL of the live camera

[1855] Output: Video stream displayed on the user interface

[1856] Update and notification methods

[1857] Step 1:

[1858] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[1859] Input: Periodic check request

[1860] Output: Latest weather information

[1861] Step 2:

[1862] When an important update is detected, the server will send a push notification to the relevant users.

[1863] Input: Latest weather information

[1864] Output: Push notification

[1865] Step 3:

[1866] The device receives push notifications and displays them on the user interface.

[1867] Input: Push notification

[1868] Output: Notification displayed in the user interface

[1869] Through the above processing steps, users can efficiently obtain information about their travel destination and receive appropriate advice based on the latest information in real time. Furthermore, by providing customized information that responds to the user's emotions, a highly satisfying travel experience can be achieved.

[1870] (Application Example 2)

[1871] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1872] Traditional food delivery systems simply deliver the food ordered by the user, but they do not adequately provide information such as travel destination details, weather, live camera footage, or personalized information based on the user's emotions. Furthermore, features such as delivery time predictions based on weather information in the delivery area and real-time delivery status checks are lacking. This has resulted in a lack of an environment where users can utilize a more comfortable and efficient service, which is a significant challenge.

[1873] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for collecting travel destination information from local residents, means for obtaining weather information from an external weather forecast API, means for generating clothing recommendations based on the acquired weather information, means for presenting weather information and clothing recommendations to travelers, means for recognizing the user's emotions and customizing information based on those emotions, means for providing live camera footage to check local conditions, means for recommending appropriate food based on the user's emotions, means for predicting delivery times based on weather information in the delivery area and notifying the user, and means for checking the delivery status in real time. This makes it possible to provide personalized information, improve delivery efficiency, and enhance the user experience, which were challenges in the past.

[1874] "Travel destination information" refers to local information about the travel destination the user is planning, such as tourist spots, restaurants, and festivals.

[1875] "Local residents" are people who live in the destination city and have the role of providing local information.

[1876] An "external weather forecast API" is an API (Application Programming Interface) for an external service that provides weather information via the internet.

[1877] "Clothing recommendations" are systems that suggest the most suitable clothing for the user based on acquired weather information.

[1878] "User emotions" refer to the mental state perceived from the user's facial expressions, tone of voice, and other similar factors.

[1879] "Means of recognizing emotions" refers to technologies that use cameras and microphones to analyze a user's emotional state.

[1880] "Customizing information" means appropriately changing the information provided according to the user's emotions and circumstances.

[1881] "Local conditions" refers to the real-time environment and events at the travel destination.

[1882] "Live camera footage" refers to video that streams the situation at the location in real time.

[1883] "Food recommendations" refer to recommending appropriate food items based on the user's emotional state and local information.

[1884] "Delivery area" refers to the region or area where food delivery services are provided.

[1885] "Predicting delivery time" means estimating the time required for delivery based on factors such as weather and traffic conditions.

[1886] "Checking delivery status in real time" means being able to instantly monitor the current location and progress of the delivery driver.

[1887] A "server" is a central computer system that processes data and provides information to users.

[1888] This document describes a system designed to provide personalized information and streamline delivery for travelers using food delivery services. This system primarily consists of a server, terminals, and an emotion engine.

[1889] Hardware and software configuration

[1890] The following hardware and software are used to implement this system:

[1891] Hardware:

[1892] User devices (smartphones, tablets)

[1893] server

[1894] Delivery driver's device (smartphone)

[1895] Camera (for live camera)

[1896] software:

[1897] Mobile applications (iOS / Android)

[1898] Web server (Node.js)

[1899] Database (MySQL)

[1900] Weather forecast APIs (such as OpenWeatherMap)

[1901] Emotion recognition engines (Amazon Rekognition, Microsoft Azure Face API, etc.)

[1902] Program processing

[1903] 1. Gathering information about your travel destination:

[1904] The user enters travel destination information from their device. For example, they might specify "Kyoto City" as their travel destination.

[1905] The device sends information about the specified travel destination to the server.

[1906] The server retrieves information collected from local residents from a database and responds to users with information such as tourist attractions and restaurants in their travel destination.

[1907] 2. Weather information and clothing recommendations:

[1908] The user sends a request from their device to the server to get weather information for their travel destination.

[1909] The server uses an external weather forecast API to obtain weather information for the specified region.

[1910] Based on the acquired weather information, the server generates a recommendation for appropriate clothing and sends it back to the device.

[1911] 3. Emotion Recognition and Personalization:

[1912] The device is equipped with a camera and microphone, which are used to analyze the user's facial expressions and voice tone.

[1913] The emotion engine analyzes this data to recognize the user's emotional state.

[1914] Based on the recognized emotional information, the server customizes the information to provide appropriate food recommendations and special messages.

[1915] 4. Weather and delivery time forecast for the delivery area:

[1916] The server uses geographical information of the delivery area to obtain weather information from a weather forecast API.

[1917] Based on weather information, the server predicts the delivery time and notifies the terminal.

[1918] 5. Provision of live camera footage:

[1919] Users request live camera footage to check the situation on site.

[1920] The server retrieves the appropriate live camera URL from the database and sends it back to the terminal.

[1921] The device displays live camera footage to the user in real time.

[1922] 6. Real-time delivery status check:

[1923] Obtain the delivery driver's current location information.

[1924] The server uses this information to notify the user of the delivery status in real time.

[1925] Examples of specific cases and prompt statements

[1926] As a concrete example, consider a case where the user is tired. When the user orders food delivery using their device, the emotion engine recognizes the user's fatigue level and recommends an "energy drink to give them a boost." Furthermore, if it's raining in the delivery area, it recommends a "warm soup."

[1927] Example of a prompt:

[1928] "When a user is tired, the app should recommend a nutritious energy drink and allow them to check the delivery driver's location via live video. Also, on rainy days, it should recommend a warm soup."

[1929] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1930] Step 1:

[1931] Gathering information about your travel destination

[1932] Input: The user enters travel destination information into the terminal.

[1933] Specific operation: The user opens an application on their device and enters a location, such as "Kyoto City," as their travel destination. The device then sends this information to the server.

[1934] Data processing: The server retrieves and searches information collected from local residents in the database.

[1935] Output: The server retrieves information on tourist spots and restaurants in Kyoto City and sends it to the terminal.

[1936] Step 2:

[1937] Obtaining weather information and clothing recommendations

[1938] Input: The user requests weather information.

[1939] Specific operation: The user sends a request from their device to the server to get weather information for their travel destination. The server uses an external weather forecast API to retrieve weather information for the specified area.

[1940] Data processing: The server analyzes the acquired weather information and generates clothing recommendations based on the weather.

[1941] Output: The server returns the generated clothing recommendations to the terminal.

[1942] Step 3:

[1943] emotion recognition

[1944] Input: User's camera video and audio data.

[1945] Specific operation: The device is equipped with a camera and microphone, which are used to capture the user's facial expressions and voice tone. The captured data is then sent to an emotion recognition engine.

[1946] Data processing: The emotion recognition engine analyzes facial expressions and voice tone to determine the user's emotional state.

[1947] Output: Sends recognized emotion data to the server.

[1948] Step 4:

[1949] Emotion-based information customization

[1950] Input: Emotion recognition data.

[1951] Specific operation: The server receives emotion recognition data and generates information (such as special food recommendations or messages) that corresponds to the user's emotional state.

[1952] Data processing: The server selects and customizes appropriate information from the database based on the emotional state.

[1953] Output: Sends customized information to the device.

[1954] Step 5:

[1955] Weather information and delivery time forecast for the delivery area.

[1956] Input: Geographical information of the delivery area.

[1957] Specific operation: The server uses geographical information of the delivery area to obtain weather information from an external weather forecast API. Based on this, the server predicts the delivery time.

[1958] Data processing: Using an algorithm based on weather information, delivery times are predicted.

[1959] Output: Notifies the device of the estimated delivery time.

[1960] Step 6:

[1961] Live camera footage provided

[1962] Input: User request for live camera footage.

[1963] Specific operation: The user requests live camera footage through their device. The server retrieves the appropriate live camera URL from the database and sends it back to the device.

[1964] Data processing: The server selects the appropriate live camera URL and returns it.

[1965] Output: The terminal displays live camera footage to the user in real time.

[1966] Step 7:

[1967] Real-time delivery status check

[1968] Input: Delivery driver's current location information.

[1969] Specific operation: The server obtains location information from the delivery driver's terminal. The server uses this information to notify the user of the delivery status in real time.

[1970] Data processing: Analyze delivery progress based on current location information.

[1971] Output: Notifies the terminal of the real-time delivery status.

[1972] Through these steps, users can receive information about their travel destination, weather, live camera footage, and personalized information tailored to their emotions, allowing them to enjoy a convenient food delivery service.

[1973] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1974] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1975] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1976] [Fourth Embodiment]

[1977] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1978] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1979] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1980] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1981] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1982] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1983] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1984] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1985] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1986] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1987] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1988] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1989] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1990] This invention is a system that comprehensively provides information such as travel destination information, weather forecasts, clothing recommendations, and live camera footage. This system operates through the cooperation of a server and a terminal (user's device).

[1991] Forms of information gathering

[1992] The user enters the region they are planning to travel to via their device. For example, if the user is planning a trip to Kyoto City, they would specify Kyoto City as their destination.

[1993] The terminal sends the travel destination information entered by the user to the server. This request is formatted as an HTTP request. The server receives this request and retrieves local information for the specified region from its database.

[1994] The server retrieves the latest information from its database, including local tourist attractions, restaurant information, and information provided by local residents, and sends it back to the terminal. For example, this includes information about tourist attractions and festivals in Kyoto City.

[1995] Generation methods for weather forecasts and clothing recommendations

[1996] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[1997] The terminal forwards the user's request to the server, sending an HTTP request that includes geographical information about the travel destination. The server receives this request and retrieves weather information for the specified region using an external weather forecast API.

[1998] The server generates clothing recommendations for the user based on the acquired weather information. For example, if the forecast for Kyoto City is cold, it will respond to the device with a recommendation such as, "You will need a warm jacket."

[1999] How to view live camera footage

[2000] Users can request live camera footage to check the local situation in real time.

[2001] The device sends a request for live camera footage to the server. The server retrieves the URL of the appropriate live camera from its database and sends it back to the device.

[2002] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface. This allows the user to check the local situation in real time.

[2003] Update and notification methods

[2004] The server periodically checks the weather forecast API and retrieves any updated weather information. If an important update is detected, it sends a push notification to the relevant users.

[2005] The device receives push notifications and displays them on the user interface. This allows users to receive real-time updates on weather conditions and other important information, which they can then incorporate into their travel plans.

[2006] As a concrete example, a user is planning a trip to Kyoto City and will use the following functions during that process.

[2007] The user enters "Kyoto City" and retrieves local information.

[2008] The server retrieves information from the database and sends a response to the terminal.

[2009] The user requests a weather forecast and clothing recommendations, and the server retrieves weather information from an external API and sends appropriate clothing advice to the user's device.

[2010] Users can view live camera footage and get real-time information about the location.

[2011] The server checks for weather forecast updates and sends push notifications to the device, providing users with the latest information.

[2012] This allows users to obtain all the necessary information in one place, making travel planning much smoother.

[2013] The following describes the processing flow.

[2014] Step 1:

[2015] The user starts up their device and enters the destination they want to go to (for example, "Kyoto City").

[2016] Step 2:

[2017] The terminal receives user input and formats that information as a request to the server.

[2018] Step 3:

[2019] The device sends a formatted request to the server. This request asks for local information about the travel destination.

[2020] Step 4:

[2021] The server receives the request and executes a query against the database to retrieve local information about "Kyoto City".

[2022] Step 5:

[2023] The server retrieves travel destination information related to "Kyoto City" from the database (such as tourist spots, restaurant information, and the latest information from local residents).

[2024] Step 6:

[2025] The server formats the acquired information into JSON format and sends it back to the terminal.

[2026] Step 7:

[2027] The terminal receives information returned from the server and displays it on the user interface. This allows the user to understand basic information about their travel destination.

[2028] Step 8:

[2029] If a user wants to know the weather in Kyoto City on a specific day, they send that request to the server via their device.

[2030] Step 9:

[2031] The device forwards the user's weather request to the server. This request includes local information and the date.

[2032] Step 10:

[2033] Based on the request received by the server, it queries an external weather forecast API for weather information.

[2034] Step 11:

[2035] The server receives weather information returned from an external weather forecast API and generates appropriate clothing recommendations based on the acquired weather information. For example, if the weather forecast is for cold weather, it will generate a recommendation such as "You need a warm jacket."

[2036] Step 12:

[2037] The server formats the weather information and generated clothing recommendations into JSON format and sends them back to the terminal.

[2038] Step 13:

[2039] The device displays weather information and clothing recommendations received by the user interface. The user can use this information to prepare for their trip.

[2040] Step 14:

[2041] If a user wants to view live camera footage from the location, they send a request to the server via their device.

[2042] Step 15:

[2043] The device forwards the live camera request to the server.

[2044] Step 16:

[2045] The server retrieves the URL of the live camera feed for "Kyoto City" from the database and sends it back to the terminal.

[2046] Step 17:

[2047] The device retrieves the video stream based on the live camera's URL and displays it on the user interface. This allows users to check the local situation in real time.

[2048] Step 18:

[2049] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[2050] Step 19:

[2051] When the server detects important weather updates, it sends that information as a push notification to the relevant users.

[2052] Step 20:

[2053] The device receives push notifications and displays them on the user interface. This allows users to receive the latest weather information in real time and respond quickly.

[2054] (Example 1)

[2055] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2056] Modern travelers have a growing need for detailed information and real-time updates on their destinations. However, obtaining the right information in one place often requires using multiple different applications and websites, which is inconvenient. Furthermore, there is no information system that allows travelers to check weather forecasts and local conditions in real time and respond quickly. As a result, it is difficult for travelers to plan efficient and comfortable trips.

[2057] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[2058] In this invention, the server includes means for collecting information about the travel destination from local residents, means for obtaining weather information from an external weather forecast API, means for generating clothing recommendations based on the obtained weather information, means for receiving regional information about the travel destination entered by the traveler, means for presenting weather information and clothing recommendations to the traveler, means for providing live camera footage to check local conditions, and means for obtaining and providing traveler information on tourist spots and restaurants related to the travel destination from a database. This enables travelers to comprehensively obtain information about their travel destination within a single system and plan their trip efficiently and comfortably.

[2059] "Travel destination information" refers to a general term for tourist attractions, restaurants, events, and other region-specific information related to the area you plan to visit.

[2060] "Local residents" refers to people who permanently live in the travel destination region and can provide up-to-date and specific information about that area.

[2061] An "external weather forecast API" refers to a programmatic interface for obtaining weather forecast data provided by a third party.

[2062] "Clothing recommendations" refer to information that suggests appropriate clothing for travelers based on acquired weather data.

[2063] A "traveler" refers to an individual or group planning a trip to a specific region.

[2064] "Local information" refers collectively to geographical, social, and economic information concerning a specific region.

[2065] A "live camera" refers to a device or system that broadcasts real-time video from a specific location over the internet.

[2066] A "database" refers to a collection of data organized for the purpose of efficiently searching, retrieving, and managing information.

[2067] "Push notification" refers to a communication method in which a server sends information to a user in real time.

[2068] A "user interface" is an interface through which a user interacts with a system, and includes visual or functional elements.

[2069] This invention is a system that comprehensively provides travelers with information such as destination information, weather forecasts, clothing recommendations, and live camera footage. This system operates through the cooperation of a server and a terminal (user's device).

[2070] The user enters the region they are planning to travel to via their device. For example, if the user is planning a trip to Kyoto City, they would enter "Kyoto City" as the destination. This input information becomes the starting point for the system.

[2071] The terminal formats the travel destination information entered by the user as an HTTP request and sends it to the server. This is done over a standard internet connection. When the server receives the request, it accesses its internal database and external weather forecast APIs (e.g., OpenWeatherMap API) to collect the necessary information.

[2072] The server first accesses an internal database to retrieve information on tourist spots, restaurants, and events in a specified area (e.g., Kyoto City). A database called GLDB (General Location Database) can be used for this retrieval. Next, the server accesses an external weather forecast API to obtain the latest weather information. This API access can be done using, for example, an OpenWeatherMap API key.

[2073] Based on the acquired weather information, the server executes a process to generate clothing recommendations suitable for the user. For example, if the forecast for Kyoto City is cold, it will generate a specific recommendation such as "You will need a warm jacket."

[2074] The terminal receives information from the server and displays it in the user interface. This user interface is built using web technologies such as HTML / CSS and JavaScript. Here, users can check information on tourist attractions and restaurants in their travel destination, as well as weather forecasts and clothing recommendations.

[2075] Users can also request live camera footage to check real-time information from the location. The device sends this request to the server, which retrieves the URL of the corresponding live camera from its database and returns it to the device. The device uses the retrieved live camera URL to obtain the video stream and displays it on the user interface.

[2076] Furthermore, the server periodically checks external weather forecast APIs and retrieves any important weather forecast updates. When an important update is detected, the server also has a function to send push notifications to the relevant users. The device receives this push notification and displays it in the user interface, allowing users to obtain the latest information about their travel destination in real time.

[2077] As a concrete example, the following series of operations can be considered.

[2078] 1. The user enters "Kyoto City" and retrieves local information.

[2079] 2. The server retrieves information on tourist spots and restaurants in Kyoto City from the database and sends it back to the terminal.

[2080] 3. The user requests a weather forecast and clothing recommendations.

[2081] 4. The server retrieves weather information from an external API, generates appropriate clothing recommendations, and sends them to the device.

[2082] 5. Users can view live camera footage and understand the real-time situation on site.

[2083] 6. The server checks for weather forecast updates and sends important update information to the device via push notification.

[2084] Examples of prompt statements include:

[2085] "Travel destination information: Kyoto City, weather forecast, clothing recommendations, live cameras"

[2086] It can be done this way.

[2087] The flow of the specific processing in Example 1 will be explained using Figure 11.

[2088] Step 1:

[2089] The user uses their device to enter information about their travel destination (e.g., "Kyoto City").

[2090] (Input) Travel destination information entered by the user.

[2091] (Processing) The terminal formats the user's input as an HTTP request.

[2092] (Output) A formatted HTTP request is generated.

[2093] Step 2:

[2094] The device sends an HTTP request to the server containing the travel destination information entered by the user.

[2095] (Input) A formatted HTTP request.

[2096] (Processing) The terminal sends a request to the server via the internet.

[2097] (Output) The server receives an HTTP request.

[2098] Step 3:

[2099] Based on the received request, the server retrieves tourist spots and restaurant information for the travel destination (e.g., "Kyoto City") from the database.

[2100] (Input) A request containing the user's travel destination information.

[2101] (Processing) The server queries the database to retrieve relevant information.

[2102] (Output) Acquired information on tourist spots and restaurants.

[2103] Step 4:

[2104] The server returns the acquired tourist spot and restaurant information to the terminal.

[2105] (Input) Information retrieved by the server from the database.

[2106] (Processing) The server formats the information and sends it to the terminal as an HTTP response.

[2107] (Output) Formatted HTTP response.

[2108] Step 5:

[2109] The terminal displays information received from the server on the user interface.

[2110] (Input) HTTP response received from the server.

[2111] (Processing) The terminal analyzes the information and displays it on the user interface.

[2112] (Output) Information displayed in a format that the user can view.

[2113] Step 6:

[2114] The user enters information into the device to request weather forecasts and clothing recommendations.

[2115] (Input) Geographical information of your travel destination.

[2116] (Processing) The user enters the necessary information into the terminal.

[2117] (Output) Input weather forecast and clothing request information.

[2118] Step 7:

[2119] The device sends an HTTP request to the server that includes weather forecasts and clothing recommendations.

[2120] (Input) Weather forecast and clothing request information entered by the user.

[2121] (Processing) The terminal formats the information as an HTTP request and sends it to the server.

[2122] (Output) HTTP requests received by the server.

[2123] Step 8:

[2124] The server accesses an external weather forecast API to obtain weather information for a specified area (e.g., "Kyoto City").

[2125] (Input) Regional information included in the HTTP request.

[2126] (Processing) The server calls an external weather forecast API to obtain weather information.

[2127] (Output) Acquired weather information.

[2128] Step 9:

[2129] The server generates clothing recommendations based on the acquired weather information.

[2130] (Input) Obtained weather information.

[2131] (Processing) The server analyzes weather information and executes an algorithm to recommend appropriate clothing.

[2132] (Output) Generated clothing recommendations.

[2133] Step 10:

[2134] The server sends the generated clothing recommendations to the device.

[2135] (Input) Clothing recommendation.

[2136] (Processing) The server formats the clothing recommendation into an HTTP response and sends it to the terminal.

[2137] (Output) Formatted HTTP response.

[2138] Step 11:

[2139] The device displays the received clothing recommendations in the user interface.

[2140] (Input) HTTP response received from the server.

[2141] (Processing) The terminal analyzes the information and displays it on the user interface.

[2142] (Output) Clothing recommendations displayed in a format viewable by the user.

[2143] Step 12:

[2144] Users request live camera footage to check real-time information from the location.

[2145] (Input) Request for live camera footage.

[2146] (Processing) The user enters a request on the terminal.

[2147] (Output) Input live camera request information.

[2148] Step 13:

[2149] The terminal sends a request for live camera footage to the server.

[2150] (Input) Live camera request information.

[2151] (Processing) The terminal formats the information as an HTTP request and sends it to the server.

[2152] (Output) HTTP requests received by the server.

[2153] Step 14:

[2154] The server retrieves the URL of the corresponding live camera from the database and sends it back to the terminal.

[2155] (Input) Regional information included in the request.

[2156] (Processing) The server queries the database to obtain the URL of the live camera.

[2157] (Output) The retrieved live camera URL.

[2158] Step 15:

[2159] The device uses the received URL of the live camera to retrieve the video stream and displays it in the user interface.

[2160] (Input) The URL of the live camera received from the server.

[2161] (Processing) The terminal acquires the video stream from the live camera and displays it on the user interface.

[2162] (Output) Live camera footage displayed in a format that can be viewed by the user.

[2163] Step 16:

[2164] The server periodically checks the weather forecast API to obtain the latest weather information.

[2165] (Input) Periodic calls to a weather forecast API.

[2166] (Processing) The server periodically calls the weather forecast API to obtain new weather information.

[2167] (Output) The latest weather information obtained.

[2168] Step 17:

[2169] When there are important updates, the server creates and sends push notifications to the relevant users.

[2170] (Input) The latest weather information obtained.

[2171] (Processing) The server generates a push notification based on important update information and sends it to the relevant users.

[2172] (Output) Sent push notification.

[2173] Step 18:

[2174] The device receives push notifications and displays them in the user interface.

[2175] (Input) Push notification sent from the server.

[2176] (Processing) The terminal analyzes the notification and displays it on the user interface.

[2177] (Output) Notification information displayed in a format that the user can view.

[2178] (Application Example 1)

[2179] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2180] A challenge for travelers is that they often need to individually check numerous information sources to obtain necessary information in a timely manner at their destination. Furthermore, when using autonomous vehicles, it is essential to be able to easily and appropriately access real-time information that changes during travel. In this situation, a system is needed that centrally provides tourist information, weather forecasts, and traffic information to improve traveler convenience. Additionally, a function that provides timely notifications of real-time information such as traffic conditions and event information is also crucial.

[2181] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[2182] In this invention, the server includes means for collecting information about the travel destination from local residents, means for obtaining weather information from an external weather forecast API, means for generating clothing recommendations based on the obtained weather information, means for presenting the weather information and clothing recommendations to the traveler, means for providing live camera footage to check local conditions, means for displaying local information about the travel destination and transit points on an in-vehicle interface, and means for providing push notifications that provide real-time traffic conditions and event information. As a result, travelers can obtain diverse information about their travel destination in one place and acquire the latest information in real time, even while traveling in an autonomous vehicle.

[2183] "Travel destination information" refers to detailed local information about the region the traveler will be visiting, such as tourist attractions, restaurants, and festivals.

[2184] "Local residents" are people who permanently live in the travel destination area and provide the latest local information and recommended spots.

[2185] An "external weather forecast API" is an external application programming interface that provides weather forecasts through a web service.

[2186] "Weather information" refers to meteorological data such as temperature, rainfall, and wind speed for a specified area.

[2187] "Clothing recommendations" are suggestions based on weather information to help users choose appropriate clothing.

[2188] "Live camera footage" refers to video streaming of real-time conditions in a designated area through cameras installed within that area.

[2189] An "in-vehicle interface" refers to devices such as displays and touchscreens installed inside an autonomous vehicle.

[2190] "Push notifications" are a system that automatically sends notifications to the user's device when there is new information or an important update.

[2191] "Real-time traffic conditions" refers to highly timely information about traffic, such as current road congestion and accident information.

[2192] "Event information" refers to information about festivals, exhibitions, special displays, and other events held at your travel destination.

[2193] The system for implementing this invention is realized as follows: The system is an information provision device for autonomous vehicles equipped with features such as providing travel destination information, generating weather forecasts and clothing recommendations, providing live camera footage, and a push notification function. The system operates in cooperation with a server, an in-vehicle interface, and user input information.

[2194] Program Overview

[2195] Each function of the system operates as follows:

[2196] 1. Input and collection of travel destination information

[2197] The user enters their travel destination using the in-car interface. This involves entering GPS information and the name of the destination city.

[2198] The in-vehicle interface sends the specified travel destination information to the server in HTTP request format.

[2199] The server retrieves tourist information, restaurant information, and other data collected from local residents from a database and displays it on the in-car interface.

[2200] 2. Provision of weather forecasts and clothing recommendations.

[2201] Users request weather information for their travel destination and their current location.

[2202] Upon receiving a request from the in-vehicle interface, the server uses an external weather forecast API to retrieve weather data.

[2203] The server generates recommendation information indicating appropriate clothing based on the acquired weather information and sends it back to the in-vehicle interface.

[2204] For example, if the temperature is below 10 degrees Celsius, recommendations such as "You need a warm jacket" will be provided, and if it's above 20 degrees Celsius, "Light clothing is fine" will be offered.

[2205] 3. Provision of live camera footage

[2206] Users can request live camera footage from their travel destination or transit point.

[2207] The server retrieves the URL of the live camera for the relevant region from the database and provides it to the in-vehicle interface.

[2208] The in-vehicle interface uses the URL of the live camera to retrieve the video stream and display it to the user.

[2209] 4. Push notifications for real-time information

[2210] The server periodically checks the weather forecast API to see if there is any new information.

[2211] When there are updates to important weather forecasts, traffic conditions, or event information, the server sends the information to the in-vehicle interface via push notifications.

[2212] The in-car interface receives push notifications and displays them on the user interface, providing users with important information in real time while they are traveling.

[2213] Hardware / software to use

[2214] Hardware:

[2215] In-vehicle interface display: Installed inside the vehicle, it facilitates information exchange between the user and the system.

[2216] GPS module: Provides location information.

[2217] Internet connection module: Enables communication for sending requests to external APIs and servers.

[2218] software:

[2219] Node.js: A JavaScript runtime for running server-side programs.

[2220] Express: Used as a web application framework to handle HTTP requests.

[2221] External weather forecast API: An external service that provides weather information for a specified region.

[2222] Database: Stores and manages travel information and live camera URLs collected from local residents.

[2223] Examples of specific cases and prompt statements

[2224] Specific example: When a user enters "Kyoto City" into the in-car interface, the system provides tourist attractions, weather forecasts, and live camera footage of Kyoto City. If the weather is cold, a clothing recommendation such as "You need a warm jacket" will also be displayed.

[2225] Prompt messages: "Please provide the weather forecast and clothing recommendations for Kyoto City.", "Display live camera footage of Kyoto City."

[2226] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[2227] Step 1:

[2228] The user enters information about their travel destination into the in-car interface.

[2229] Input: The user enters the name of their travel destination into the interface.

[2230] Operation: The in-vehicle interface receives the entered travel destination information and sends it to the server in the form of an HTTP request.

[2231] Output: The name of the travel destination is sent as a request to the server.

[2232] Step 2:

[2233] The server retrieves local information.

[2234] Input: Travel destination information sent by the user arrives at the server.

[2235] Operation: Based on the travel destination information, the server retrieves relevant local information (tourist attractions, restaurant information, etc.) from the database.

[2236] Output: The acquired local information is sent back to the in-vehicle interface.

[2237] Step 3:

[2238] The in-vehicle interface displays local information.

[2239] Input: Receive local information returned from the server.

[2240] Operation: The in-vehicle interface displays the received local information on the user interface.

[2241] Output: Local information (tourist attractions, restaurant information, etc.) is displayed to the user.

[2242] Step 4:

[2243] The user requests weather information and clothing recommendations.

[2244] Input: The user interacts with the interface to enter a request for weather information.

[2245] Operation: The in-vehicle interface sends weather information requests to the server in HTTP request format.

[2246] Output: Geographic information of the travel destination is sent to the server as a weather information request.

[2247] Step 5:

[2248] The server retrieves weather information and generates clothing recommendations.

[2249] Input: Receive weather information requests sent by users.

[2250] Operation: The server uses an external weather forecast API to obtain weather information and generates clothing recommendations based on that information. For example, if the temperature is low, it will generate a recommendation such as "You need a warm jacket."

[2251] Output: Weather information and clothing recommendations are sent back to the in-vehicle interface.

[2252] Step 6:

[2253] The in-car interface displays weather information and clothing recommendations.

[2254] Input: Receive weather information and clothing recommendations sent from the server.

[2255] Operation: The in-vehicle interface displays received weather information and clothing recommendations on the user interface.

[2256] Output: The user is shown weather information and clothing recommendations.

[2257] Step 7:

[2258] A user requests live camera footage.

[2259] Input: The user interacts with the interface to submit a request for live camera footage.

[2260] Operation: The in-vehicle interface sends live camera video requests to the server in HTTP request format.

[2261] Output: Geographic information of the travel destination is sent to the server as a request for live camera footage.

[2262] Step 8:

[2263] The server retrieves the URL of the live camera and provides the video feed.

[2264] Input: Receive requests for live camera footage sent by users.

[2265] Operation: The server retrieves the URL of the live camera in the relevant area from the database and provides it to the in-vehicle interface.

[2266] Output: The URL of the live camera is sent back to the in-vehicle interface.

[2267] Step 9:

[2268] The in-vehicle interface displays live camera footage.

[2269] Input: Receive the URL of the live camera sent from the server.

[2270] Operation: The in-vehicle interface uses the live camera's URL to retrieve the video stream and displays it on the user interface.

[2271] Output: The user will see the live camera feed.

[2272] Step 10:

[2273] The server automatically checks weather forecasts and traffic information and sends push notifications.

[2274] Input: Requests to weather forecast APIs and traffic information APIs that are executed automatically at regular intervals.

[2275] Operation: The server periodically checks external APIs and retrieves any important updates. If important information is retrieved, it is sent to the in-vehicle interface in the form of a push notification.

[2276] Output: Updated weather forecasts and traffic information are pushed to the in-car interface.

[2277] Step 11:

[2278] The in-car interface displays push notifications.

[2279] Input: Receive push notifications sent from the server.

[2280] Operation: The in-vehicle interface displays push notifications on the user interface to inform the user of important information.

[2281] Output: The user will be shown important updates to weather forecasts and traffic information.

[2282] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[2283] This invention provides travel destination information, weather forecasts, clothing recommendations, and live camera footage, as well as a system that recognizes user emotions and customizes information based on those emotions. This system is implemented using a server, terminals, and an emotion engine.

[2284] Forms of information gathering

[2285] The user enters the region they are planning to travel to via their device. For example, if the user is planning a trip to Kyoto City, they would specify Kyoto City as their destination.

[2286] The terminal sends the travel destination information entered by the user to the server. This request is formatted as an HTTP request. The server receives this request and retrieves local information for the specified region from its database.

[2287] The server retrieves the latest information from its database, including local tourist attractions, restaurant information, and information provided by local residents, and sends it back to the terminal. For example, this includes information about tourist attractions and festivals in Kyoto City.

[2288] Generation methods for weather forecasts and clothing recommendations

[2289] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[2290] The terminal forwards the user's request to the server, sending an HTTP request that includes geographical information about the travel destination. The server receives this request and retrieves weather information for the specified region using an external weather forecast API.

[2291] The server generates clothing recommendations for the user based on the acquired weather information. For example, if the forecast for Kyoto City is cold, it will respond to the device with a recommendation such as, "You will need a warm jacket."

[2292] Forms of emotional engines

[2293] To recognize the user's emotions, the device is equipped with a camera and microphone, which are used to analyze the user's facial expressions and tone of voice. The emotion engine analyzes this data to recognize the user's emotional state.

[2294] The device sends recognized emotion information to the server. The server uses this information to customize weather information and clothing recommendations. For example, if the user is feeling stressed, it provides information on tourist spots suitable for relaxation and clothing recommendations accordingly.

[2295] How to view live camera footage

[2296] Users can request live camera footage to check the local situation in real time.

[2297] The device sends a request for live camera footage to the server. The server retrieves the URL of the appropriate live camera from its database and sends it back to the device.

[2298] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface. This allows the user to check the local situation in real time.

[2299] Update and notification methods

[2300] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[2301] When an important update is detected, the server will send a push notification to the relevant users.

[2302] The device receives push notifications and displays them on the user interface. This allows users to receive real-time updates on weather conditions and other important information, which they can then incorporate into their travel plans.

[2303] As a concrete example, a user is planning a trip to Kyoto City and will use the following functions during that process.

[2304] The user enters "Kyoto City" and retrieves local information.

[2305] The server retrieves information from the database and sends a response to the terminal.

[2306] The user requests a weather forecast and clothing recommendations, and the server retrieves weather information from an external API and sends appropriate clothing advice to the user's device.

[2307] Users can view live camera footage and get real-time information about the location.

[2308] The server checks for weather forecast updates and sends push notifications to the device, providing users with the latest information.

[2309] The emotion engine recognizes the user's emotions, and the server provides customized information based on that data.

[2310] This allows users to obtain all the necessary information in one place, making travel planning smoother. Furthermore, it enables personalized information delivery tailored to the user's emotional state, resulting in a more satisfying travel experience.

[2311] The following describes the processing flow.

[2312] Step 1:

[2313] The user starts up their device and enters the destination they want to go to (for example, "Kyoto City").

[2314] Step 2:

[2315] The terminal receives user input and formats that information as a request to the server.

[2316] Step 3:

[2317] The device sends a formatted request to the server. This request asks for local information about the travel destination.

[2318] Step 4:

[2319] The server receives the request and executes a query against the database to retrieve local information about "Kyoto City".

[2320] Step 5:

[2321] The server retrieves travel destination information related to "Kyoto City" from the database (such as tourist spots, restaurant information, and the latest information from local residents).

[2322] Step 6:

[2323] The server formats the acquired information into JSON format and sends it back to the terminal.

[2324] Step 7:

[2325] The terminal receives information returned from the server and displays it on the user interface. This allows the user to understand basic information about their travel destination.

[2326] Step 8:

[2327] If a user wants to know the weather in Kyoto City on a specific day, they send that request to the server via their device.

[2328] Step 9:

[2329] The device forwards the user's weather request to the server. This request includes local information and the date.

[2330] Step 10:

[2331] Based on the request received by the server, it queries an external weather forecast API for weather information.

[2332] Step 11:

[2333] The server receives weather information returned from an external weather forecast API and generates appropriate clothing recommendations based on the acquired weather information. For example, if the weather forecast is for cold weather, it will generate a recommendation such as "You need a warm jacket."

[2334] Step 12:

[2335] The server formats the weather information and generated clothing recommendations into JSON format and sends them back to the terminal.

[2336] Step 13:

[2337] The device displays weather information and clothing recommendations received by the user interface. The user can use this information to prepare for their trip.

[2338] Step 14:

[2339] If a user wants to view live camera footage from the location, they send a request to the server via their device.

[2340] Step 15:

[2341] The device forwards the live camera request to the server.

[2342] Step 16:

[2343] The server retrieves the URL of the live camera feed for "Kyoto City" from the database and sends it back to the terminal.

[2344] Step 17:

[2345] The device retrieves the video stream based on the live camera's URL and displays it on the user interface. This allows users to check the local situation in real time.

[2346] Step 18:

[2347] The system collects emotional data (such as facial expressions and voice tone) from the user through the camera and microphone built into their device.

[2348] Step 19:

[2349] The device sends collected emotional data to an emotion engine, which then recognizes the user's emotional state.

[2350] Step 20:

[2351] The emotion engine analyzes the user's emotions and sends the results back to the device.

[2352] Step 21:

[2353] The device sends recognized emotion information to the server.

[2354] Step 22:

[2355] The server customizes weather information, clothing recommendations, and other information based on emotional data. For example, if a user is feeling stressed, it will provide information on tourist spots and cafes suitable for relaxation.

[2356] Step 23:

[2357] The server formats the customized information into JSON format and sends it back to the terminal.

[2358] Step 24:

[2359] The device displays the customized information it receives in the user interface. The user can use this information to optimize their travel plan.

[2360] Step 25:

[2361] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[2362] Step 26:

[2363] When the server detects important weather updates, it sends that information as a push notification to the relevant users.

[2364] Step 27:

[2365] The device receives push notifications and displays them on the user interface. This allows users to receive the latest weather information in real time and respond quickly.

[2366] (Example 2)

[2367] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2368] There is a need for a system that allows users planning trips to efficiently obtain information about their destination and receive recommendations for appropriate clothing based on weather information. However, in conventional systems, the information is scattered and not provided in a centralized manner. Furthermore, there is a problem in that information is not customized based on the user's emotional state, making it difficult to provide personalized information. In addition, the provision of live camera footage to check real-time conditions at the destination is not integrated, so users face the inconvenience of gathering information from multiple sources.

[2369] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[2370] In this invention, the server includes means for collecting travel destination information from local residents, means for obtaining weather information from an external weather forecast API, and means for generating clothing recommendations based on the acquired weather information. This allows users to centrally obtain travel destination information and receive appropriate clothing recommendations based on weather information. The invention also includes means for providing live camera footage to check local conditions, and an emotion engine that recognizes and analyzes the user's emotions using the terminal's camera and microphone, and customizes weather information and clothing recommendations based on the recognized user emotion information. This enables personalized information provision according to the user's emotional state, not only allowing travel planning to proceed more smoothly but also providing a more satisfying travel experience.

[2371] "Travel destination information" refers to detailed information about tourist attractions, restaurants, events, festivals, and other information related to the region where the user is planning to travel.

[2372] "Local residents" refer to people who actually live in the area where the user is planning to travel, and the information they provide is the most up-to-date local information.

[2373] An "external weather forecast API" is an application programming interface for a service that provides weather forecast information via the internet.

[2374] "Weather information" refers to information about current and future weather conditions (temperature, precipitation, wind speed, etc.) for a specific region.

[2375] "Clothing recommendations" refer to information that suggests appropriate clothing to the user based on acquired weather information.

[2376] A "traveler" is a user who is planning a trip.

[2377] "Live camera footage" refers to a video stream that captures real-time footage of the local situation in a specific area and distributes it via the internet.

[2378] A "device" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[2379] A "database" is a collection of data used to systematically store and manage information, and it is accessible from a server.

[2380] "User interface" refers to the screens and operating methods that serve as the point of contact between a system and a user when inputting or outputting information.

[2381] An "HTTP request" is a communication protocol used to request the transmission or reception of information from a server over the internet.

[2382] An "emotion engine" is software that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

[2383] "Push notifications" are a technology that allows a server to send information to a user's device in real time.

[2384] This invention provides travel destination information, weather forecasts, clothing recommendations, and live camera footage, as well as a system that recognizes user emotions and customizes information based on those emotions. This system is implemented using a server, terminals, and an emotion engine.

[2385] Forms of information gathering

[2386] The user enters the region they are planning to travel to via their device. For example, they might enter "Kyoto City".

[2387] The terminal sends the travel destination information entered by the user to the server as an HTTP request. The server receives this request and retrieves local information for the specified region from its database.

[2388] The server retrieves the latest information from its database, including local tourist attractions, restaurant information, and information provided by local residents, and sends it back to the terminal. For example, this includes information about tourist attractions and festivals in Kyoto City.

[2389] Example of a prompt:

[2390] Please enter "Kyoto City" as your travel destination.

[2391] Generation methods for weather forecasts and clothing recommendations

[2392] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[2393] The terminal forwards the user's request to the server, sending an HTTP request that includes geographical information about the travel destination.

[2394] The server receives this request and uses an external weather forecast API to retrieve weather information for the specified area. Based on the retrieved weather information, it generates clothing recommendations suitable for the user. For example, if the forecast for Kyoto City is cold, it will return a recommendation to the device such as, "You will need a warm jacket."

[2395] Example of a prompt:

[2396] Could you please tell me the weather forecast for Kyoto City?

[2397] Forms of emotional engines

[2398] To recognize the user's emotions, the device is equipped with a camera and microphone, which are used to analyze the user's facial expressions and tone of voice.

[2399] The emotion engine analyzes this data to recognize the user's emotional state.

[2400] The device sends the recognized emotion information to the server.

[2401] The server uses this information to customize weather forecasts and clothing recommendations. For example, if a user is feeling stressed, it will provide information on relaxing tourist spots and appropriate clothing recommendations.

[2402] Example of a prompt:

[2403] Please recommend some places based on your current emotional state.

[2404] How to view live camera footage

[2405] Users request live camera footage to check the local situation in real time.

[2406] The device sends a request for live camera footage to the server.

[2407] The server retrieves the URL of the appropriate live camera from the database and sends it back to the terminal.

[2408] The device uses the live camera's URL to retrieve the video stream and displays it on the user interface. This allows the user to check the local situation in real time.

[2409] Example of a prompt:

[2410] Please show me the live camera footage of Kyoto City.

[2411] Update and notification methods

[2412] The server periodically checks the weather forecast API to see if there is any new information or important updates.

[2413] When an important update is detected, the server will send a push notification to the relevant users.

[2414] The device receives push notifications and displays them on the user interface. This allows users to receive real-time updates on weather conditions and other important information, which they can then incorporate into their travel plans.

[2415] Example of a prompt:

[2416] Please provide real-time notifications of the latest weather forecast for Kyoto City.

[2417] This system allows users to efficiently receive a series of pieces of information and plan a comfortable trip. Furthermore, it enables personalized information delivery tailored to the user's emotional state, resulting in a highly satisfying travel experience.

[2418] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2419] Forms of information gathering

[2420] Step 1:

[2421] The user enters the region they are planning to travel to via their device. For example, they might enter "Kyoto City".

[2422] Input: Travel destination (e.g., "Kyoto City")

[2423] Output: Entered travel destination information

[2424] Step 2:

[2425] The terminal formats the travel destination information entered by the user as an HTTP request.

[2426] Input: Entered travel destination information

[2427] Output: HTTP request containing travel destination information

[2428] Step 3:

[2429] The device sends travel destination information to the server as an HTTP request.

[2430] Input: HTTP request containing travel destination information

[2431] Output: Sending a request to the server

[2432] Step 4:

[2433] The server receives this request and retrieves local information for the specified region from the database.

[2434] Input: HTTP request containing travel destination information

[2435] Output: Local information (e.g., tourist spots, restaurant information, etc.)

[2436] Step 5:

[2437] The server returns the retrieved information to the terminal in JSON format.

[2438] Input: Local information

[2439] Output: HTTP response for the terminal

[2440] Step 6:

[2441] The terminal parses the received JSON data and displays it in the user interface.

[2442] Input: Response from server

[2443] Output: Local information displayed in the user interface

[2444] Generation methods for weather forecasts and clothing recommendations

[2445] Step 1:

[2446] If a user wants to know the weather information for their travel destination, they send a request to the server via their device.

[2447] Input: Weather information request

[2448] Output: Request for terminal

[2449] Step 2:

[2450] The device generates an HTTP request containing geographical information about the travel destination and sends it to the server.

[2451] Input: Weather information request

[2452] Output: HTTP request containing geographical information

[2453] Step 3:

[2454] The server receives this request and uses an external weather forecast API to retrieve weather information for the specified area.

[2455] Input: HTTP request containing geographical information

[2456] Output: Weather information

[2457] Step 4:

[2458] The server generates clothing recommendations suitable for the user based on the acquired weather information.

[2459] Input: Weather information

[2460] Output: Clothing recommendation (e.g., "You need a warm jacket")

[2461] Step 5:

[2462] The server returns the generated recommendations to the device.

[2463] Input: Clothing Recommendation

[2464] Output: HTTP response for the terminal

[2465] Step 6:

[2466] The device displays the received clothing recommendations in the user interface.

[2467] Input: Response from server

[2468] Output: Clothing recommendations displayed in the user interface

[2469] Forms of emotional engines

[2470] Step 1:

[2471] The device is equipped with a camera and microphone to collect the user's facial expressions and voice tone.

[2472] Input: User's facial expressions and tone of voice

[2473] Output: Collected data

[2474] Step 2:

[2475] The emotion engine analyzes this data to recognize the user's emotional state.

[2476] Input: Collected data

[2477] Output: Recognized emotional state

[2478] Step 3:

[2479] The device sends the recognized emotion information to the server.

[2480] Input: Recognized emotional state

[2481] Output: HTTP request to the server

[2482] Step 4:

[2483] The server uses this information to customize weather forecasts and clothing recommendations.

[2484] Input: Emotional information

[2485] Output: Customized weather information and clothing recommendations

[2486] Ste...

Claims

1. Methods for gathering information about travel destinations from local residents, Methods for obtaining weather information from external weather forecast APIs, A means for generating clothing recommendations based on acquired weather information, A means of presenting weather information and clothing recommendations to travelers, A means of providing live camera footage to check the situation on site, A system that includes this.

2. A means of storing information collected from local residents in a database, A means of displaying travel destination information, weather information, and clothing recommendations on the user interface, The system according to claim 1, further comprising:

3. A means of streaming live camera footage in response to user requests, A means of periodically obtaining updated weather forecast information and sending push notifications to users, The system according to claim 1, further comprising:

Images