system

Abstract

We provide the system. [Solution] A means of obtaining the user's current location information, A means of obtaining the user's destination information, A means of obtaining information about the nearest public transport based on the current location, A method for calculating possible travel routes available after the last train has departed, A means of calculating the time required, cost, and distance traveled for each travel route, A means of presenting the user with multiple travel routes, A means of performing the necessary actions based on the travel route selected by the user, A system that includes this.

Description

Technical Field

[0003]

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern urban life, when going out during the time when public transportation ends, it is important to find an appropriate means of returning home. However, if one misses the last train, in many cases the available means of transportation are limited, and it is difficult to find an efficient and economical way to get home. Therefore, there is a need for a system that allows users to return home in the optimal way for themselves even when they miss the last train. Specifically, a system that proposes an optimal way to return home by combining multiple means of transportation is required.

Means for Solving the Problems

[0005] The present invention solves the above problem with a system that includes means for acquiring the user's current location information, means for acquiring the user's destination information, means for acquiring information on the nearest public transportation based on the current location, means for calculating candidate travel routes available when the last train has departed, means for calculating the time, cost, and distance traveled for each travel route, means for presenting multiple travel routes to the user, and means for performing necessary actions based on the travel route selected by the user. Specifically, by presenting a route as a candidate travel route that uses public transportation to a specific point and then proceeds on foot or by taxi from there, the invention provides a way for the user to return home efficiently and economically even if they miss the last train.

[0006] A "user" refers to someone who uses this system to find a way to get home after missing the last train.

[0007] "Current location information" refers to the geographical location information of the user's actual location at the present time, and is obtained through GPS functionality or manual input.

[0008] "Destination information" refers to the place the user ultimately wants to reach, specifically the user's home address.

[0009] "Public transportation" refers to means of transport that can be used by the general public, such as trains and buses, where departure and arrival times and routes are predetermined.

[0010] "Last train" refers to the final service of the day on a particular public transportation system.

[0011] A "travel route" refers to the path a user takes to reach their destination from their current location, and may involve combining multiple modes of transportation.

[0012] "Walking" refers to the means by which a user travels by walking on their own two feet.

[0013] A "taxi" refers to a vehicle used by passengers who pay a fare to their destination, and is generally used after the last public transport service has departed.

[0014] "Estimated time" refers to the total time it takes for a user to travel from their current location to their destination.

[0015] "Cost" refers to the monetary cost that a user pays when using a particular mode of transportation.

[0016] "Distance traveled" refers to the physical distance a user travels from their current location to their destination.

[0017] An "action" refers to a specific operation or instruction that the system performs based on the travel route selected by the user, and includes things like booking a taxi or receiving notifications. [Brief explanation of the drawing]

[0018] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8]It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0020] First, the terms used in the following description will be described.

[0021] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] ---

[0040] This invention is a system that suggests an efficient and economical route home for users who have missed the last train. Embodiments of this invention will be described in detail below.

[0041] ---

[0042] System Overview

[0043] This system suggests the most efficient way to get home based on the user's current location and destination information. Specifically, it calculates and presents candidate routes that combine information on public transport, taxis, and walking.

[0044] ---

[0045] Program Processing Description

[0046] 1. Obtaining user information

[0047] User: Launch the app and use the automatic location acquisition function to determine your current location. If GPS functionality is unavailable, manually enter your address.

[0048] User: Enter your home address.

[0049] 2. Obtain the nearest station and the last train time.

[0050] Terminal: Searches for the nearest station from the current location and retrieves the last train time for that station from the public transport database.

[0051] 3. Proposed solutions for when you miss the last train.

[0052] Server: Check if the time for the last train has passed the current time.

[0053] Server: If you miss the last train, calculate the following route.

[0054] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[0055] Server: Searches for the nearest other train station from the current location to the home address and calculates the walking route from there.

[0056] Route 2: Go as far as you can by train, then take a taxi home.

[0057] Server: Calculates the furthest point reachable by train and estimates the taxi fare from there to home.

[0058] Route 3: Walk home entirely.

[0059] Server: Calculates the walking route from the current location to home and displays the estimated time.

[0060] Route 4: Go home entirely by taxi.

[0061] Server: Estimate the taxi fare from the current location to home.

[0062] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[0063] Server: Searches for entertainment and hotels near the current location and displays reservation information.

[0064] 4. Route Optimization and Suggestion

[0065] Server: Calculates the required time, cost, and distance for each route, and calculates the optimal route based on the conditions set by the user.

[0066] Server: Generates a route list in order of priority and sends it to the terminal along with detailed information.

[0067] 5. Providing directions to the user

[0068] Terminal: Displays a list of suggested routes and visually shows the details of each route (estimated time, cost, distance).

[0069] 6. User selection and action execution

[0070] User: Select the most suitable route from the displayed options and press the "Select" button on the screen.

[0071] Terminal: Based on the user's selection, it displays options for performing necessary actions (calling a taxi, making a reservation for entertainment).

[0072] User: Select the appropriate action (e.g., call a taxi) and follow the on-screen instructions.

[0073] Terminal: Performs the specified action, for example, automatically contacting a taxi company.

[0074] ---

[0075] Specific example:

[0076] scenario:

[0077] User: I'm at Shinjuku Station.

[0078] Destination: My home in Naka Ward, Yokohama City.

[0079] Terminal: Confirm that Shinjuku Station is the nearest station and check the last train time (e.g., 23:30) obtained from the public transport database.

[0080] Server: Confirming that you have missed the last train at the current time (e.g., 23:45), the following route is suggested.

[0081] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[0082] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[0083] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[0084] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[0085] Route 5: Stay at a hotel near Shinjuku and return home on the first train the next day (hotel fee: 3000 yen).

[0086] User: Select "Route 2" and initiate a taxi call on the device.

[0087] Terminal: Contact a taxi company and arrange for a taxi to your current location.

[0088] Thus, the present invention provides users who have missed the last train with the most suitable way to get home, thereby reducing the burden on users.

[0089] The following describes the processing flow.

[0090] Step 1:

[0091] The user launches the app and uses the automatic location acquisition function to determine their current location. If GPS functionality is unavailable, the user manually enters their current address.

[0092] Step 2:

[0093] The user enters their home address. This allows the destination information to be identified.

[0094] Step 3:

[0095] The device searches for the nearest public transport station based on the user's current latitude and longitude. This identifies the user's nearest station.

[0096] Step 4:

[0097] The terminal retrieves the last train time for the nearest station from the public transport database. This database contains operational information for each station.

[0098] Step 5:

[0099] The server checks the current time and compares it to the last train time of the nearest station. It then determines whether the current time is past the last train time.

[0100] Step 6:

[0101] If the server determines that you have missed the last train, it will calculate several possible routes home. Specifically, these include the following routes:

[0102] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[0103] The server searches for the nearest other train station from your current location to your home address and calculates the walking route from there.

[0104] Route 2: Go as far as you can by train, then take a taxi home.

[0105] The server calculates the furthest point you can reach by train and then estimates the taxi fare from there to your home.

[0106] Route 3: Walk home entirely.

[0107] The server calculates the walking route from the current location to home and displays the estimated time.

[0108] Route 4: Go home entirely by taxi.

[0109] The server estimates the taxi fare from your current location to your home.

[0110] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[0111] The server searches for entertainment and hotels near the current location and displays reservation information.

[0112] Step 7:

[0113] The server calculates the time, cost, and distance for each route, and then calculates the optimal route based on the conditions set by the user (e.g., "fastest" or "most economical").

[0114] Step 8:

[0115] The server generates a route list in order of priority and sends it to the terminal, along with detailed information.

[0116] Step 9:

[0117] The device displays a list of suggested routes and visually shows the details of each route (estimated time, cost, and distance).

[0118] Step 10:

[0119] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen.

[0120] Step 11:

[0121] The device displays options to perform the necessary action (call a taxi, make a reservation for entertainment) based on the user's selection.

[0122] Step 12:

[0123] The user selects the appropriate action (for example, calling a taxi) and proceeds by following the on-screen instructions.

[0124] Step 13:

[0125] The device performs a specified action, for example, automatically contacting a taxi company.

[0126] Step 14:

[0127] The server monitors the route and actions selected by the user, and updates route information or suggests alternative routes as needed.

[0128] Step 15:

[0129] The device tracks the user's current location and displays an alert if they get lost or need further assistance.

[0130] Step 16:

[0131] The user finally arrives home and chooses to close the app.

[0132] (Example 1)

[0133] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0134] One challenge is that it is difficult for users who miss the last train to find an efficient and economical way to get home. Another challenge is the lack of a system that simultaneously evaluates travel time, cost, and distance to provide the optimal route when users are looking for a way to get home.

[0135] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0136] In this invention, the server includes means for acquiring the user's current location information, means for acquiring the user's destination information, means for acquiring information on the nearest public transportation based on the current location, means for calculating candidate routes available when the last train has departed, means for calculating the time, cost, and distance for each route, means for presenting multiple routes to the user, means for performing necessary actions based on the route selected by the user, means for calculating walking routes based on the proposed routes, means for estimating taxi fares based on the proposed routes, and means for searching for and displaying accommodation information based on the proposed routes. This makes it possible to efficiently and economically provide the optimal route home to a user who has missed the last train.

[0137] A "user" refers to an individual who uses the system to search for a route home.

[0138] "Current location information" refers to geographical data indicating the user's current location.

[0139] "Destination information" refers to geographical data about the location where the user is aiming to return home.

[0140] "Public transportation information" refers to timetables and operating status information for modes of transport such as trains and buses.

[0141] "Last train time" refers to the departure time and service information of the last train running on that day.

[0142] "Possible travel routes" refer to multiple routes that a user can choose to take to get home.

[0143] A "walking route" refers to the path a user takes to reach their destination on foot.

[0144] "Taxi fare estimate" refers to a predicted value of the fare incurred when using a taxi.

[0145] "Accommodation facilities" refer to facilities that users can use for temporary lodging (such as hotels and entertainment venues).

[0146] "Action" refers to a specific action taken as part of the means of getting home (e.g., calling a taxi, booking accommodation).

[0147] A "server" refers to a computer system that processes data and provides information based on requests from users.

[0148] "Terminal" refers to a device (such as a smartphone or tablet) that a user uses to access the system.

[0149] This invention is a system that suggests an efficient and economical route home for users who have missed the last train. The embodiments of this invention will be described in detail below.

[0150] This system is implemented through various operations by servers, terminals, and users. The hardware used includes terminals such as smartphones and tablets, while small computers and servers perform the main processing. The software used includes GPS functionality for acquiring location information and APIs (e.g., Google® Maps API, NAVITIME, etc.) for acquiring public transportation information.

[0151] User information retrieval

[0152] The user launches the app on their device and enables automatic location acquisition. The device uses its GPS function to determine its current location and sends the location information to the server. If GPS is unavailable, the user can also manually enter their address. Furthermore, the user enters their destination information (where they will return home) into their device.

[0153] Obtain the nearest station and the last train time.

[0154] The device sends its current location information to the server. The server uses an API to retrieve public transportation information, specifically the nearest station and its last train time. This retrieved information is sent to the device and displayed to the user.

[0155] Suggestions for what to do if you miss the last train.

[0156] The server checks if the user has missed the last train by comparing the retrieved last train time with the current time. If the user has missed the last train, it calculates the following possible travel routes.

[0157] Route 1: Take the train to a station that is not the closest but is nearby, and then walk home from there. The server searches for the nearest other station from your current location to your destination and calculates the walking route.

[0158] Route 2: Go as far as you can by train, then take a taxi home. The server calculates the furthest point you can reach by train and estimates the taxi fare from that point to your destination.

[0159] Route 3: Walk home entirely. The server calculates the walking route from the current location to the destination and compiles the estimated time.

[0160] Route 4: Take a taxi home. The server estimates the taxi fare from your current location to your destination.

[0161] Route 5: Stay at a nearby accommodation and return home the next morning. The server searches for accommodations near your current location and displays reservation information.

[0162] Route optimization and suggestions

[0163] The server comprehensively calculates the time, cost, and distance for each potential travel route and selects the optimal route based on user-defined conditions (e.g., minimizing costs, prioritizing time). Finally, the server generates a route list in order of priority and sends it to the terminal along with detailed information.

[0164] Providing route guidance to users

[0165] The device displays the route list received from the server on the app screen. Details of each route (estimated time, cost, distance) are displayed visually, allowing the user to scroll and review them.

[0166] User selection and action execution

[0167] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen. Based on this selection, the terminal suggests and executes the necessary actions (e.g., calling a taxi, making a hotel reservation). Once the user completes the operation, the terminal automatically performs the instructed actions and contacts the taxi company or accommodation.

[0168] Specific example

[0169] scenario:

[0170] User: I'm at Shinjuku Station.

[0171] Destination: My home in Naka Ward, Yokohama City.

[0172] Terminal:

[0173] The nearest station is Shinjuku Station, and the last train time (e.g., 23:30) obtained from the public transport database is checked. It is confirmed that the last train has been missed from the current time (e.g., 23:45), and the server suggests the following route.

[0174] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[0175] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[0176] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[0177] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[0178] Route 5: Stay at an accommodation near Shinjuku and return home on the first train the next day (accommodation fee: 3000 yen).

[0179] user:

[0180] Select "Route 2" and then use the terminal to call a taxi.

[0181] Terminal:

[0182] The system contacts a taxi company and arranges for a taxi to be dispatched to the user's current location. This ensures that users have an efficient and economical way to get home, even if they miss the last train.

[0183] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0184] System program processing flow

[0185] Step 1:

[0186] The user launches the app on their smartphone or tablet. When the user presses the "Automatically acquire current location" button, the device enables its GPS function and acquires the current location. GPS data is input, and the current location information is output. Specifically, the device acquires data from the GPS sensor and displays it as the current location.

[0187] Step 2:

[0188] The user manually enters their current address (if GPS is unavailable). Once the user enters the address and presses the "Confirm" button, the manually entered address is output as the current location information. Specifically, the user's input is retrieved in text format and saved to an internal database.

[0189] Step 3:

[0190] The user enters their home address. Once the address is entered into the terminal's input form and the "Next" button is pressed, destination information is output. Specifically, the user's input is retrieved in text format and saved to an internal database.

[0191] Step 4:

[0192] The device sends its current location and destination information to the server. The current location and destination information are used as input, and the server receives this information as output. Specifically, the device sends data to the server via the internet.

[0193] Step 5:

[0194] The server accesses a public transport API (e.g., Google Maps API) to retrieve the nearest station and last train time. Current location information is used as input, and the output is data on the nearest station and last train time. Specifically, the server sends an API request and saves the received data to an internal database.

[0195] Step 6:

[0196] The server compares the current time with the last train time to determine if the user has missed the last train. The current time and last train time are used as input, and the output is a determination of whether the user missed the last train. Specifically, the server executes an algorithm to compare the current time with the last train time.

[0197] Step 7:

[0198] If you miss the last train, the server calculates possible travel routes. Current location information, destination information, and public transport information are used as input, and multiple travel routes are returned as output. Specifically, the following routes are calculated:

[0199] Route 1: Take the train to a station that is not the closest but is nearby from your current location, and then walk home from there.

[0200] The server searches for the nearest station from the current location and calculates the walking route. The output provides information about the station and the walking route.

[0201] Route 2: Go as far as you can by train, then take a taxi home.

[0202] The server calculates the final destination reachable by train and estimates the taxi fare. The output provides the final destination and the estimated taxi fare.

[0203] Route 3: Walk home entirely.

[0204] The server calculates the walking route from the current location to the destination. The output includes the walking route and the estimated time.

[0205] Route 4: Go home entirely by taxi.

[0206] The server estimates the taxi fare from the current location to the destination. The taxi fare is returned as output.

[0207] Route 5: Stay at a nearby accommodation and return home the next day.

[0208] The server searches for accommodations near the current location and displays a list of accommodations and reservation information. Accommodation information is provided as output.

[0209] Step 8:

[0210] The server calculates the time, cost, and distance for each route and selects the optimal route based on user-defined conditions. Route candidate data is used as input, and the output is a list of optimal routes. Specifically, the server executes a route evaluation algorithm and calculates the evaluation results.

[0211] Step 9:

[0212] The server sends the optimal route list to the terminal. The route list is used as input, and the terminal receives the route list as output. Specifically, the server sends data over the internet, and the terminal receives it.

[0213] Step 10:

[0214] The app displays the route list received by the device on the app screen. It visually displays the estimated time, cost, and distance for each route so that the user can review the route details. Specifically, the device generates the display data and renders it on the screen.

[0215] Step 11:

[0216] The user selects a route and presses the "Select" button. The selected route information is used as input, and the selection result is obtained as output. Specifically, the user's selection is received and sent to the server.

[0217] Step 12:

[0218] The device performs actions based on the user's selections (e.g., calling a taxi, making a hotel reservation). User selection information is used as input, and the output is the result of executing a specific action. Specifically, the device calls the corresponding API to contact the taxi or accommodation provider.

[0219] (Application Example 1)

[0220] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0221] While systems already exist that suggest efficient and economical routes home for users who miss the last train, there is a similar need for technology that calculates efficient and economical delivery routes in the food delivery industry. In particular, since reducing time and costs is a critical issue in the food delivery industry, a system that can provide efficient delivery routes is essential.

[0222] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0223] In this invention, the server includes means for acquiring the user's current location information, means for acquiring destination information, means for acquiring information on the nearest public transport, means for calculating candidate available travel routes, means for calculating the time, cost, and distance for each travel route, means for presenting multiple travel routes, means for performing necessary actions based on the travel route selected by the user, and means for calculating an efficient delivery route for food delivery. This makes it possible to provide the optimal travel route in terms of both time and cost.

[0224] "User" refers to an individual or group that uses the system.

[0225] "Current location information" refers to the latitude and longitude data of the user's current location.

[0226] "Destination information" refers to data such as the address and latitude / longitude of the place the user is heading to.

[0227] "Public transportation" refers to mass transportation methods such as buses, trains, and subways.

[0228] "Possible travel routes" refer to multiple options for routes and means of transportation to reach a destination.

[0229] "Travel time" refers to the time required to travel from your current location to your destination.

[0230] "Cost" refers to the economic cost required to travel from your current location to your destination.

[0231] "Distance traveled" refers to the physical distance from your current location to your destination.

[0232] "Food delivery" refers to a service that delivers food and beverages to a specified location.

[0233] An "efficient delivery route" is a route optimized to minimize delivery time and costs.

[0234] "Means of performing actions" refer to methods such as calling a car or taxi or booking accommodation based on the travel route selected by the user.

[0235] This invention is a system that proposes efficient and economical delivery routes for food delivery. Based on the user's current location and destination information, this system calculates and presents a delivery route combining multiple modes of transportation to the user. Embodiments of this invention will be described in detail below.

[0236] Server Processing

[0237] The server first obtains the user's current location and destination information. Current location information is obtained using GPS or an IP geolocation service. Destination information is obtained using the Google Maps API from the address entered by the user, providing latitude and longitude. This ensures the server has accurate location information for both the current and destination locations.

[0238] Next, the server retrieves information about the nearest public transport. To do this, it consults a public transport database to obtain the nearest train station or bus stop and the time of the last train. If the last train has already departed, the server calculates route options for each mode of transportation (bicycle, public transport, walking).

[0239] Route options for each mode of transport are calculated based on their average speed, determining the travel time, cost, and distance. For example, a standard of 15 km / h is used for cycling, 30 km / h for public transport, and 5 km / h for walking. In this way, the server can obtain detailed information for all included routes.

[0240] Terminal processing

[0241] The terminal visually displays multiple route options sent from the server to the user. Each route clearly indicates the estimated travel time, cost, and distance, and presents the user with its advantages and disadvantages. Based on this information, the user can choose the most suitable mode of transportation.

[0242] Based on the route selected by the user, the device performs the necessary actions. For example, if the user requests a taxi or makes a hotel reservation, the device automatically arranges these using the corresponding API.

[0243] Specific example

[0244] User's current location: Shibuya Station

[0245] Destination: Restaurants near Ebisu Station

[0246] Optimal route: Travel by bicycle (approximately 10 minutes)

[0247] In this specific example, if the user is at Shibuya Station, the server obtains location information for both the user's current location and destination, and selects a bicycle as the most efficient route. This bicycle route is calculated to take approximately 10 minutes and is then presented to the user.

[0248] Example of a prompt

[0249] An example of a prompt message to input to a generative AI model is as follows:

[0250] text

[0251] Develop an application that suggests the most efficient route for delivery drivers when a user orders food delivery. Please use the following information as a basis.

[0252] Current location: Shibuya Station

[0253] Destination: Restaurants near Ebisu Station

[0254] Available modes of transportation: bicycle, public transport, walking

[0255] Calculate the travel time and distance for each mode of transportation and suggest the optimal route.

[0256] This allows users to obtain efficient routes in real time and reach their destinations quickly and economically.

[0257] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0258] Step 1:

[0259] The server obtains the user's current location information. Using GPS functionality or an IP geolocation service, it obtains the latitude and longitude of the user's current location from the user's device. This allows the server to obtain the user's precise location information as input data. The output data is the latitude and longitude information of the current location.

[0260] Step 2:

[0261] The user enters the destination address. The device retrieves this entered address data and uses the Google Maps API to obtain the corresponding latitude and longitude. The server receives this latitude and longitude information as input data and stores it as destination information. The output data is the latitude and longitude information of the destination.

[0262] Step 3:

[0263] The server retrieves information about the nearest public transport based on the user's current location. It consults a public transport database to obtain the nearest train station or bus stop, as well as the time of the last train or bus. The server receives this information as input data and stores it as information about the nearest public transport. The output data includes the name, location, and time of the last train or bus of the nearest public transport.

[0264] Step 4:

[0265] The server calculates possible travel routes. It calculates route options for each mode of transport (bicycle, public transport, walking) from the current location to the destination, and calculates the estimated time, cost, and distance for each. The server receives these calculation results as input data and stores them as travel route options. The output data consists of the estimated time, cost, and distance for each mode of transport.

[0266] Step 5:

[0267] The server presents the user with the results of each calculated travel route. The terminal visually displays the detailed information (estimated time, cost, distance) of each travel route received from the server. The user selects the optimal route based on this information. The input data is the route candidate information received from the server, and the output data is the route details displayed on the terminal.

[0268] Step 6:

[0269] The user selects the optimal route from several presented travel routes. The terminal receives the user's selection and sends it to the server. The input data is the route information selected by the user, and the output data is the selected route information sent to the server.

[0270] Step 7:

[0271] The server performs the necessary actions based on the travel route selected by the user. For example, it automatically arranges things like calling a taxi or booking accommodation using APIs corresponding to each route. The input data is the route information selected by the user, and the output data is the result of each arrangement action.

[0272] This allows users to obtain efficient and economical travel routes and take appropriate actions quickly.

[0273] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0274] ---

[0275] This invention combines a system that suggests efficient and economical routes home when a user misses the last train with an emotion engine that recognizes the user's emotions. Based on the user's current location and destination information, the system suggests the most suitable way home. By utilizing the emotion engine, the suggested travel route can be adjusted according to the user's emotional state.

[0276] ---

[0277] System Overview

[0278] This system acquires the user's current location information and destination information, and presents the optimal route to go home efficiently and economically even if the last train is missed. It also recognizes the user's emotional state and adjusts the travel route based on that emotion.

[0279] ---

[0280] Explanation of Program Processing

[0281] 1. Acquisition of User Information

[0282] User: Launch the app and identify the current location using the automatic location acquisition function. If the GPS function is not available, manually enter the address.

[0283] User: Enter the address of home. This identifies the destination information.

[0284] 2. Acquisition of the Nearest Station and the Last Train Time

[0285] Terminal: Search for the nearest station from the current location and acquire the last train time of that station from the public transportation database.

[0286] 3. Recognition of the User's Emotional State

[0287] Terminal: Use the camera and microphone in the app to recognize the emotion from the user's expression and voice tone.

[0288] Terminal: The emotion engine analyzes the user's emotional state and identifies, for example, fatigue, stress, relaxation, etc.

[0289] 4. Proposals for Countermeasures in Case of Missing the Last Train

[0290] Server: Check whether the time of the last train has passed the current time.

[0291] Server: If the last train is missed, calculate the following routes.

[0292] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[0293] Server: Searches for the nearest other train station from the current location to the home address and calculates the walking route from there.

[0294] Route 2: Go as far as you can by train, then take a taxi home.

[0295] Server: Calculates the furthest point reachable by train and estimates the taxi fare from there to home.

[0296] Route 3: Walk home entirely.

[0297] Server: Calculates the walking route from the current location to home and displays the estimated time.

[0298] Route 4: Go home entirely by taxi.

[0299] Server: Estimate the taxi fare from the current location to home.

[0300] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[0301] Server: Searches for entertainment and hotels near the current location and displays reservation information.

[0302] 5. Route Optimization and Suggestions

[0303] Server: Calculates the time, cost, and distance for each route, and adjusts the optimal route based on the user's emotional state.

[0304] Server: If the user is tired, prioritize suggesting a relaxing route; if they prioritize economical options, suggest a low-cost route.

[0305] Server: Generates a route list in order of priority and sends it to the terminal along with detailed information.

[0306] 6. Route Presentation to the User

[0307] Terminal: Display a list of each proposed route and visually display the details of each route (required time, cost, travel distance, emotional considerations).

[0308] 7. User Selection and Action Execution

[0309] The user selects the most suitable method from the displayed routes and presses the "Select" button on the screen.

[0310] The terminal displays options for executing the necessary actions (calling a taxi, making a reservation for enjoyment) based on the user's selection.

[0311] The user selects an appropriate action (e.g., calling a taxi) and proceeds according to the operations on the screen.

[0312] The terminal executes the specified action and automatically contacts, for example, a taxi company.

[0313] ---

[0314] Specific Example:

[0315] Scenario:

[0316] User: Is at Shinjuku Station.

[0317] Destination: Home in Naka-ku, Yokohama City.

[0318] User's Emotion: Recognized as tired through analysis of expression and voice.

[0319] Terminal: Confirm that Shinjuku Station is the nearest station and obtain the last train time (e.g., 23:30) from the public transportation database.

[0320] Server: Confirming that you have missed the last train at the current time (e.g., 23:45), the following route is suggested.

[0321] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[0322] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[0323] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[0324] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[0325] Route 5: Stay at a hotel near Shinjuku and return home on the first train the next day (hotel fee: 3000 yen).

[0326] Server: Since the server recognizes the user is tired, it prioritizes displaying the most comfortable and fastest "Route 4".

[0327] User: Select "Route 4" and initiate a taxi call on the device.

[0328] Terminal: Contact a taxi company and arrange for a taxi to your current location.

[0329] In this way, by utilizing the emotion engine, it is possible to provide the optimal way for users to return home according to their emotions, further reducing the burden on users.

[0330] The following describes the processing flow.

[0331] Step 1:

[0332] The user launches the app and uses the automatic location acquisition function to determine their current location. If GPS functionality is unavailable, the user manually enters their current address.

[0333] Step 2:

[0334] The user enters their home address. This allows the destination information to be identified.

[0335] Step 3:

[0336] The device searches for the nearest public transport station based on the user's current latitude and longitude. This identifies the user's nearest station.

[0337] Step 4:

[0338] The terminal retrieves the last train time for the nearest station from the public transport database. This database contains operational information for each station.

[0339] Step 5:

[0340] The server checks the current time and compares it to the last train time of the nearest station. It then determines whether the current time is past the last train time.

[0341] Step 6:

[0342] The device activates the user's facial recognition function and uses the built-in camera to capture the user's facial expressions. It also uses the voice recognition function to record what the user says and transfers this to emotion recognition software.

[0343] Step 7:

[0344] The server's emotion engine analyzes captured facial expressions and voice data to identify the user's emotional state (e.g., fatigue, stress, relaxation).

[0345] Step 8:

[0346] The server takes your emotional state into account and calculates the following return route. Specifically:

[0347] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[0348] The server searches for the nearest other train station from your current location to your home address and calculates the walking route from there.

[0349] Route 2: Go as far as you can by train, then take a taxi home.

[0350] The server calculates the furthest point you can reach by train and then estimates the taxi fare from there to your home.

[0351] Route 3: Walk home entirely.

[0352] The server calculates the walking route from the current location to home and displays the estimated time.

[0353] Route 4: Go home entirely by taxi.

[0354] The server estimates the taxi fare from your current location to your home.

[0355] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[0356] The server searches for entertainment and hotels near the current location and displays reservation information.

[0357] Step 9:

[0358] The server calculates the time, cost, and distance for each route and adjusts the optimal route based on the user's emotional state. If the user is highly fatigued, it prioritizes a comfortable and fast route.

[0359] Step 10:

[0360] The server generates a route list in order of priority and sends it to the terminal, along with detailed information.

[0361] Step 11:

[0362] The device displays a list of suggested routes and visually shows the details of each route (estimated time, cost, distance, and emotional considerations).

[0363] Step 12:

[0364] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen.

[0365] Step 13:

[0366] The device displays options to perform the necessary action (call a taxi, make a reservation for entertainment) based on the user's selection.

[0367] Step 14:

[0368] The user selects the appropriate action (for example, calling a taxi) and proceeds by following the on-screen instructions.

[0369] Step 15:

[0370] The device performs the specified action, for example, automatically contacting a taxi company. It also completes the booking process if a reservation is required.

[0371] Step 16:

[0372] The server monitors the route and actions selected by the user, and updates route information or suggests alternative routes as needed.

[0373] Step 17:

[0374] The device tracks the user's current location and displays an alert if they get lost or need further assistance.

[0375] Step 18:

[0376] After the user finally arrives home, they can choose to exit the app and then it will close.

[0377] (Example 2)

[0378] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0379] In modern urban life, missing the last train is a common occurrence, and finding an efficient and economical way to get home is not easy. Furthermore, the lack of consideration for the user's emotional state can increase fatigue and stress. Conventional systems have been unable to suggest appropriate routes that take the user's emotional state into account.

[0380] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0381] In this invention, the server includes means for acquiring the user's current location information, means for acquiring the user's destination information, and means for calculating multiple candidate travel routes. This makes it possible to propose the optimal travel route while taking into account the user's emotional state.

[0382] "Means of obtaining the user's current location information" refers to a function that determines the user's current location through GPS functionality or manual input by the user.

[0383] "Means for obtaining user destination information" refers to the function of inputting and obtaining information about the address or location set by the user as their destination.

[0384] "Means of obtaining information on the nearest public transport" refers to a function that searches for and retrieves the location and timetable of the nearest public transport based on the user's current location.

[0385] "Means for calculating multiple travel route options" refers to a function that calculates and suggests multiple travel routes that allow a user to reach their destination efficiently and economically, even if they miss the last train.

[0386] "Means for calculating the time, cost, and distance of each travel route" refers to a function that calculates in detail the time, cost, distance, etc., for each proposed travel route.

[0387] "Means of presenting multiple travel routes to the user" refers to a function that visually displays multiple calculated travel routes to the user, making it easier to compare and consider them.

[0388] "Means of performing necessary actions based on the travel route selected by the user" refers to a function that performs specific actions based on the travel route selected by the user, such as arranging a taxi or making a hotel reservation.

[0389] "A means of recognizing the user's emotional state and adjusting the travel route based on those emotions" refers to a function that uses cameras and microphones to analyze the user's emotions from their facial expressions and tone of voice, and then adjusts and suggests the optimal travel route according to those emotions.

[0390]

[0391] This invention is a system that suggests an efficient and economical route home when a user misses the last train, and further combines it with an emotion engine that recognizes the user's emotions to provide the optimal route according to the user's state. Specific embodiments of this invention are described below.

[0392] System Configuration

[0393] Hardware and software

[0394] 1. User's device:

[0395] Smartphone or tablet

[0396] Features include GPS, camera, and microphone.

[0397] 2. Server:

[0398] Servers with high-speed data processing capabilities

[0399] API that integrates with public transport databases

[0400] Emotion analysis engine

[0401] Specific names to use

[0402] GPS function: A standard geolocation API for obtaining the user's current location information.

[0403] Emotion analysis engine: OpenCV is used for facial expression analysis, and Azure Cognitive Services is used for speech analysis.

[0404] Public transport APIs: Google Maps API and public transport databases

[0405] Overview of program processing

[0406] 1. User's device:

[0407] Launch the app and use the GPS function to obtain your current location.

[0408] If the current location cannot be determined, the user will manually enter the address.

[0409] 2. User's device:

[0410] The user enters their home address as destination information.

[0411] 3. Terminal:

[0412] Use the public transport API to retrieve information about the nearest public transport.

[0413] Calculate the distance and travel time from your current location to the nearest station.

[0414] 4. Device:

[0415] The camera and microphone are activated to collect data on the user's face and voice.

[0416] The data is sent to an emotion analysis engine to analyze the user's emotional state (e.g., fatigue, stress, relaxation).

[0417] 5. Server:

[0418] Based on data transmitted from the device, it receives current location, destination, nearest station, last train time, and sentiment data.

[0419] The last train times are retrieved from the public transport database and compared with the current time.

[0420] 6. Server:

[0421] If you miss the last train, calculate multiple routes home.

[0422] For each route, calculate the required time, cost, and distance traveled.

[0423] 7. Server:

[0424] Based on the results of emotion analysis, the optimal route is selected. For example, if the user is tired, taking a taxi home is prioritized.

[0425] 8. Terminal:

[0426] Visually display multiple calculated routes. Detailed information for each route (estimated time, cost, distance, and emotional considerations) is displayed in a list.

[0427] 9. User:

[0428] Select the best route from the available options and press the "Select" button.

[0429] 10. Terminal:

[0430] Based on the selected route, the system will perform the necessary actions (such as calling a taxi or making a reservation for entertainment).

[0431] Specific example

[0432] scenario

[0433] User: I'm at Shinjuku Station.

[0434] Destination: My home in Naka Ward, Yokohama City.

[0435] User's emotions: Analysis of facial expressions and voice indicates that the user is tired.

[0436] Flow of operations

[0437] 1. Terminal: Confirm that Shinjuku Station is the nearest station and retrieve the last train time (e.g., 23:30) from the public transport database.

[0438] 2. Server: Confirms that the user has missed the last train from the current time (e.g., 23:45) and suggests the following route.

[0439] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[0440] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[0441] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[0442] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[0443] Route 5: Stay at a hotel near Shinjuku and return home on the first train the next day (hotel fee: 3000 yen).

[0444] 3. Server: Since the server recognizes that the user is tired, it prioritizes displaying the most comfortable and fastest "Route 4".

[0445] 4. User: Select "Route 4" and initiate a taxi call on the device.

[0446] 5. Terminal: Contact a taxi company and arrange for a taxi to your current location.

[0447] Example of a prompt

[0448] The following prompt statements can be used to generate a detailed explanation of the program's processing using a generative AI model.

[0449] Point down

[0450] We have developed a system that suggests efficient and economical routes home for users who have missed the last train. This system incorporates an emotion engine that recognizes the user's emotions. Please explain in detail how this system works, including specific processing steps and actions. Also, please include specific user actions and system responses in your explanation.

[0451] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0452] Step 1:

[0453] User: Launch the app and press the "Find Current Location" button using the GPS function. If GPS is enabled, your current location will be automatically acquired. If GPS is unavailable, you will be taken to the "Manual Address Input" screen and can manually enter your address.

[0454] Input: Location data from GPS, or manually entered address.

[0455] Output: User's current location information.

[0456] Specific operation: The app performs an operation to obtain the current location. If GPS is enabled, location information is obtained automatically. If GPS is disabled, the user enters an address.

[0457] Step 2:

[0458] User: Enter your home address in the address input field within the app and press the "Set Destination" button.

[0459] Input: Address data entered by the user.

[0460] Output: Destination information.

[0461] Specific operation: The user enters their home address in the address input field within the app and presses the submit button, registering the destination information in the system.

[0462] Step 3:

[0463] Terminal: Based on the current location information, it queries the public transport API to obtain information about the nearest public transport. At this time, it also calculates the distance and travel time from the current location to the nearest station.

[0464] Input: Current location information.

[0465] Output: Information on the nearest public transport, distance, and travel time.

[0466] Specific operation: The device sends its current location information to the public transport API to obtain information about the nearest train station or bus stop. Furthermore, it calculates the travel time and distance from the current location to the nearest public transport.

[0467] Step 4:

[0468] Device: Activates the camera and microphone and performs "facial expression analysis" and "voice analysis." Captures the user's face with the camera and sends the data to the emotion analysis engine.

[0469] Input: Face image captured by camera, voice recorded by microphone.

[0470] Output: User sentiment data.

[0471] Specific operation: The camera and microphone are used to collect the user's facial expressions and voice, and send them to an emotion analysis engine. As a result of the analysis, the user's emotional state is determined, such as "fatigue," "stress," or "relaxation."

[0472] Step 5:

[0473] Server: Receives current location information, destination information, nearest station information, last train time, and sentiment data sent from the terminal.

[0474] Input: Current location information, destination information, nearest station information, last train time, sentiment data.

[0475] Output: Integrated data of various types of information.

[0476] Specific operation: Integrates all the information sent from the terminal and stores it in a database. It also retrieves the last train time from the public transport database and compares it with the current time.

[0477] Step 6:

[0478] Server: If you have missed the last train, calculate several possible routes home. For each route, calculate the time required, cost, and distance traveled.

[0479] Input: Last train time, current time, destination information.

[0480] Output: Multiple travel route options, estimated travel time, cost, and distance for each route.

[0481] Specific actions: Compare the current time with the last train time, and if the last train has been missed, suggest multiple routes home. Calculate the travel time, cost, and distance for each route.

[0482] Step 7:

[0483] Server: Based on the results of sentiment analysis, the server selects the optimal route considering the user's emotional state. If the user is tired, priority is given to taking a taxi home.

[0484] Input: Candidate travel routes, sentiment analysis data.

[0485] Output: An optimal travel route based on emotions.

[0486] Specific operation: Based on the emotion analysis results, the system selects and adjusts the optimal travel route that reflects the user's emotional state.

[0487] Step 8:

[0488] Terminal: Displays a list of multiple routes sent from the server. Visually displays the estimated time, cost, distance, and sentiment considerations for each route.

[0489] Input: Route data sent from the server.

[0490] Output: A list of routes presented to the user.

[0491] Specific operation: Visually display detailed data for each route and present it in a list format for easy comparison by the user.

[0492] Step 9:

[0493] User: Select the best route from the multiple routes presented and press the "Select" button on the screen.

[0494] Input: User-selected route.

[0495] Output: Selected travel route.

[0496] Specific operation: The user selects the optimal route from the list of routes on the screen and presses the "Select" button.

[0497] Step 10:

[0498] Terminal: Performs necessary actions (such as calling a taxi or making a reservation) based on the travel route selected by the user. If calling a taxi, it contacts a taxi company and arranges for a taxi to be dispatched to the user's current location.

[0499] Input: The travel route selected by the user.

[0500] Output: Action performed (e.g., arranging a taxi).

[0501] Specific operation: Based on the route selected by the user, the device automatically performs the necessary actions. If a taxi is requested, it will automatically contact a taxi company and arrange for a taxi to be dispatched to the user's current location.

[0502] (Application Example 2)

[0503] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0504] In modern society, it is particularly difficult to return home efficiently and economically if one misses the last train. However, there is a need not only to provide transportation, but also to propose the optimal way to return home that takes into account the user's emotional state. This invention aims to solve this problem by proposing a system that provides a comfortable and efficient way to return home, taking into account the emotional state of the user when they miss the last train.

[0505] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0506] In this invention, the server includes means for acquiring the user's current location information, means for acquiring the user's destination information, means for acquiring information on the nearest public transportation based on the current location, means for calculating candidate travel routes available when the time for the last train has passed, means for calculating the time, cost, and distance for each travel route, means for recognizing the user's emotional state and adjusting the travel route based on the emotional state, means for prioritizing the presentation of comfortable travel options when the user is fatigued based on the emotional state, means for presenting the user with multiple travel routes, means for performing necessary actions based on the travel route selected by the user, and means for summoning an autonomous vehicle and providing a means of transportation from the user's current location to their destination. This enables the provision of an optimal way to return home that takes into account the user's emotional state, allowing for a comfortable and efficient return home.

[0507] A "user" is an individual who uses the system to input their current location and destination information and receive suggestions for the optimal route home.

[0508] "Current location information" refers to the location information of the user's current location when using the system.

[0509] "Destination information" refers to the location information of the place the user wants to return to using the system.

[0510] "Public transport" refers to means of transportation such as trains, buses, and ferries that operate according to specific routes and schedules.

[0511] "Last train" refers to the time when public transportation services cease operations for the day.

[0512] A "travel route" refers to the path or method a user takes to travel from their current location to their destination.

[0513] "Travel time" refers to the time it takes to reach a destination using a specific travel route.

[0514] "Expenses" refer to the financial expenditure necessary to use a particular travel route.

[0515] "Distance traveled" refers to the actual distance traveled along a specific route.

[0516] "Emotional state" refers to the user's psychological or physical condition, including, for example, fatigue, stress, and relaxation.

[0517] An "emotion engine" is a program that recognizes and analyzes a user's emotional state.

[0518] An "action" is a specific operation or procedure performed based on the travel route selected by the user.

[0519] An "autonomous vehicle" is a vehicle that does not require a driver and is operated automatically by a computer.

[0520] This invention is a system that suggests an efficient and economical route home when a user misses the last train, and it recognizes the user's emotional state to provide the optimal way to get home. This system uses a smartphone as its main interface and combines an emotion recognition engine with an autonomous vehicle API to suggest a route home that meets the user's individual needs.

[0521] The system consists of the following elements:

[0522] 1. Obtaining user information

[0523] The user launches a smartphone application and enters their current location and destination information. Current location information is either automatically obtained using GPS or entered manually. Destination information is also entered manually by the user.

[0524] 2. Recognition of emotional states

[0525] The system uses the smartphone's camera and microphone to analyze the user's facial expressions and voice tone to recognize their emotional state. This process utilizes software called EmotionRecognition.

[0526] 3. Calculation of the optimal route

[0527] Based on your current location, destination, and emotional state, the system calculates the optimal route home. The calculated route includes factors such as travel time, cost, distance, and comfort level.

[0528] 4. Provision of transportation

[0529] Based on the route calculated and determined to be the most suitable by the user, an autonomous vehicle is dispatched. The SelfDrivingCarAPI is used to select the appropriate autonomous vehicle and dispatch it to the user's specified current location.

[0530] Specifically, the system operates in the following scenario:

[0531] scenario:

[0532] If a user is at Shinjuku Station, has missed the last train, and is trying to return home to Naka Ward, Yokohama City, the user's smartphone app is launched, retrieves "Shinjuku Station" as the current location, and inputs "Naka ​​Ward, Yokohama City" as the destination. If the emotion recognition engine determines from the user's facial expression and voice that they are "tired," the system prioritizes displaying a comfortable route home.

[0533] For example, routes such as "traveling by train from Shinjuku to Yokohama and then taking a taxi from there" or "taking a taxi directly home from Shinjuku" are suggested. If the user chooses the latter, an autonomous vehicle will pick them up at Shinjuku Station and drop them off at their home in Naka Ward, Yokohama City.

[0534] The specific software and hardware used to perform the above processes are as follows:

[0535] Software used:

[0536] EmotionRecognition (emotion recognition engine)

[0537] SelfDrivingCarAPI (Autonomous Vehicle API)

[0538] Hardware used:

[0539] Smartphone (acquisition of user's current location information and sentiment recognition)

[0540] Autonomous vehicles (providing transportation to the user's destination)

[0541] Prompt example:

[0542] Create an app that, when a user misses the last train, will summon an autonomous vehicle and suggest an efficient and economical route home. The app will obtain the user's current location and destination, and calculate the optimal route based on the user's emotional state. Specifically, it will prioritize displaying comfortable routes that take into account the user's fatigue level, and summon an autonomous taxi if necessary.

[0543] In this way, the present invention can provide an optimal method of returning home that takes into account the user's emotional state, making it possible to return home comfortably and efficiently.

[0544] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0545] Step 1:

[0546] The user launches the application on their smartphone and enters their current location and destination information.

[0547] Input: User's current location (GPS information or manual input) and destination (manual input).

[0548] Operation: The app automatically obtains the current location using GPS, or the user manually enters the current location and destination.

[0549] Step 2:

[0550] The device retrieves information about the nearest public transportation based on the user's current location.

[0551] Input: Current location information.

[0552] Function: Searches the public transport database for the nearest train station or bus stop and retrieves that information.

[0553] Output: Information on the nearest public transport.

[0554] Step 3:

[0555] The terminal retrieves the last train times from the public transport database.

[0556] Input: Information about the nearest public transportation.

[0557] Operation: Retrieves the last train time from the database.

[0558] Output: Last train time.

[0559] Step 4:

[0560] The device uses the smartphone's camera and microphone to recognize the user's emotional state.

[0561] Input: User's facial expressions and voice data.

[0562] Operation: Using EmotionRecognition (an emotion recognition engine), it analyzes the user's facial expressions and voice tone to identify their emotional state.

[0563] Output: User's emotional state (fatigue, stress, relaxation, etc.).

[0564] Step 5:

[0565] The server compares the current time with the time of the last train to determine whether the user has missed the last train.

[0566] Input: Current location information, last train time.

[0567] Operation: Compares the current time with the time of the last train to determine if the last train has been missed.

[0568] Output: The result of determining whether you missed the last train.

[0569] Step 6:

[0570] The server calculates possible travel routes available if the last train is missed.

[0571] Input: Current location information, destination information, emotional state.

[0572] Function: Calculates the time, cost, and distance traveled for each travel route (walking, train + walking, train + taxi, direct taxi).

[0573] Output: Candidate travel routes.

[0574] Step 7:

[0575] The server adjusts and suggests the optimal travel route based on the user's emotional state.

[0576] Input: Candidate travel routes, emotional state.

[0577] Operation: Depending on the user's emotional state, for example, if they are fatigued, the system will prioritize displaying comfortable routes. Other parameters (such as travel time, cost, and distance) will also be considered and adjusted accordingly.

[0578] Output: Optimal travel route.

[0579] Step 8:

[0580] The server presents the user with multiple optimal travel routes.

[0581] Input: Optimal travel route.

[0582] Operation: Visually displays detailed route information (estimated time, cost, distance, comfort level, etc.) on the app screen.

[0583] Output: A visually presented travel route.

[0584] Step 9:

[0585] The system performs the necessary actions based on the travel route selected by the user.

[0586] Input: The route selected by the user.

[0587] Operation: Depending on the user's selection, it will perform actions such as calling an autonomous vehicle, arranging a taxi, or booking accommodation.

[0588] Output: Actions performed (e.g., arranging an autonomous vehicle, booking a taxi, booking accommodation).

[0589] Step 10:

[0590] The device notifies the user that the action has been completed.

[0591] Input: The result of the action performed.

[0592] Action: The app notifies the user that the action is complete.

[0593] Output: Notification of action completion.

[0594] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0595] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0596] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0597] [Second Embodiment]

[0598] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0599] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0600] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0601] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0602] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0603] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0604] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0605] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0606] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0607] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0608] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0609] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0610] ---

[0611] This invention is a system that suggests an efficient and economical route home for users who have missed the last train. Embodiments of this invention will be described in detail below.

[0612] ---

[0613] System Overview

[0614] This system suggests the most efficient way to get home based on the user's current location and destination information. Specifically, it calculates and presents candidate routes that combine information on public transport, taxis, and walking.

[0615] ---

[0616] Program Processing Description

[0617] 1. Obtaining user information

[0618] User: Launch the app and use the automatic location acquisition function to determine your current location. If GPS functionality is unavailable, manually enter your address.

[0619] User: Enter your home address.

[0620] 2. Obtain the nearest station and the last train time.

[0621] Terminal: Searches for the nearest station from the current location and retrieves the last train time for that station from the public transport database.

[0622] 3. Proposed solutions for when you miss the last train.

[0623] Server: Check if the time for the last train has passed the current time.

[0624] Server: If you miss the last train, calculate the following route.

[0625] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[0626] Server: Searches for the nearest other train station from the current location to the home address and calculates the walking route from there.

[0627] Route 2: Go as far as you can by train, then take a taxi home.

[0628] Server: Calculates the furthest point reachable by train and estimates the taxi fare from there to home.

[0629] Route 3: Walk home entirely.

[0630] Server: Calculates the walking route from the current location to home and displays the estimated time.

[0631] Route 4: Go home entirely by taxi.

[0632] Server: Estimate the taxi fare from the current location to home.

[0633] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[0634] Server: Searches for entertainment and hotels near the current location and displays reservation information.

[0635] 4. Route Optimization and Suggestion

[0636] Server: Calculates the required time, cost, and distance for each route, and calculates the optimal route based on the conditions set by the user.

[0637] Server: Generates a route list in order of priority and sends it to the terminal along with detailed information.

[0638] 5. Providing directions to the user

[0639] Terminal: Displays a list of suggested routes and visually shows the details of each route (estimated time, cost, distance).

[0640] 6. User selection and action execution

[0641] User: Select the most suitable route from the displayed options and press the "Select" button on the screen.

[0642] Terminal: Based on the user's selection, it displays options for performing necessary actions (calling a taxi, making a reservation for entertainment).

[0643] User: Select the appropriate action (e.g., call a taxi) and follow the on-screen instructions.

[0644] Terminal: Performs the specified action, for example, automatically contacting a taxi company.

[0645] ---

[0646] Specific example:

[0647] scenario:

[0648] User: I'm at Shinjuku Station.

[0649] Destination: My home in Naka Ward, Yokohama City.

[0650] Terminal: Confirm that Shinjuku Station is the nearest station and check the last train time (e.g., 23:30) obtained from the public transport database.

[0651] Server: Confirming that you have missed the last train at the current time (e.g., 23:45), the following route is suggested.

[0652] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[0653] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[0654] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[0655] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[0656] Route 5: Stay at a hotel near Shinjuku and return home on the first train the next day (hotel fee: 3000 yen).

[0657] User: Select "Route 2" and initiate a taxi call on the device.

[0658] Terminal: Contact a taxi company and arrange for a taxi to your current location.

[0659] Thus, the present invention provides users who have missed the last train with the most suitable way to get home, thereby reducing the burden on users.

[0660] The following describes the processing flow.

[0661] Step 1:

[0662] The user launches the app and uses the automatic location acquisition function to determine their current location. If GPS functionality is unavailable, the user manually enters their current address.

[0663] Step 2:

[0664] The user enters their home address. This allows the destination information to be identified.

[0665] Step 3:

[0666] The device searches for the nearest public transport station based on the user's current latitude and longitude. This identifies the user's nearest station.

[0667] Step 4:

[0668] The terminal retrieves the last train time for the nearest station from the public transport database. This database contains operational information for each station.

[0669] Step 5:

[0670] The server checks the current time and compares it to the last train time of the nearest station. It then determines whether the current time is past the last train time.

[0671] Step 6:

[0672] If the server determines that you have missed the last train, it will calculate several possible routes home. Specifically, these include the following routes:

[0673] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[0674] The server searches for the nearest other train station from your current location to your home address and calculates the walking route from there.

[0675] Route 2: Go as far as you can by train, then take a taxi home.

[0676] The server calculates the furthest point you can reach by train and then estimates the taxi fare from there to your home.

[0677] Route 3: Walk home entirely.

[0678] The server calculates the walking route from the current location to home and displays the estimated time.

[0679] Route 4: Go home entirely by taxi.

[0680] The server estimates the taxi fare from your current location to your home.

[0681] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[0682] The server searches for entertainment and hotels near the current location and displays reservation information.

[0683] Step 7:

[0684] The server calculates the time, cost, and distance for each route, and then calculates the optimal route based on the conditions set by the user (e.g., "fastest" or "most economical").

[0685] Step 8:

[0686] The server generates a route list in order of priority and sends it to the terminal, along with detailed information.

[0687] Step 9:

[0688] The device displays a list of suggested routes and visually shows the details of each route (estimated time, cost, and distance).

[0689] Step 10:

[0690] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen.

[0691] Step 11:

[0692] The device displays options to perform the necessary action (call a taxi, make a reservation for entertainment) based on the user's selection.

[0693] Step 12:

[0694] The user selects the appropriate action (for example, calling a taxi) and proceeds by following the on-screen instructions.

[0695] Step 13:

[0696] The device performs a specified action, for example, automatically contacting a taxi company.

[0697] Step 14:

[0698] The server monitors the route and actions selected by the user, and updates route information or suggests alternative routes as needed.

[0699] Step 15:

[0700] The device tracks the user's current location and displays an alert if they get lost or need further assistance.

[0701] Step 16:

[0702] The user finally arrives home and chooses to close the app.

[0703] (Example 1)

[0704] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0705] One challenge is that it is difficult for users who miss the last train to find an efficient and economical way to get home. Another challenge is the lack of a system that simultaneously evaluates travel time, cost, and distance to provide the optimal route when users are looking for a way to get home.

[0706] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0707] In this invention, the server includes means for acquiring the user's current location information, means for acquiring the user's destination information, means for acquiring information on the nearest public transportation based on the current location, means for calculating candidate routes available when the last train has departed, means for calculating the time, cost, and distance for each route, means for presenting multiple routes to the user, means for performing necessary actions based on the route selected by the user, means for calculating walking routes based on the proposed routes, means for estimating taxi fares based on the proposed routes, and means for searching for and displaying accommodation information based on the proposed routes. This makes it possible to efficiently and economically provide the optimal route home to a user who has missed the last train.

[0708] A "user" refers to an individual who uses the system to search for a route home.

[0709] "Current location information" refers to geographical data indicating the user's current location.

[0710] "Destination information" refers to geographical data about the location where the user is aiming to return home.

[0711] "Public transportation information" refers to timetables and operating status information for modes of transport such as trains and buses.

[0712] "Last train time" refers to the departure time and service information of the last train running on that day.

[0713] "Possible travel routes" refer to multiple routes that a user can choose to take to get home.

[0714] A "walking route" refers to the path a user takes to reach their destination on foot.

[0715] "Taxi fare estimate" refers to a predicted value of the fare incurred when using a taxi.

[0716] "Accommodation facilities" refer to facilities that users can use for temporary lodging (such as hotels and entertainment venues).

[0717] "Action" refers to a specific action taken as part of the means of getting home (e.g., calling a taxi, booking accommodation).

[0718] A "server" refers to a computer system that processes data and provides information based on requests from users.

[0719] "Terminal" refers to a device (such as a smartphone or tablet) that a user uses to access the system.

[0720] This invention is a system that suggests an efficient and economical route home for users who have missed the last train. The embodiments of this invention will be described in detail below.

[0721] This system is implemented through various operations performed by servers, terminals, and users. The hardware used includes terminals such as smartphones and tablets, while small computers and servers perform the main processing. The software used includes GPS functionality for acquiring location information and APIs (e.g., Google Maps API, NAVITIME, etc.) for acquiring public transportation information.

[0722] User information retrieval

[0723] The user launches the app on their device and enables automatic location acquisition. The device uses its GPS function to determine its current location and sends the location information to the server. If GPS is unavailable, the user can also manually enter their address. Furthermore, the user enters their destination information (where they will return home) into their device.

[0724] Obtain the nearest station and the last train time.

[0725] The device sends its current location information to the server. The server uses an API to retrieve public transportation information, specifically the nearest station and its last train time. This retrieved information is sent to the device and displayed to the user.

[0726] Suggestions for what to do if you miss the last train.

[0727] The server checks if the user has missed the last train by comparing the retrieved last train time with the current time. If the user has missed the last train, it calculates the following possible travel routes.

[0728] Route 1: Take the train to a station that is not the closest but is nearby, and then walk home from there. The server searches for the nearest other station from your current location to your destination and calculates the walking route.

[0729] Route 2: Go as far as you can by train, then take a taxi home. The server calculates the furthest point you can reach by train and estimates the taxi fare from that point to your destination.

[0730] Route 3: Walk home entirely. The server calculates the walking route from the current location to the destination and compiles the estimated time.

[0731] Route 4: Take a taxi home. The server estimates the taxi fare from your current location to your destination.

[0732] Route 5: Stay at a nearby accommodation and return home the next morning. The server searches for accommodations near your current location and displays reservation information.

[0733] Route optimization and suggestions

[0734] The server comprehensively calculates the time, cost, and distance for each potential travel route and selects the optimal route based on user-defined conditions (e.g., minimizing costs, prioritizing time). Finally, the server generates a route list in order of priority and sends it to the terminal along with detailed information.

[0735] Providing route guidance to users

[0736] The device displays the route list received from the server on the app screen. Details of each route (estimated time, cost, distance) are displayed visually, allowing the user to scroll and review them.

[0737] User selection and action execution

[0738] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen. Based on this selection, the terminal suggests and executes the necessary actions (e.g., calling a taxi, making a hotel reservation). Once the user completes the operation, the terminal automatically performs the instructed actions and contacts the taxi company or accommodation.

[0739] Specific example

[0740] scenario:

[0741] User: I'm at Shinjuku Station.

[0742] Destination: My home in Naka Ward, Yokohama City.

[0743] Terminal:

[0744] The nearest station is Shinjuku Station, and the last train time (e.g., 23:30) is checked using the public transport database. The server confirms that the last train has been missed from the current time (e.g., 23:45) and suggests the following route.

[0745] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[0746] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[0747] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[0748] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[0749] Route 5: Stay at an accommodation near Shinjuku and return home on the first train the next day (accommodation fee: 3000 yen).

[0750] user:

[0751] Select "Route 2" and then use the terminal to call a taxi.

[0752] Terminal:

[0753] The system contacts a taxi company and arranges for a taxi to be dispatched to the user's current location. This ensures that users have an efficient and economical way to get home, even if they miss the last train.

[0754] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0755] System program processing flow

[0756] Step 1:

[0757] The user launches the app on their smartphone or tablet. When the user presses the "Automatically acquire current location" button, the device enables its GPS function and acquires the current location. GPS data is input, and the current location information is output. Specifically, the device acquires data from the GPS sensor and displays it as the current location.

[0758] Step 2:

[0759] The user manually enters their current address (if GPS is unavailable). Once the user enters the address and presses the "Confirm" button, the manually entered address is output as the current location information. Specifically, the user's input is retrieved in text format and saved to an internal database.

[0760] Step 3:

[0761] The user enters their home address. Once the address is entered into the terminal's input form and the "Next" button is pressed, destination information is output. Specifically, the user's input is retrieved in text format and saved to an internal database.

[0762] Step 4:

[0763] The device sends its current location and destination information to the server. The current location and destination information are used as input, and the server receives this information as output. Specifically, the device sends data to the server via the internet.

[0764] Step 5:

[0765] The server accesses a public transport API (e.g., Google Maps API) to retrieve the nearest station and last train time. Current location information is used as input, and the output is data on the nearest station and last train time. Specifically, the server sends an API request and saves the received data to an internal database.

[0766] Step 6:

[0767] The server compares the current time with the last train time to determine if the user has missed the last train. The current time and last train time are used as input, and the output is a determination of whether the user missed the last train. Specifically, the server executes an algorithm to compare the current time with the last train time.

[0768] Step 7:

[0769] If you miss the last train, the server calculates possible travel routes. Current location information, destination information, and public transport information are used as input, and multiple travel routes are returned as output. Specifically, the following routes are calculated:

[0770] Route 1: Take the train to a station that is not the closest but is nearby from your current location, and then walk home from there.

[0771] The server searches for the nearest station from the current location and calculates the walking route. The output provides information about the station and the walking route.

[0772] Route 2: Go as far as you can by train, then take a taxi home.

[0773] The server calculates the final destination reachable by train and estimates the taxi fare. The output provides the final destination and the estimated taxi fare.

[0774] Route 3: Walk home entirely.

[0775] The server calculates the walking route from the current location to the destination. The output includes the walking route and the estimated time.

[0776] Route 4: Go home entirely by taxi.

[0777] The server estimates the taxi fare from the current location to the destination. The taxi fare is returned as output.

[0778] Route 5: Stay at a nearby accommodation and return home the next day.

[0779] The server searches for accommodations near the current location and displays a list of accommodations and reservation information. Accommodation information is provided as output.

[0780] Step 8:

[0781] The server calculates the time, cost, and distance for each route and selects the optimal route based on user-defined conditions. Route candidate data is used as input, and the output is a list of optimal routes. Specifically, the server executes a route evaluation algorithm and calculates the evaluation results.

[0782] Step 9:

[0783] The server sends the optimal route list to the terminal. The route list is used as input, and the terminal receives the route list as output. Specifically, the server sends data over the internet, and the terminal receives it.

[0784] Step 10:

[0785] The app displays the route list received by the device on the app screen. It visually displays the estimated time, cost, and distance for each route so the user can review the route details. Specifically, the device generates the display data and renders it on the screen.

[0786] Step 11:

[0787] The user selects a route and presses the "Select" button. The selected route information is used as input, and the selection result is obtained as output. Specifically, the user's selection is received and sent to the server.

[0788] Step 12:

[0789] The device performs actions based on the user's selections (e.g., calling a taxi, making a hotel reservation). User selection information is used as input, and the output is the result of executing a specific action. Specifically, the device calls the corresponding API to contact the taxi or accommodation provider.

[0790] (Application Example 1)

[0791] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0792] While systems already exist that suggest efficient and economical routes home for users who miss the last train, there is a similar need for technology that calculates efficient and economical delivery routes in the food delivery industry. In particular, since reducing time and costs is a critical issue in the food delivery industry, a system that can provide efficient delivery routes is essential.

[0793] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0794] In this invention, the server includes means for acquiring the user's current location information, means for acquiring destination information, means for acquiring information on the nearest public transport, means for calculating candidate available travel routes, means for calculating the time, cost, and distance for each travel route, means for presenting multiple travel routes, means for performing necessary actions based on the travel route selected by the user, and means for calculating an efficient delivery route for food delivery. This makes it possible to provide the optimal travel route in terms of both time and cost.

[0795] "User" refers to an individual or group that uses the system.

[0796] "Current location information" refers to the latitude and longitude data of the user's current location.

[0797] "Destination information" refers to data such as the address and latitude / longitude of the place the user is heading to.

[0798] "Public transportation" refers to mass transportation methods such as buses, trains, and subways.

[0799] "Possible travel routes" refer to multiple options for routes and means of transportation to reach a destination.

[0800] "Travel time" refers to the time required to travel from your current location to your destination.

[0801] "Cost" refers to the economic cost required to travel from your current location to your destination.

[0802] "Distance traveled" refers to the physical distance from your current location to your destination.

[0803] "Food delivery" refers to a service that delivers food and beverages to a specified location.

[0804] An "efficient delivery route" is a route optimized to minimize delivery time and costs.

[0805] "Means of performing actions" refer to methods such as calling a car or taxi or booking accommodation based on the travel route selected by the user.

[0806] This invention is a system that proposes efficient and economical delivery routes for food delivery. Based on the user's current location and destination information, this system calculates and presents a delivery route combining multiple modes of transportation to the user. Embodiments of this invention will be described in detail below.

[0807] Server Processing

[0808] The server first obtains the user's current location and destination information. Current location information is obtained using GPS or an IP geolocation service. Destination information is obtained using the Google Maps API from the address entered by the user, providing latitude and longitude. This ensures the server has accurate location information for both the current and destination locations.

[0809] Next, the server retrieves information about the nearest public transport. To do this, it consults a public transport database to obtain the nearest train station or bus stop and the time of the last train. If the last train has already departed, the server calculates route options for each mode of transportation (bicycle, public transport, walking).

[0810] Route options for each mode of transport are calculated based on their average speed, determining the travel time, cost, and distance. For example, a standard of 15 km / h is used for cycling, 30 km / h for public transport, and 5 km / h for walking. In this way, the server can obtain detailed information for all included routes.

[0811] Terminal processing

[0812] The terminal visually displays multiple route options sent from the server to the user. Each route clearly indicates the estimated travel time, cost, and distance, and presents the user with its advantages and disadvantages. Based on this information, the user can choose the most suitable mode of transportation.

[0813] Based on the route selected by the user, the device performs the necessary actions. For example, if the user requests a taxi or makes a hotel reservation, the device automatically arranges these using the corresponding API.

[0814] Specific example

[0815] User's current location: Shibuya Station

[0816] Destination: Restaurants near Ebisu Station

[0817] Optimal route: Travel by bicycle (approximately 10 minutes)

[0818] In this specific example, if the user is at Shibuya Station, the server retrieves the user's current and destination location information and selects a bicycle as the most efficient route. This bicycle route is calculated to take approximately 10 minutes and is then presented to the user.

[0819] Example of a prompt

[0820] An example of a prompt statement to be input to a generative AI model is as follows:

[0821] text

[0822] Develop an application that suggests the most efficient route for delivery drivers when a user orders food delivery. Please use the following information as a basis.

[0823] Current location: Shibuya Station

[0824] Destination: Restaurants near Ebisu Station

[0825] Available modes of transportation: bicycle, public transport, walking

[0826] Calculate the travel time and distance for each mode of transportation and suggest the optimal route.

[0827] This allows users to obtain efficient routes in real time and reach their destinations quickly and economically.

[0828] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0829] Step 1:

[0830] The server obtains the user's current location information. Using GPS functionality or an IP geolocation service, it obtains the latitude and longitude of the user's current location from the user's device. This allows the server to obtain the user's precise location information as input data. The output data is the latitude and longitude information of the current location.

[0831] Step 2:

[0832] The user enters the destination address. The device retrieves this entered address data and uses the Google Maps API to obtain the corresponding latitude and longitude. The server receives this latitude and longitude information as input data and stores it as destination information. The output data is the latitude and longitude information of the destination.

[0833] Step 3:

[0834] The server retrieves information about the nearest public transport based on the user's current location. It consults a public transport database to obtain the nearest train station or bus stop, as well as the time of the last train or bus. The server receives this information as input data and stores it as information about the nearest public transport. The output data includes the name, location, and time of the last train or bus of the nearest public transport.

[0835] Step 4:

[0836] The server calculates possible travel routes. It calculates route options for each mode of transport (bicycle, public transport, walking) from the current location to the destination, and calculates the estimated time, cost, and distance for each. The server receives these calculation results as input data and stores them as travel route options. The output data consists of the estimated time, cost, and distance for each mode of transport.

[0837] Step 5:

[0838] The server presents the user with the results of each calculated travel route. The terminal visually displays the detailed information (estimated time, cost, distance) of each travel route received from the server. The user selects the optimal route based on this information. The input data is the route candidate information received from the server, and the output data is the route details displayed on the terminal.

[0839] Step 6:

[0840] The user selects the optimal route from several presented travel routes. The terminal receives the user's selection and sends it to the server. The input data is the route information selected by the user, and the output data is the selected route information sent to the server.

[0841] Step 7:

[0842] The server performs the necessary actions based on the travel route selected by the user. For example, it automatically arranges things like calling a taxi or booking accommodation using APIs corresponding to each route. The input data is the route information selected by the user, and the output data is the result of each arrangement action.

[0843] This allows users to obtain efficient and economical travel routes and take appropriate actions quickly.

[0844] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0845] ---

[0846] This invention combines a system that suggests efficient and economical routes home when a user misses the last train with an emotion engine that recognizes the user's emotions. Based on the user's current location and destination information, the system suggests the most suitable way home. By utilizing the emotion engine, the suggested travel route can be adjusted according to the user's emotional state.

[0847] ---

[0848] System Overview

[0849] This system acquires the user's current location and destination information and suggests the optimal route to get home efficiently and economically, even if the user misses the last train. It also recognizes the user's emotional state and adjusts the travel route based on those emotions.

[0850] ---

[0851] Program Processing Description

[0852] 1. Obtaining user information

[0853] User: Launch the app and use the automatic location acquisition function to determine your current location. If GPS functionality is unavailable, manually enter your address.

[0854] User: Enter your home address. This will identify your destination.

[0855] 2. Obtain the nearest station and the last train time.

[0856] Terminal: Searches for the nearest station from the current location and retrieves the last train time for that station from the public transport database.

[0857] 3. Recognizing the user's emotional state

[0858] Device: The app uses its in-app camera and microphone to recognize emotions from the user's facial expressions and tone of voice.

[0859] Device: The emotion engine analyzes the user's emotional state, identifying things like fatigue, stress, and relaxation.

[0860] 4. Proposed solutions for when you miss the last train.

[0861] Server: Check if the time for the last train has passed the current time.

[0862] Server: If you miss the last train, calculate the following route.

[0863] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[0864] Server: Searches for the nearest other train station from the current location to the home address and calculates the walking route from there.

[0865] Route 2: Go as far as you can by train, then take a taxi home.

[0866] Server: Calculates the furthest point reachable by train and estimates the taxi fare from there to home.

[0867] Route 3: Walk home entirely.

[0868] Server: Calculates the walking route from the current location to home and displays the estimated time.

[0869] Route 4: Go home entirely by taxi.

[0870] Server: Estimate the taxi fare from the current location to home.

[0871] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[0872] Server: Searches for entertainment and hotels near the current location and displays reservation information.

[0873] 5. Route Optimization and Suggestions

[0874] Server: Calculates the time, cost, and distance for each route, and adjusts the optimal route based on the user's emotional state.

[0875] Server: If the user is tired, prioritize suggesting a relaxing route; if they prioritize economical options, suggest a low-cost route.

[0876] Server: Generates a route list in order of priority and sends it to the terminal along with detailed information.

[0877] 6. Providing directions to the user

[0878] Terminal: Displays a list of each proposed route and visually shows the details of each route (estimated time, cost, distance, and emotional considerations).

[0879] 7. User selection and action execution

[0880] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen.

[0881] The device displays options to perform the necessary action (call a taxi, make a reservation for entertainment) based on the user's selection.

[0882] The user selects the appropriate action (for example, calling a taxi) and proceeds by following the on-screen instructions.

[0883] The device performs a specified action, for example, automatically contacting a taxi company.

[0884] ---

[0885] Specific example:

[0886] scenario:

[0887] User: I'm at Shinjuku Station.

[0888] Destination: My home in Naka Ward, Yokohama City.

[0889] User's emotions: Analysis of facial expressions and voice indicates that the user is tired.

[0890] Terminal: Confirm that Shinjuku Station is the nearest station and retrieve the last train time (e.g., 23:30) from the public transport database.

[0891] Server: Confirming that you have missed the last train at the current time (e.g., 23:45), the following route is suggested.

[0892] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[0893] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[0894] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[0895] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[0896] Route 5: Stay at a hotel near Shinjuku and return home on the first train the next day (hotel fee: 3000 yen).

[0897] Server: Since the server recognizes the user is tired, it prioritizes displaying the most comfortable and fastest "Route 4".

[0898] User: Select "Route 4" and initiate a taxi call on the device.

[0899] Terminal: Contact a taxi company and arrange for a taxi to your current location.

[0900] In this way, by utilizing the emotion engine, it is possible to provide the optimal way for users to return home according to their emotions, further reducing the burden on users.

[0901] The following describes the processing flow.

[0902] Step 1:

[0903] The user launches the app and uses the automatic location acquisition function to determine their current location. If GPS functionality is unavailable, the user manually enters their current address.

[0904] Step 2:

[0905] The user enters their home address. This allows the destination information to be identified.

[0906] Step 3:

[0907] The device searches for the nearest public transport station based on the user's current latitude and longitude. This identifies the user's nearest station.

[0908] Step 4:

[0909] The terminal retrieves the last train time for the nearest station from the public transport database. This database contains operational information for each station.

[0910] Step 5:

[0911] The server checks the current time and compares it to the last train time of the nearest station. It then determines whether the current time is past the last train time.

[0912] Step 6:

[0913] The device activates the user's facial recognition function and uses the built-in camera to capture the user's facial expressions. It also uses the voice recognition function to record what the user says and transfers this to emotion recognition software.

[0914] Step 7:

[0915] The server's emotion engine analyzes captured facial expressions and voice data to identify the user's emotional state (e.g., fatigue, stress, relaxation).

[0916] Step 8:

[0917] The server takes your emotional state into account and calculates the following return route. Specifically:

[0918] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[0919] The server searches for the nearest other train station from your current location to your home address and calculates the walking route from there.

[0920] Route 2: Go as far as you can by train, then take a taxi home.

[0921] The server calculates the furthest point you can reach by train and then estimates the taxi fare from there to your home.

[0922] Route 3: Walk home entirely.

[0923] The server calculates the walking route from the current location to home and displays the estimated time.

[0924] Route 4: Go home entirely by taxi.

[0925] The server estimates the taxi fare from your current location to your home.

[0926] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[0927] The server searches for entertainment and hotels near the current location and displays reservation information.

[0928] Step 9:

[0929] The server calculates the time, cost, and distance for each route and adjusts the optimal route based on the user's emotional state. If the user is highly fatigued, it prioritizes a comfortable and fast route.

[0930] Step 10:

[0931] The server generates a route list in order of priority and sends it to the terminal, along with detailed information.

[0932] Step 11:

[0933] The device displays a list of suggested routes and visually shows the details of each route (estimated time, cost, distance, and emotional considerations).

[0934] Step 12:

[0935] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen.

[0936] Step 13:

[0937] The device displays options to perform the necessary action (call a taxi, make a reservation for entertainment) based on the user's selection.

[0938] Step 14:

[0939] The user selects the appropriate action (for example, calling a taxi) and proceeds by following the on-screen instructions.

[0940] Step 15:

[0941] The device performs the specified action, for example, automatically contacting a taxi company. It also completes the booking process if a reservation is required.

[0942] Step 16:

[0943] The server monitors the route and actions selected by the user, and updates route information or suggests alternative routes as needed.

[0944] Step 17:

[0945] The device tracks the user's current location and displays an alert if they get lost or need further assistance.

[0946] Step 18:

[0947] After the user finally arrives home, they can choose to exit the app and then it will close.

[0948] (Example 2)

[0949] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0950] In modern urban life, missing the last train is a common occurrence, and finding an efficient and economical way to get home is not easy. Furthermore, the lack of consideration for the user's emotional state can increase fatigue and stress. Conventional systems have been unable to suggest appropriate routes that take the user's emotional state into account.

[0951] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0952] In this invention, the server includes means for acquiring the user's current location information, means for acquiring the user's destination information, and means for calculating multiple candidate travel routes. This makes it possible to propose the optimal travel route while taking into account the user's emotional state.

[0953] "Means of obtaining the user's current location information" refers to a function that determines the user's current location through GPS functionality or manual input by the user.

[0954] "Means for obtaining user destination information" refers to the function of inputting and obtaining information about the address or location set by the user as their destination.

[0955] "Means of obtaining information on the nearest public transport" refers to a function that searches for and retrieves the location and timetable of the nearest public transport based on the user's current location.

[0956] "Means for calculating multiple travel route options" refers to a function that calculates and suggests multiple travel routes that allow a user to reach their destination efficiently and economically, even if they miss the last train.

[0957] "Means for calculating the time, cost, and distance of each travel route" refers to a function that calculates in detail the time, cost, distance, etc., for each proposed travel route.

[0958] "Means of presenting multiple travel routes to the user" refers to a function that visually displays multiple calculated travel routes to the user, making it easier to compare and consider them.

[0959] "Means of performing necessary actions based on the travel route selected by the user" refers to a function that performs specific actions based on the travel route selected by the user, such as arranging a taxi or making a hotel reservation.

[0960] "A means of recognizing the user's emotional state and adjusting the travel route based on those emotions" refers to a function that uses cameras and microphones to analyze the user's emotions from their facial expressions and tone of voice, and then adjusts and suggests the optimal travel route according to those emotions.

[0961]

[0962] This invention is a system that suggests an efficient and economical route home when a user misses the last train, and further combines it with an emotion engine that recognizes the user's emotions to provide the optimal route according to the user's state. Specific embodiments of this invention are described below.

[0963] System Configuration

[0964] Hardware and software

[0965] 1. User's device:

[0966] Smartphone or tablet

[0967] Features include GPS, camera, and microphone.

[0968] 2. Server:

[0969] Servers with high-speed data processing capabilities

[0970] API that integrates with public transport databases

[0971] Emotion analysis engine

[0972] Specific names to use

[0973] GPS function: A standard geolocation API for obtaining the user's current location information.

[0974] Emotion analysis engine: OpenCV is used for facial expression analysis, and Azure Cognitive Services is used for speech analysis.

[0975] Public transport APIs: Google Maps API and public transport databases

[0976] Overview of program processing

[0977] 1. User's device:

[0978] Launch the app and use the GPS function to obtain your current location.

[0979] If the current location cannot be determined, the user will manually enter the address.

[0980] 2. User's device:

[0981] The user enters their home address as destination information.

[0982] 3. Terminal:

[0983] Use the public transport API to retrieve information about the nearest public transport.

[0984] Calculate the distance and travel time from your current location to the nearest station.

[0985] 4. Device:

[0986] The camera and microphone are activated to collect data on the user's face and voice.

[0987] The data is sent to an emotion analysis engine to analyze the user's emotional state (e.g., fatigue, stress, relaxation).

[0988] 5. Server:

[0989] Based on data transmitted from the device, it receives current location, destination, nearest station, last train time, and sentiment data.

[0990] The last train times are retrieved from the public transport database and compared with the current time.

[0991] 6. Server:

[0992] If you miss the last train, calculate multiple routes home.

[0993] For each route, calculate the required time, cost, and distance traveled.

[0994] 7. Server:

[0995] Based on the results of emotion analysis, the optimal route is selected. For example, if the user is tired, taking a taxi home is prioritized.

[0996] 8. Terminal:

[0997] Visually display multiple calculated routes. Detailed information for each route (estimated time, cost, distance, and emotional considerations) is displayed in a list.

[0998] 9. User:

[0999] Select the best route from the available options and press the "Select" button.

[1000] 10. Terminal:

[1001] Based on the selected route, the system will perform the necessary actions (such as calling a taxi or making a reservation for entertainment).

[1002] Specific example

[1003] scenario

[1004] User: I'm at Shinjuku Station.

[1005] Destination: My home in Naka Ward, Yokohama City.

[1006] User's emotions: Analysis of facial expressions and voice indicates that the user is tired.

[1007] Flow of operations

[1008] 1. Terminal: Confirm that Shinjuku Station is the nearest station and retrieve the last train time (e.g., 23:30) from the public transport database.

[1009] 2. Server: Confirms that the user has missed the last train from the current time (e.g., 23:45) and suggests the following route.

[1010] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[1011] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[1012] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[1013] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[1014] Route 5: Stay at a hotel near Shinjuku and return home on the first train the next day (hotel fee: 3000 yen).

[1015] 3. Server: Since the server recognizes that the user is tired, it prioritizes displaying the most comfortable and fastest "Route 4".

[1016] 4. User: Select "Route 4" and initiate a taxi call on the device.

[1017] 5. Terminal: Contact a taxi company and arrange for a taxi to your current location.

[1018] Example of a prompt

[1019] The following prompt statements can be used to generate a detailed explanation of the program's processing using a generative AI model.

[1020] Point down

[1021] We have developed a system that suggests efficient and economical routes home for users who have missed the last train. This system incorporates an emotion engine that recognizes the user's emotions. Please explain in detail how this system works, including specific processing steps and actions. Also, please include specific user actions and system responses in your explanation.

[1022] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1023] Step 1:

[1024] User: Launch the app and press the "Find Current Location" button using the GPS function. If GPS is enabled, your current location will be automatically acquired. If GPS is unavailable, you will be taken to the "Manual Address Input" screen and can manually enter your address.

[1025] Input: Location data from GPS, or manually entered address.

[1026] Output: User's current location information.

[1027] Specific operation: The app performs an operation to obtain the current location. If GPS is enabled, location information is obtained automatically. If GPS is disabled, the user enters an address.

[1028] Step 2:

[1029] User: Enter your home address in the address input field within the app and press the "Set Destination" button.

[1030] Input: Address data entered by the user.

[1031] Output: Destination information.

[1032] Specific operation: The user enters their home address in the address input field within the app and presses the submit button, registering the destination information in the system.

[1033] Step 3:

[1034] Terminal: Based on the current location information, it queries the public transport API to obtain information about the nearest public transport. At this time, it also calculates the distance and travel time from the current location to the nearest station.

[1035] Input: Current location information.

[1036] Output: Information on the nearest public transport, distance, and travel time.

[1037] Specific operation: The device sends its current location information to the public transport API to obtain information about the nearest train station or bus stop. Furthermore, it calculates the travel time and distance from the current location to the nearest public transport.

[1038] Step 4:

[1039] Device: Activates the camera and microphone and performs "facial expression analysis" and "voice analysis." Captures the user's face with the camera and sends the data to the emotion analysis engine.

[1040] Input: Face image captured by camera, voice recorded by microphone.

[1041] Output: User sentiment data.

[1042] Specific operation: The camera and microphone are used to collect the user's facial expressions and voice, and send them to an emotion analysis engine. As a result of the analysis, the user's emotional state is determined, such as "fatigue," "stress," or "relaxation."

[1043] Step 5:

[1044] Server: Receives current location information, destination information, nearest station information, last train time, and sentiment data sent from the terminal.

[1045] Input: Current location information, destination information, nearest station information, last train time, sentiment data.

[1046] Output: Integrated data of various types of information.

[1047] Specific operation: Integrates all the information sent from the terminal and stores it in a database. It also retrieves the last train time from the public transport database and compares it with the current time.

[1048] Step 6:

[1049] Server: If you have missed the last train, calculate several possible routes home. For each route, calculate the time required, cost, and distance traveled.

[1050] Input: Last train time, current time, destination information.

[1051] Output: Multiple travel route options, estimated travel time, cost, and distance for each route.

[1052] Specific actions: Compare the current time with the last train time, and if the last train has been missed, suggest multiple routes home. Calculate the travel time, cost, and distance for each route.

[1053] Step 7:

[1054] Server: Based on the results of sentiment analysis, the server selects the optimal route considering the user's emotional state. If the user is tired, priority is given to taking a taxi home.

[1055] Input: Candidate travel routes, sentiment analysis data.

[1056] Output: An optimal travel route based on emotions.

[1057] Specific operation: Based on the emotion analysis results, the system selects and adjusts the optimal travel route that reflects the user's emotional state.

[1058] Step 8:

[1059] Terminal: Displays a list of multiple routes sent from the server. Visually displays the estimated time, cost, distance, and sentiment considerations for each route.

[1060] Input: Route data sent from the server.

[1061] Output: A list of routes presented to the user.

[1062] Specific operation: Visually display detailed data for each route and present it in a list format for easy comparison by the user.

[1063] Step 9:

[1064] User: Select the best route from the multiple routes presented and press the "Select" button on the screen.

[1065] Input: User-selected route.

[1066] Output: Selected travel route.

[1067] Specific operation: The user selects the optimal route from the list of routes on the screen and presses the "Select" button.

[1068] Step 10:

[1069] Terminal: Performs necessary actions (such as calling a taxi or making a reservation) based on the travel route selected by the user. If calling a taxi, it contacts a taxi company and arranges for a taxi to be dispatched to the user's current location.

[1070] Input: The travel route selected by the user.

[1071] Output: Action performed (e.g., arranging a taxi).

[1072] Specific operation: Based on the route selected by the user, the device automatically performs the necessary actions. If a taxi is requested, it will automatically contact a taxi company and arrange for a taxi to be dispatched to the user's current location.

[1073] (Application Example 2)

[1074] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1075] In modern society, it is particularly difficult to return home efficiently and economically if one misses the last train. However, there is a need not only to provide transportation, but also to propose the optimal way to return home that takes into account the user's emotional state. This invention aims to solve this problem by proposing a system that provides a comfortable and efficient way to return home, taking into account the emotional state of the user when they miss the last train.

[1076] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1077] In this invention, the server includes means for acquiring the user's current location information, means for acquiring the user's destination information, means for acquiring information on the nearest public transportation based on the current location, means for calculating candidate travel routes available when the time for the last train has passed, means for calculating the time, cost, and distance for each travel route, means for recognizing the user's emotional state and adjusting the travel route based on the emotional state, means for prioritizing the presentation of comfortable travel options when the user is fatigued based on the emotional state, means for presenting the user with multiple travel routes, means for performing necessary actions based on the travel route selected by the user, and means for summoning an autonomous vehicle and providing a means of transportation from the user's current location to their destination. This enables the provision of an optimal way to return home that takes into account the user's emotional state, allowing for a comfortable and efficient return home.

[1078] A "user" is an individual who uses the system to input their current location and destination information and receive suggestions for the optimal route home.

[1079] "Current location information" refers to the location information of the user's current location when using the system.

[1080] "Destination information" refers to the location information of the place the user wants to return to using the system.

[1081] "Public transport" refers to means of transportation such as trains, buses, and ferries that operate according to specific routes and schedules.

[1082] "Last train" refers to the time when public transportation services cease operations for the day.

[1083] A "travel route" refers to the path or method a user takes to travel from their current location to their destination.

[1084] "Travel time" refers to the time it takes to reach a destination using a specific travel route.

[1085] "Expenses" refer to the financial expenditure necessary to use a particular travel route.

[1086] "Distance traveled" refers to the actual distance traveled along a specific route.

[1087] "Emotional state" refers to the user's psychological or physical condition, including, for example, fatigue, stress, and relaxation.

[1088] An "emotion engine" is a program that recognizes and analyzes a user's emotional state.

[1089] An "action" is a specific operation or procedure performed based on the travel route selected by the user.

[1090] An "autonomous vehicle" is a vehicle that does not require a driver and is operated automatically by a computer.

[1091] This invention is a system that suggests an efficient and economical route home when a user misses the last train, and it recognizes the user's emotional state to provide the optimal way to get home. This system uses a smartphone as its main interface and combines an emotion recognition engine with an autonomous vehicle API to suggest a route home that meets the user's individual needs.

[1092] The system consists of the following elements:

[1093] 1. Obtaining user information

[1094] The user launches a smartphone application and enters their current location and destination information. Current location information is either automatically obtained using GPS or entered manually. Destination information is also entered manually by the user.

[1095] 2. Recognition of emotional states

[1096] The system uses the smartphone's camera and microphone to analyze the user's facial expressions and voice tone to recognize their emotional state. This process utilizes software called EmotionRecognition.

[1097] 3. Calculation of the optimal route

[1098] Based on your current location, destination, and emotional state, the system calculates the optimal route home. The calculated route includes factors such as travel time, cost, distance, and comfort level.

[1099] 4. Provision of transportation

[1100] Based on the route calculated and determined to be the most suitable by the user, an autonomous vehicle is dispatched. The SelfDrivingCarAPI is used to select the appropriate autonomous vehicle and dispatch it to the user's specified current location.

[1101] Specifically, the system operates in the following scenario:

[1102] scenario:

[1103] If a user is at Shinjuku Station, has missed the last train, and is trying to return home to Naka Ward, Yokohama City, the user's smartphone app is launched, retrieves "Shinjuku Station" as the current location, and inputs "Naka ​​Ward, Yokohama City" as the destination. If the emotion recognition engine determines from the user's facial expression and voice that they are "tired," the system prioritizes displaying a comfortable route home.

[1104] For example, routes such as "traveling by train from Shinjuku to Yokohama and then taking a taxi from there" or "taking a taxi directly home from Shinjuku" are suggested. If the user chooses the latter, an autonomous vehicle will pick them up at Shinjuku Station and drop them off at their home in Naka Ward, Yokohama City.

[1105] The specific software and hardware used to perform the above processes are as follows:

[1106] Software used:

[1107] EmotionRecognition (emotion recognition engine)

[1108] SelfDrivingCarAPI (Autonomous Vehicle API)

[1109] Hardware used:

[1110] Smartphone (acquisition of user's current location information and sentiment recognition)

[1111] Autonomous vehicles (providing transportation to the user's destination)

[1112] Prompt example:

[1113] Create an app that, when a user misses the last train, will summon an autonomous vehicle and suggest an efficient and economical route home. The app will obtain the user's current location and destination, and calculate the optimal route based on the user's emotional state. Specifically, it will prioritize displaying comfortable routes that take into account the user's fatigue level, and summon an autonomous taxi if necessary.

[1114] In this way, the present invention can provide an optimal method of returning home that takes into account the user's emotional state, making it possible to return home comfortably and efficiently.

[1115] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1116] Step 1:

[1117] The user launches the application on their smartphone and enters their current location and destination information.

[1118] Input: User's current location (GPS information or manual input) and destination (manual input).

[1119] Operation: The app automatically obtains the current location using GPS, or the user manually enters the current location and destination.

[1120] Step 2:

[1121] The device retrieves information about the nearest public transportation based on the user's current location.

[1122] Input: Current location information.

[1123] Function: Searches the public transport database for the nearest train station or bus stop and retrieves that information.

[1124] Output: Information on the nearest public transport.

[1125] Step 3:

[1126] The terminal retrieves the last train times from the public transport database.

[1127] Input: Information about the nearest public transportation.

[1128] Operation: Retrieves the last train time from the database.

[1129] Output: Last train time.

[1130] Step 4:

[1131] The device uses the smartphone's camera and microphone to recognize the user's emotional state.

[1132] Input: User's facial expressions and voice data.

[1133] Operation: Using EmotionRecognition (an emotion recognition engine), it analyzes the user's facial expressions and voice tone to identify their emotional state.

[1134] Output: User's emotional state (fatigue, stress, relaxation, etc.).

[1135] Step 5:

[1136] The server compares the current time with the time of the last train to determine whether the user has missed the last train.

[1137] Input: Current location information, last train time.

[1138] Operation: Compares the current time with the time of the last train to determine if the last train has been missed.

[1139] Output: The result of determining whether you missed the last train.

[1140] Step 6:

[1141] The server calculates possible travel routes available if the last train is missed.

[1142] Input: Current location information, destination information, emotional state.

[1143] Function: Calculates the time, cost, and distance traveled for each travel route (walking, train + walking, train + taxi, direct taxi).

[1144] Output: Candidate travel routes.

[1145] Step 7:

[1146] The server adjusts and suggests the optimal travel route based on the user's emotional state.

[1147] Input: Candidate travel routes, emotional state.

[1148] Operation: Depending on the user's emotional state, for example, if they are fatigued, the system will prioritize displaying comfortable routes. Other parameters (such as travel time, cost, and distance) will also be considered and adjusted accordingly.

[1149] Output: Optimal travel route.

[1150] Step 8:

[1151] The server presents the user with multiple optimal travel routes.

[1152] Input: Optimal travel route.

[1153] Operation: Visually displays detailed route information (estimated time, cost, distance, comfort level, etc.) on the app screen.

[1154] Output: A visually presented travel route.

[1155] Step 9:

[1156] The system performs the necessary actions based on the travel route selected by the user.

[1157] Input: The route selected by the user.

[1158] Operation: Depending on the user's selection, it will perform actions such as calling an autonomous vehicle, arranging a taxi, or booking accommodation.

[1159] Output: Actions performed (e.g., arranging an autonomous vehicle, booking a taxi, booking accommodation).

[1160] Step 10:

[1161] The device notifies the user that the action has been completed.

[1162] Input: The result of the action performed.

[1163] Action: The app notifies the user that the action is complete.

[1164] Output: Notification of action completion.

[1165] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1166] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1167] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1168] [Third Embodiment]

[1169] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1170] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1171] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1172] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1173] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1174] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1175] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1176] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1177] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1178] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1179] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1180] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1181] ---

[1182] This invention is a system that suggests an efficient and economical route home for users who have missed the last train. Embodiments of this invention will be described in detail below.

[1183] ---

[1184] System Overview

[1185] This system suggests the most efficient way to get home based on the user's current location and destination information. Specifically, it calculates and presents candidate routes that combine information on public transport, taxis, and walking.

[1186] ---

[1187] Program Processing Description

[1188] 1. Obtaining user information

[1189] User: Launch the app and use the automatic location acquisition function to determine your current location. If GPS functionality is unavailable, manually enter your address.

[1190] User: Enter your home address.

[1191] 2. Obtain the nearest station and the last train time.

[1192] Terminal: Searches for the nearest station from the current location and retrieves the last train time for that station from the public transport database.

[1193] 3. Proposed solutions for when you miss the last train.

[1194] Server: Check if the time for the last train has passed the current time.

[1195] Server: If you miss the last train, calculate the following route.

[1196] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[1197] Server: Searches for the nearest other train station from the current location to the home address and calculates the walking route from there.

[1198] Route 2: Go as far as you can by train, then take a taxi home.

[1199] Server: Calculates the furthest point reachable by train and estimates the taxi fare from there to home.

[1200] Route 3: Walk home entirely.

[1201] Server: Calculates the walking route from the current location to home and displays the estimated time.

[1202] Route 4: Go home entirely by taxi.

[1203] Server: Estimate the taxi fare from the current location to home.

[1204] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[1205] Server: Searches for entertainment and hotels near the current location and displays reservation information.

[1206] 4. Route Optimization and Suggestion

[1207] Server: Calculates the required time, cost, and distance for each route, and calculates the optimal route based on the conditions set by the user.

[1208] Server: Generates a route list in order of priority and sends it to the terminal along with detailed information.

[1209] 5. Providing directions to the user

[1210] Terminal: Displays a list of suggested routes and visually shows the details of each route (estimated time, cost, distance).

[1211] 6. User selection and action execution

[1212] User: Select the most suitable route from the displayed options and press the "Select" button on the screen.

[1213] Terminal: Based on the user's selection, it displays options for performing necessary actions (calling a taxi, making a reservation for entertainment).

[1214] User: Select the appropriate action (e.g., call a taxi) and follow the on-screen instructions.

[1215] Terminal: Performs the specified action, for example, automatically contacting a taxi company.

[1216] ---

[1217] Specific example:

[1218] scenario:

[1219] User: I'm at Shinjuku Station.

[1220] Destination: My home in Naka Ward, Yokohama City.

[1221] Terminal: Confirm that Shinjuku Station is the nearest station and check the last train time (e.g., 23:30) obtained from the public transport database.

[1222] Server: Confirming that you have missed the last train at the current time (e.g., 23:45), the following route is suggested.

[1223] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[1224] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[1225] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[1226] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[1227] Route 5: Stay at a hotel near Shinjuku and return home on the first train the next day (hotel fee: 3000 yen).

[1228] User: Select "Route 2" and initiate a taxi call on the device.

[1229] Terminal: Contact a taxi company and arrange for a taxi to your current location.

[1230] Thus, the present invention provides users who have missed the last train with the most suitable way to get home, thereby reducing the burden on users.

[1231] The following describes the processing flow.

[1232] Step 1:

[1233] The user launches the app and uses the automatic location acquisition function to determine their current location. If GPS functionality is unavailable, the user manually enters their current address.

[1234] Step 2:

[1235] The user enters their home address. This allows the destination information to be identified.

[1236] Step 3:

[1237] The device searches for the nearest public transport station based on the user's current latitude and longitude. This identifies the user's nearest station.

[1238] Step 4:

[1239] The terminal retrieves the last train time for the nearest station from the public transport database. This database contains operational information for each station.

[1240] Step 5:

[1241] The server checks the current time and compares it to the last train time of the nearest station. It then determines whether the current time is past the last train time.

[1242] Step 6:

[1243] If the server determines that you have missed the last train, it will calculate several possible routes home. Specifically, these include the following routes:

[1244] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[1245] The server searches for the nearest other train station from your current location to your home address and calculates the walking route from there.

[1246] Route 2: Go as far as you can by train, then take a taxi home.

[1247] The server calculates the furthest point you can reach by train and then estimates the taxi fare from there to your home.

[1248] Route 3: Walk home entirely.

[1249] The server calculates the walking route from the current location to home and displays the estimated time.

[1250] Route 4: Go home entirely by taxi.

[1251] The server estimates the taxi fare from your current location to your home.

[1252] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[1253] The server searches for entertainment and hotels near the current location and displays reservation information.

[1254] Step 7:

[1255] The server calculates the time, cost, and distance for each route, and then calculates the optimal route based on the conditions set by the user (e.g., "fastest" or "most economical").

[1256] Step 8:

[1257] The server generates a route list in order of priority and sends it to the terminal, along with detailed information.

[1258] Step 9:

[1259] The device displays a list of suggested routes and visually shows the details of each route (estimated time, cost, and distance).

[1260] Step 10:

[1261] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen.

[1262] Step 11:

[1263] The device displays options to perform the necessary action (call a taxi, make a reservation for entertainment) based on the user's selection.

[1264] Step 12:

[1265] The user selects the appropriate action (for example, calling a taxi) and proceeds by following the on-screen instructions.

[1266] Step 13:

[1267] The device performs a specified action, for example, automatically contacting a taxi company.

[1268] Step 14:

[1269] The server monitors the route and actions selected by the user, and updates route information or suggests alternative routes as needed.

[1270] Step 15:

[1271] The device tracks the user's current location and displays an alert if they get lost or need further assistance.

[1272] Step 16:

[1273] The user finally arrives home and chooses to close the app.

[1274] (Example 1)

[1275] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1276] One challenge is that it is difficult for users who miss the last train to find an efficient and economical way to get home. Another challenge is the lack of a system that simultaneously evaluates travel time, cost, and distance to provide the optimal route when users are looking for a way to get home.

[1277] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1278] In this invention, the server includes means for acquiring the user's current location information, means for acquiring the user's destination information, means for acquiring information on the nearest public transportation based on the current location, means for calculating candidate routes available when the last train has departed, means for calculating the time, cost, and distance for each route, means for presenting multiple routes to the user, means for performing necessary actions based on the route selected by the user, means for calculating walking routes based on the proposed routes, means for estimating taxi fares based on the proposed routes, and means for searching for and displaying accommodation information based on the proposed routes. This makes it possible to efficiently and economically provide the optimal route home to a user who has missed the last train.

[1279] A "user" refers to an individual who uses the system to search for a route home.

[1280] "Current location information" refers to geographical data indicating the user's current location.

[1281] "Destination information" refers to geographical data about the location where the user is aiming to return home.

[1282] "Public transportation information" refers to timetables and operating status information for modes of transport such as trains and buses.

[1283] "Last train time" refers to the departure time and service information of the last train running on that day.

[1284] "Possible travel routes" refer to multiple routes that a user can choose to take to get home.

[1285] A "walking route" refers to the path a user takes to reach their destination on foot.

[1286] "Taxi fare estimate" refers to a predicted value of the fare incurred when using a taxi.

[1287] "Accommodation facilities" refer to facilities that users can use for temporary lodging (such as hotels and entertainment venues).

[1288] "Action" refers to a specific action taken as part of the means of getting home (e.g., calling a taxi, booking accommodation).

[1289] A "server" refers to a computer system that processes data and provides information based on requests from users.

[1290] "Terminal" refers to a device (such as a smartphone or tablet) that a user uses to access the system.

[1291] This invention is a system that suggests an efficient and economical route home for users who have missed the last train. The embodiments of this invention will be described in detail below.

[1292] This system is implemented through various operations performed by servers, terminals, and users. The hardware used includes terminals such as smartphones and tablets, while small computers and servers perform the main processing. The software used includes GPS functionality for acquiring location information and APIs (e.g., Google Maps API, NAVITIME, etc.) for acquiring public transportation information.

[1293] User information retrieval

[1294] The user launches the app on their device and enables automatic location acquisition. The device uses its GPS function to determine its current location and sends the location information to the server. If GPS is unavailable, the user can also manually enter their address. Furthermore, the user enters their destination information (where they will return home) into their device.

[1295] Obtain the nearest station and the last train time.

[1296] The device sends its current location information to the server. The server uses an API to retrieve public transportation information, specifically the nearest station and its last train time. This retrieved information is sent to the device and displayed to the user.

[1297] Suggestions for what to do if you miss the last train.

[1298] The server checks if the user has missed the last train by comparing the retrieved last train time with the current time. If the user has missed the last train, it calculates the following possible travel routes.

[1299] Route 1: Take the train to a station that is not the closest but is nearby, and then walk home from there. The server searches for the nearest other station from your current location to your destination and calculates the walking route.

[1300] Route 2: Go as far as you can by train, then take a taxi home. The server calculates the furthest point you can reach by train and estimates the taxi fare from that point to your destination.

[1301] Route 3: Walk home entirely. The server calculates the walking route from the current location to the destination and compiles the estimated time.

[1302] Route 4: Take a taxi home. The server estimates the taxi fare from your current location to your destination.

[1303] Route 5: Stay at a nearby accommodation and return home the next morning. The server searches for accommodations near your current location and displays reservation information.

[1304] Route optimization and suggestions

[1305] The server comprehensively calculates the time, cost, and distance for each potential travel route and selects the optimal route based on user-defined conditions (e.g., minimizing costs, prioritizing time). Finally, the server generates a route list in order of priority and sends it to the terminal along with detailed information.

[1306] Providing route guidance to users

[1307] The device displays the route list received from the server on the app screen. Details of each route (estimated time, cost, distance) are displayed visually, allowing the user to scroll and review them.

[1308] User selection and action execution

[1309] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen. Based on this selection, the terminal suggests and executes the necessary actions (e.g., calling a taxi, making a hotel reservation). Once the user completes the operation, the terminal automatically performs the instructed actions and contacts the taxi company or accommodation.

[1310] Specific example

[1311] scenario:

[1312] User: I'm at Shinjuku Station.

[1313] Destination: My home in Naka Ward, Yokohama City.

[1314] Terminal:

[1315] The nearest station is Shinjuku Station, and the last train time (e.g., 23:30) obtained from the public transport database is checked. It is confirmed that the last train has been missed from the current time (e.g., 23:45), and the server suggests the following route.

[1316] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[1317] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[1318] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[1319] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[1320] Route 5: Stay at an accommodation near Shinjuku and return home on the first train the next day (accommodation fee: 3000 yen).

[1321] user:

[1322] Select "Route 2" and then use the terminal to call a taxi.

[1323] Terminal:

[1324] The system contacts a taxi company and arranges for a taxi to be dispatched to the user's current location. This ensures that users have an efficient and economical way to get home, even if they miss the last train.

[1325] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1326] System program processing flow

[1327] Step 1:

[1328] The user launches the app on their smartphone or tablet. When the user presses the "Automatically acquire current location" button, the device enables its GPS function and acquires the current location. GPS data is input, and the current location information is output. Specifically, the device acquires data from the GPS sensor and displays it as the current location.

[1329] Step 2:

[1330] The user manually enters their current address (if GPS is unavailable). Once the user enters the address and presses the "Confirm" button, the manually entered address is output as the current location information. Specifically, the user's input is retrieved in text format and saved to an internal database.

[1331] Step 3:

[1332] The user enters their home address. Once the address is entered into the terminal's input form and the "Next" button is pressed, destination information is output. Specifically, the user's input is retrieved in text format and saved to an internal database.

[1333] Step 4:

[1334] The device sends its current location and destination information to the server. The current location and destination information are used as input, and the server receives this information as output. Specifically, the device sends data to the server via the internet.

[1335] Step 5:

[1336] The server accesses a public transport API (e.g., Google Maps API) to retrieve the nearest station and last train time. Current location information is used as input, and the output is data on the nearest station and last train time. Specifically, the server sends an API request and saves the received data to an internal database.

[1337] Step 6:

[1338] The server compares the current time with the last train time to determine if the user has missed the last train. The current time and last train time are used as input, and the output is a determination of whether the user missed the last train. Specifically, the server executes an algorithm to compare the current time with the last train time.

[1339] Step 7:

[1340] If you miss the last train, the server calculates possible travel routes. Current location information, destination information, and public transport information are used as input, and multiple travel routes are returned as output. Specifically, the following routes are calculated:

[1341] Route 1: Take the train to a station that is not the closest but is nearby from your current location, and then walk home from there.

[1342] The server searches for the nearest station from the current location and calculates the walking route. The output provides information about the station and the walking route.

[1343] Route 2: Go as far as you can by train, then take a taxi home.

[1344] The server calculates the final destination reachable by train and estimates the taxi fare. The output provides the final destination and the estimated taxi fare.

[1345] Route 3: Walk home entirely.

[1346] The server calculates the walking route from the current location to the destination. The output includes the walking route and the estimated time.

[1347] Route 4: Go home entirely by taxi.

[1348] The server estimates the taxi fare from the current location to the destination. The taxi fare is returned as output.

[1349] Route 5: Stay at a nearby accommodation and return home the next day.

[1350] The server searches for accommodations near the current location and displays a list of accommodations and reservation information. Accommodation information is provided as output.

[1351] Step 8:

[1352] The server calculates the time, cost, and distance for each route and selects the optimal route based on user-defined conditions. Route candidate data is used as input, and the output is a list of optimal routes. Specifically, the server executes a route evaluation algorithm and calculates the evaluation results.

[1353] Step 9:

[1354] The server sends the optimal route list to the terminal. The route list is used as input, and the terminal receives the route list as output. Specifically, the server sends data over the internet, and the terminal receives it.

[1355] Step 10:

[1356] The app displays the route list received by the device on the app screen. It visually displays the estimated time, cost, and distance for each route so that the user can review the route details. Specifically, the device generates the display data and renders it on the screen.

[1357] Step 11:

[1358] The user selects a route and presses the "Select" button. The selected route information is used as input, and the selection result is obtained as output. Specifically, the user's selection is received and sent to the server.

[1359] Step 12:

[1360] The device performs actions based on the user's selections (e.g., calling a taxi, making a hotel reservation). User selection information is used as input, and the output is the result of executing a specific action. Specifically, the device calls the corresponding API to contact the taxi or accommodation provider.

[1361] (Application Example 1)

[1362] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1363] While systems already exist that suggest efficient and economical routes home for users who miss the last train, there is a similar need for technology that calculates efficient and economical delivery routes in the food delivery industry. In particular, since reducing time and costs is a critical issue in the food delivery industry, a system that can provide efficient delivery routes is essential.

[1364] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1365] In this invention, the server includes means for acquiring the user's current location information, means for acquiring destination information, means for acquiring information on the nearest public transport, means for calculating candidate available travel routes, means for calculating the time, cost, and distance for each travel route, means for presenting multiple travel routes, means for performing necessary actions based on the travel route selected by the user, and means for calculating an efficient delivery route for food delivery. This makes it possible to provide the optimal travel route in terms of both time and cost.

[1366] "User" refers to an individual or group that uses the system.

[1367] "Current location information" refers to the latitude and longitude data of the user's current location.

[1368] "Destination information" refers to data such as the address and latitude / longitude of the place the user is heading to.

[1369] "Public transportation" refers to mass transportation methods such as buses, trains, and subways.

[1370] "Possible travel routes" refer to multiple options for routes and means of transportation to reach a destination.

[1371] "Travel time" refers to the time required to travel from your current location to your destination.

[1372] "Cost" refers to the economic cost required to travel from your current location to your destination.

[1373] "Distance traveled" refers to the physical distance from your current location to your destination.

[1374] "Food delivery" refers to a service that delivers food and beverages to a specified location.

[1375] An "efficient delivery route" is a route optimized to minimize delivery time and costs.

[1376] "Means of performing actions" refer to methods such as calling a car or taxi or booking accommodation based on the travel route selected by the user.

[1377] This invention is a system that proposes efficient and economical delivery routes for food delivery. Based on the user's current location and destination information, this system calculates and presents a delivery route combining multiple modes of transportation to the user. Embodiments of this invention will be described in detail below.

[1378] Server Processing

[1379] The server first obtains the user's current location and destination information. Current location information is obtained using GPS or an IP geolocation service. Destination information is obtained using the Google Maps API from the address entered by the user, providing latitude and longitude. This ensures the server has accurate location information for both the current and destination locations.

[1380] Next, the server retrieves information about the nearest public transport. To do this, it consults a public transport database to obtain the nearest train station or bus stop and the time of the last train. If the last train has already departed, the server calculates route options for each mode of transportation (bicycle, public transport, walking).

[1381] Route options for each mode of transport are calculated based on their average speed, determining the travel time, cost, and distance. For example, a standard of 15 km / h is used for cycling, 30 km / h for public transport, and 5 km / h for walking. In this way, the server can obtain detailed information for all included routes.

[1382] Terminal processing

[1383] The terminal visually displays multiple route options sent from the server to the user. Each route clearly indicates the estimated travel time, cost, and distance, and presents the user with its advantages and disadvantages. Based on this information, the user can choose the most suitable mode of transportation.

[1384] Based on the route selected by the user, the device performs the necessary actions. For example, if the user requests a taxi or makes a hotel reservation, the device automatically arranges these using the corresponding API.

[1385] Specific example

[1386] User's current location: Shibuya Station

[1387] Destination: Restaurants near Ebisu Station

[1388] Optimal route: Travel by bicycle (approximately 10 minutes)

[1389] In this specific example, if the user is at Shibuya Station, the server obtains location information for both the user's current location and destination, and selects a bicycle as the most efficient route. This bicycle route is calculated to take approximately 10 minutes and is then presented to the user.

[1390] Example of a prompt

[1391] An example of a prompt message to input to a generative AI model is as follows:

[1392] text

[1393] Develop an application that suggests the most efficient route for delivery drivers when a user orders food delivery. Please use the following information as a basis.

[1394] Current location: Shibuya Station

[1395] Destination: Restaurants near Ebisu Station

[1396] Available modes of transportation: bicycle, public transport, walking

[1397] Calculate the travel time and distance for each mode of transportation and suggest the optimal route.

[1398] This allows users to obtain efficient routes in real time and reach their destinations quickly and economically.

[1399] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1400] Step 1:

[1401] The server obtains the user's current location information. Using GPS functionality or an IP geolocation service, it obtains the latitude and longitude of the user's current location from the user's device. This allows the server to obtain the user's precise location information as input data. The output data is the latitude and longitude information of the current location.

[1402] Step 2:

[1403] The user enters the destination address. The device retrieves this entered address data and uses the Google Maps API to obtain the corresponding latitude and longitude. The server receives this latitude and longitude information as input data and stores it as destination information. The output data is the latitude and longitude information of the destination.

[1404] Step 3:

[1405] The server retrieves information about the nearest public transport based on the user's current location. It consults a public transport database to obtain the nearest train station or bus stop, as well as the time of the last train or bus. The server receives this information as input data and stores it as information about the nearest public transport. The output data includes the name, location, and time of the last train or bus of the nearest public transport.

[1406] Step 4:

[1407] The server calculates possible travel routes. It calculates route options for each mode of transport (bicycle, public transport, walking) from the current location to the destination, and calculates the estimated time, cost, and distance for each. The server receives these calculation results as input data and stores them as travel route options. The output data consists of the estimated time, cost, and distance for each mode of transport.

[1408] Step 5:

[1409] The server presents the user with the results of each calculated travel route. The terminal visually displays the detailed information (estimated time, cost, distance) of each travel route received from the server. The user selects the optimal route based on this information. The input data is the route candidate information received from the server, and the output data is the route details displayed on the terminal.

[1410] Step 6:

[1411] The user selects the optimal route from several presented travel routes. The terminal receives the user's selection and sends it to the server. The input data is the route information selected by the user, and the output data is the selected route information sent to the server.

[1412] Step 7:

[1413] The server performs the necessary actions based on the travel route selected by the user. For example, it automatically arranges things like calling a taxi or booking accommodation using APIs corresponding to each route. The input data is the route information selected by the user, and the output data is the result of each arrangement action.

[1414] This allows users to obtain efficient and economical travel routes and take appropriate actions quickly.

[1415] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1416] ---

[1417] This invention combines a system that suggests efficient and economical routes home when a user misses the last train with an emotion engine that recognizes the user's emotions. Based on the user's current location and destination information, the system suggests the most suitable way home. By utilizing the emotion engine, the suggested travel route can be adjusted according to the user's emotional state.

[1418] ---

[1419] System Overview

[1420] This system acquires the user's current location and destination information and suggests the optimal route to get home efficiently and economically, even if the user misses the last train. It also recognizes the user's emotional state and adjusts the travel route based on those emotions.

[1421] ---

[1422] Program Processing Description

[1423] 1. Obtaining user information

[1424] User: Launch the app and use the automatic location acquisition function to determine your current location. If GPS functionality is unavailable, manually enter your address.

[1425] User: Enter your home address. This will identify your destination.

[1426] 2. Obtain the nearest station and the last train time.

[1427] Terminal: Searches for the nearest station from the current location and retrieves the last train time for that station from the public transport database.

[1428] 3. Recognizing the user's emotional state

[1429] Device: The app uses its in-app camera and microphone to recognize emotions from the user's facial expressions and tone of voice.

[1430] Device: The emotion engine analyzes the user's emotional state, identifying things like fatigue, stress, and relaxation.

[1431] 4. Proposed solutions for when you miss the last train.

[1432] Server: Check if the time for the last train has passed the current time.

[1433] Server: If you miss the last train, calculate the following route.

[1434] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[1435] Server: Searches for the nearest other train station from the current location to the home address and calculates the walking route from there.

[1436] Route 2: Go as far as you can by train, then take a taxi home.

[1437] Server: Calculates the furthest point reachable by train and estimates the taxi fare from there to home.

[1438] Route 3: Walk home entirely.

[1439] Server: Calculates the walking route from the current location to home and displays the estimated time.

[1440] Route 4: Go home entirely by taxi.

[1441] Server: Estimate the taxi fare from the current location to home.

[1442] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[1443] Server: Searches for entertainment and hotels near the current location and displays reservation information.

[1444] 5. Route Optimization and Suggestions

[1445] Server: Calculates the time, cost, and distance for each route, and adjusts the optimal route based on the user's emotional state.

[1446] Server: If the user is tired, prioritize suggesting a relaxing route; if they prioritize economical options, suggest a low-cost route.

[1447] Server: Generates a route list in order of priority and sends it to the terminal along with detailed information.

[1448] 6. Providing directions to the user

[1449] Terminal: Displays a list of each proposed route and visually shows the details of each route (estimated time, cost, distance, and emotional considerations).

[1450] 7. User selection and action execution

[1451] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen.

[1452] The device displays options to perform the necessary action (call a taxi, make a reservation for entertainment) based on the user's selection.

[1453] The user selects the appropriate action (for example, calling a taxi) and proceeds by following the on-screen instructions.

[1454] The device performs a specified action, for example, automatically contacting a taxi company.

[1455] ---

[1456] Specific example:

[1457] scenario:

[1458] User: I'm at Shinjuku Station.

[1459] Destination: My home in Naka Ward, Yokohama City.

[1460] User's emotions: Analysis of facial expressions and voice indicates that the user is tired.

[1461] Terminal: Confirm that Shinjuku Station is the nearest station and retrieve the last train time (e.g., 23:30) from the public transport database.

[1462] Server: Confirming that you have missed the last train at the current time (e.g., 23:45), the following route is suggested.

[1463] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[1464] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[1465] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[1466] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[1467] Route 5: Stay at a hotel near Shinjuku and return home on the first train the next day (hotel fee: 3000 yen).

[1468] Server: Since the server recognizes the user is tired, it prioritizes displaying the most comfortable and fastest "Route 4".

[1469] User: Select "Route 4" and initiate a taxi call on the device.

[1470] Terminal: Contact a taxi company and arrange for a taxi to your current location.

[1471] In this way, by utilizing the emotion engine, it is possible to provide the optimal way for users to return home according to their emotions, further reducing the burden on users.

[1472] The following describes the processing flow.

[1473] Step 1:

[1474] The user launches the app and uses the automatic location acquisition function to determine their current location. If GPS functionality is unavailable, the user manually enters their current address.

[1475] Step 2:

[1476] The user enters their home address. This allows the destination information to be identified.

[1477] Step 3:

[1478] The device searches for the nearest public transport station based on the user's current latitude and longitude. This identifies the user's nearest station.

[1479] Step 4:

[1480] The terminal retrieves the last train time for the nearest station from the public transport database. This database contains operational information for each station.

[1481] Step 5:

[1482] The server checks the current time and compares it to the last train time of the nearest station. It then determines whether the current time is past the last train time.

[1483] Step 6:

[1484] The device activates the user's facial recognition function and uses its built-in camera to capture the user's facial expressions. It also uses its voice recognition function to record what the user says and transfers this recording to emotion recognition software.

[1485] Step 7:

[1486] The server's emotion engine analyzes captured facial expressions and voice data to identify the user's emotional state (e.g., fatigue, stress, relaxation).

[1487] Step 8:

[1488] The server takes your emotional state into account and calculates the following return route. Specifically:

[1489] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[1490] The server searches for the nearest other train station from your current location to your home address and calculates the walking route from there.

[1491] Route 2: Go as far as you can by train, then take a taxi home.

[1492] The server calculates the furthest point you can reach by train and then estimates the taxi fare from there to your home.

[1493] Route 3: Walk home entirely.

[1494] The server calculates the walking route from the current location to home and displays the estimated time.

[1495] Route 4: Go home entirely by taxi.

[1496] The server estimates the taxi fare from your current location to your home.

[1497] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[1498] The server searches for entertainment and hotels near the current location and displays reservation information.

[1499] Step 9:

[1500] The server calculates the time, cost, and distance for each route and adjusts the optimal route based on the user's emotional state. If the user is highly fatigued, it prioritizes a comfortable and fast route.

[1501] Step 10:

[1502] The server generates a route list in order of priority and sends it to the terminal, along with detailed information.

[1503] Step 11:

[1504] The device displays a list of suggested routes and visually shows the details of each route (estimated time, cost, distance, and emotional considerations).

[1505] Step 12:

[1506] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen.

[1507] Step 13:

[1508] The device displays options to perform the necessary action (call a taxi, make a reservation for entertainment) based on the user's selection.

[1509] Step 14:

[1510] The user selects the appropriate action (for example, calling a taxi) and proceeds by following the on-screen instructions.

[1511] Step 15:

[1512] The device performs the specified action, for example, automatically contacting a taxi company. It also completes the booking process if a reservation is required.

[1513] Step 16:

[1514] The server monitors the route and actions selected by the user, and updates route information or suggests alternative routes as needed.

[1515] Step 17:

[1516] The device tracks the user's current location and displays an alert if they get lost or need further assistance.

[1517] Step 18:

[1518] After the user finally arrives home, they can choose to exit the app and then it will close.

[1519] (Example 2)

[1520] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1521] In modern urban life, missing the last train is a common occurrence, and finding an efficient and economical way to get home is not easy. Furthermore, the lack of consideration for the user's emotional state can increase fatigue and stress. Conventional systems have been unable to suggest appropriate routes that take the user's emotional state into account.

[1522] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1523] In this invention, the server includes means for acquiring the user's current location information, means for acquiring the user's destination information, and means for calculating multiple candidate travel routes. This makes it possible to propose the optimal travel route while taking into account the user's emotional state.

[1524] "Means of obtaining the user's current location information" refers to a function that determines the user's current location through GPS functionality or manual input by the user.

[1525] "Means for obtaining user destination information" refers to the function of inputting and obtaining information about the address or location set by the user as their destination.

[1526] "Means of obtaining information on the nearest public transport" refers to a function that searches for and retrieves the location and timetable of the nearest public transport based on the user's current location.

[1527] "Method for calculating multiple travel route options" refers to a function that calculates and suggests multiple travel routes that allow a user to reach their destination efficiently and economically, even if they miss the last train.

[1528] "Means for calculating the time, cost, and distance of each travel route" refers to a function that calculates in detail the time, cost, distance, etc., for each proposed travel route.

[1529] "Means of presenting multiple travel routes to the user" refers to a function that visually displays multiple calculated travel routes to the user, making it easier to compare and consider them.

[1530] "Means of performing necessary actions based on the travel route selected by the user" refers to a function that performs specific actions based on the travel route selected by the user, such as arranging a taxi or making a hotel reservation.

[1531] "A means of recognizing the user's emotional state and adjusting the travel route based on those emotions" refers to a function that uses cameras and microphones to analyze the user's emotions from their facial expressions and tone of voice, and then adjusts and suggests the optimal travel route according to those emotions.

[1532]

[1533] This invention is a system that suggests an efficient and economical route home when a user misses the last train, and further combines it with an emotion engine that recognizes the user's emotions to provide the optimal route according to the user's state. Specific embodiments of this invention are described below.

[1534] System Configuration

[1535] Hardware and software

[1536] 1. User's device:

[1537] Smartphone or tablet

[1538] Features include GPS, camera, and microphone.

[1539] 2. Server:

[1540] Servers with high-speed data processing capabilities

[1541] API that integrates with public transport databases

[1542] Emotion analysis engine

[1543] Specific names to use

[1544] GPS function: A standard geolocation API for obtaining the user's current location information.

[1545] Emotion analysis engine: OpenCV is used for facial expression analysis, and Azure Cognitive Services is used for speech analysis.

[1546] Public transport APIs: Google Maps API and public transport databases

[1547] Overview of program processing

[1548] 1. User's device:

[1549] Launch the app and use the GPS function to obtain your current location.

[1550] If the current location cannot be determined, the user will manually enter the address.

[1551] 2. User's device:

[1552] The user enters their home address as destination information.

[1553] 3. Terminal:

[1554] Use the public transport API to retrieve information about the nearest public transport.

[1555] Calculate the distance and travel time from your current location to the nearest station.

[1556] 4. Device:

[1557] The camera and microphone are activated to collect data on the user's face and voice.

[1558] The data is sent to an emotion analysis engine to analyze the user's emotional state (e.g., fatigue, stress, relaxation).

[1559] 5. Server:

[1560] Based on data transmitted from the device, it receives current location, destination, nearest station, last train time, and sentiment data.

[1561] The last train times are retrieved from the public transport database and compared with the current time.

[1562] 6. Server:

[1563] If you miss the last train, calculate multiple routes home.

[1564] For each route, calculate the required time, cost, and distance traveled.

[1565] 7. Server:

[1566] Based on the results of emotion analysis, the optimal route is selected. For example, if the user is tired, taking a taxi home is prioritized.

[1567] 8. Terminal:

[1568] Visually display multiple calculated routes. Detailed information for each route (estimated time, cost, distance, and emotional considerations) is displayed in a list.

[1569] 9. User:

[1570] Select the best route from the available options and press the "Select" button.

[1571] 10. Terminal:

[1572] Based on the selected route, the system will perform the necessary actions (such as calling a taxi or making a reservation for entertainment).

[1573] Specific example

[1574] scenario

[1575] User: I'm at Shinjuku Station.

[1576] Destination: My home in Naka Ward, Yokohama City.

[1577] User's emotions: Analysis of facial expressions and voice indicates that the user is tired.

[1578] Flow of operations

[1579] 1. Terminal: Confirm that Shinjuku Station is the nearest station and retrieve the last train time (e.g., 23:30) from the public transport database.

[1580] 2. Server: Confirms that the user has missed the last train from the current time (e.g., 23:45) and suggests the following route.

[1581] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[1582] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[1583] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[1584] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[1585] Route 5: Stay at a hotel near Shinjuku and return home on the first train the next day (hotel fee: 3000 yen).

[1586] 3. Server: Since the server recognizes that the user is tired, it prioritizes displaying the most comfortable and fastest "Route 4".

[1587] 4. User: Select "Route 4" and initiate a taxi call on the device.

[1588] 5. Terminal: Contact a taxi company and arrange for a taxi to your current location.

[1589] Example of a prompt

[1590] The following prompt statements can be used to generate a detailed explanation of the program's processing using a generative AI model.

[1591] Point down

[1592] We have developed a system that suggests efficient and economical routes home for users who have missed the last train. This system incorporates an emotion engine that recognizes the user's emotions. Please explain in detail how this system works, including specific processing steps and actions. Also, please include specific user actions and system responses in your explanation.

[1593] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1594] Step 1:

[1595] User: Launch the app and press the "Find Current Location" button using the GPS function. If GPS is enabled, your current location will be automatically acquired. If GPS is unavailable, you will be taken to the "Manual Address Input" screen and can manually enter your address.

[1596] Input: Location data from GPS, or manually entered address.

[1597] Output: User's current location information.

[1598] Specific operation: The app performs an operation to obtain the current location. If GPS is enabled, location information is obtained automatically. If GPS is disabled, the user enters an address.

[1599] Step 2:

[1600] User: Enter your home address in the address input field within the app and press the "Set Destination" button.

[1601] Input: Address data entered by the user.

[1602] Output: Destination information.

[1603] Specific operation: The user enters their home address in the address input field within the app and presses the submit button, registering the destination information in the system.

[1604] Step 3:

[1605] Terminal: Based on the current location information, it queries the public transport API to obtain information about the nearest public transport. At this time, it also calculates the distance and travel time from the current location to the nearest station.

[1606] Input: Current location information.

[1607] Output: Information on the nearest public transport, distance, and travel time.

[1608] Specific operation: The device sends its current location information to the public transport API to obtain information about the nearest train station or bus stop. Furthermore, it calculates the travel time and distance from the current location to the nearest public transport.

[1609] Step 4:

[1610] Device: Activates the camera and microphone and performs "facial expression analysis" and "voice analysis." Captures the user's face with the camera and sends the data to the emotion analysis engine.

[1611] Input: Face image captured by camera, voice recorded by microphone.

[1612] Output: User sentiment data.

[1613] Specific operation: The camera and microphone are used to collect the user's facial expressions and voice, and send them to an emotion analysis engine. As a result of the analysis, the user's emotional state is determined, such as "fatigue," "stress," or "relaxation."

[1614] Step 5:

[1615] Server: Receives current location information, destination information, nearest station information, last train time, and sentiment data sent from the terminal.

[1616] Input: Current location information, destination information, nearest station information, last train time, sentiment data.

[1617] Output: Integrated data of various types of information.

[1618] Specific operation: Integrates all the information sent from the terminal and stores it in a database. It also retrieves the last train time from the public transport database and compares it with the current time.

[1619] Step 6:

[1620] Server: If you have missed the last train, calculate several possible routes home. For each route, calculate the time required, cost, and distance traveled.

[1621] Input: Last train time, current time, destination information.

[1622] Output: Multiple travel route options, estimated travel time, cost, and distance for each route.

[1623] Specific actions: Compare the current time with the last train time, and if the last train has been missed, suggest multiple routes home. Calculate the travel time, cost, and distance for each route.

[1624] Step 7:

[1625] Server: Based on the results of sentiment analysis, the server selects the optimal route considering the user's emotional state. If the user is tired, priority is given to taking a taxi home.

[1626] Input: Candidate travel routes, sentiment analysis data.

[1627] Output: An optimal travel route based on emotions.

[1628] Specific operation: Based on the emotion analysis results, the system selects and adjusts the optimal travel route that reflects the user's emotional state.

[1629] Step 8:

[1630] Terminal: Displays a list of multiple routes sent from the server. Visually displays the estimated time, cost, distance, and sentiment considerations for each route.

[1631] Input: Route data sent from the server.

[1632] Output: A list of routes presented to the user.

[1633] Specific operation: Visually display detailed data for each route and present it in a list format for easy comparison by the user.

[1634] Step 9:

[1635] User: Select the best route from the multiple routes presented and press the "Select" button on the screen.

[1636] Input: User-selected route.

[1637] Output: Selected travel route.

[1638] Specific operation: The user selects the optimal route from the list of routes on the screen and presses the "Select" button.

[1639] Step 10:

[1640] Terminal: Performs necessary actions (such as calling a taxi or making a reservation) based on the travel route selected by the user. If calling a taxi, it contacts a taxi company and arranges for a taxi to be dispatched to the user's current location.

[1641] Input: The travel route selected by the user.

[1642] Output: Action performed (e.g., arranging a taxi).

[1643] Specific operation: Based on the route selected by the user, the device automatically performs the necessary actions. If a taxi is requested, it will automatically contact a taxi company and arrange for a taxi to be dispatched to the user's current location.

[1644] (Application Example 2)

[1645] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1646] In modern society, it is particularly difficult to return home efficiently and economically if one misses the last train. However, there is a need not only to provide transportation, but also to propose the optimal way to return home that takes into account the user's emotional state. This invention aims to solve this problem by proposing a system that provides a comfortable and efficient way to return home, taking into account the emotional state of the user when they miss the last train.

[1647] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1648] In this invention, the server includes means for acquiring the user's current location information, means for acquiring the user's destination information, means for acquiring information on the nearest public transportation based on the current location, means for calculating candidate travel routes available when the time for the last train has passed, means for calculating the time, cost, and distance for each travel route, means for recognizing the user's emotional state and adjusting the travel route based on the emotional state, means for prioritizing the presentation of comfortable travel options when the user is fatigued based on the emotional state, means for presenting the user with multiple travel routes, means for performing necessary actions based on the travel route selected by the user, and means for summoning an autonomous vehicle and providing a means of transportation from the user's current location to their destination. This enables the provision of an optimal way to return home that takes the user's emotional state into consideration, allowing for a comfortable and efficient return home.

[1649] A "user" is an individual who uses the system to input their current location and destination information and receive suggestions for the optimal route home.

[1650] "Current location information" refers to the location information of the user's current location when using the system.

[1651] "Destination information" refers to the location information of the place the user wants to return to using the system.

[1652] "Public transport" refers to means of transportation such as trains, buses, and ferries that operate according to specific routes and schedules.

[1653] "Last train" refers to the time when public transportation services cease operations for the day.

[1654] A "travel route" refers to the path or method a user takes to travel from their current location to their destination.

[1655] "Travel time" refers to the time it takes to reach a destination using a specific travel route.

[1656] "Expenses" refer to the financial expenditure necessary to use a particular travel route.

[1657] "Distance traveled" refers to the actual distance traveled along a specific route.

[1658] "Emotional state" refers to the user's psychological or physical condition, including, for example, fatigue, stress, and relaxation.

[1659] An "emotion engine" is a program that recognizes and analyzes a user's emotional state.

[1660] An "action" is a specific operation or procedure performed based on the travel route selected by the user.

[1661] An "autonomous vehicle" is a vehicle that does not require a driver and is operated automatically by a computer.

[1662] This invention is a system that suggests an efficient and economical route home when a user misses the last train, and it recognizes the user's emotional state to provide the optimal way to get home. This system uses a smartphone as its main interface and combines an emotion recognition engine with an autonomous vehicle API to suggest a route home that meets the user's individual needs.

[1663] The system consists of the following elements:

[1664] 1. Obtaining user information

[1665] The user launches a smartphone application and enters their current location and destination information. Current location information is either automatically obtained using GPS or entered manually. Destination information is also entered manually by the user.

[1666] 2. Recognition of emotional states

[1667] The system uses the smartphone's camera and microphone to analyze the user's facial expressions and voice tone to recognize their emotional state. This process utilizes software called EmotionRecognition.

[1668] 3. Calculation of the optimal route

[1669] Based on your current location, destination, and emotional state, the system calculates the optimal route home. The calculated route includes factors such as travel time, cost, distance, and comfort level.

[1670] 4. Provision of transportation

[1671] Based on the route calculated and determined to be the most suitable by the user, an autonomous vehicle is dispatched. The SelfDrivingCarAPI is used to select the appropriate autonomous vehicle and dispatch it to the user's specified current location.

[1672] Specifically, the system operates in the following scenario:

[1673] scenario:

[1674] If a user is at Shinjuku Station, has missed the last train, and is trying to return home to Naka Ward, Yokohama City, the user's smartphone app is launched, retrieves "Shinjuku Station" as the current location, and inputs "Naka ​​Ward, Yokohama City" as the destination. If the emotion recognition engine determines from the user's facial expression and voice that they are "tired," the system prioritizes displaying a comfortable route home.

[1675] For example, routes such as "traveling by train from Shinjuku to Yokohama and then taking a taxi from there" or "taking a taxi directly home from Shinjuku" are suggested. If the user chooses the latter, an autonomous vehicle will pick them up at Shinjuku Station and drop them off at their home in Naka Ward, Yokohama City.

[1676] The specific software and hardware used to perform the above processes are as follows:

[1677] Software used:

[1678] EmotionRecognition (emotion recognition engine)

[1679] SelfDrivingCarAPI (Autonomous Vehicle API)

[1680] Hardware used:

[1681] Smartphone (acquisition of user's current location information and sentiment recognition)

[1682] Autonomous vehicles (providing transportation to the user's destination)

[1683] Prompt example:

[1684] Create an app that, when a user misses the last train, will summon an autonomous vehicle and suggest an efficient and economical route home. The app will obtain the user's current location and destination, and calculate the optimal route based on the user's emotional state. Specifically, it will prioritize displaying comfortable routes that take into account the user's fatigue level, and summon an autonomous taxi if necessary.

[1685] In this way, the present invention can provide an optimal method of returning home that takes into account the user's emotional state, making it possible to return home comfortably and efficiently.

[1686] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1687] Step 1:

[1688] The user launches the application on their smartphone and enters their current location and destination information.

[1689] Input: User's current location (GPS information or manual input) and destination (manual input).

[1690] Operation: The app automatically obtains the current location using GPS, or the user manually enters the current location and destination.

[1691] Step 2:

[1692] The device retrieves information about the nearest public transportation based on the user's current location.

[1693] Input: Current location information.

[1694] Function: Searches the public transport database for the nearest train station or bus stop and retrieves that information.

[1695] Output: Information on the nearest public transport.

[1696] Step 3:

[1697] The terminal retrieves the last train times from the public transport database.

[1698] Input: Information about the nearest public transportation.

[1699] Operation: Retrieves the last train time from the database.

[1700] Output: Last train time.

[1701] Step 4:

[1702] The device uses the smartphone's camera and microphone to recognize the user's emotional state.

[1703] Input: User's facial expressions and voice data.

[1704] Operation: Using EmotionRecognition (an emotion recognition engine), it analyzes the user's facial expressions and voice tone to identify their emotional state.

[1705] Output: User's emotional state (fatigue, stress, relaxation, etc.).

[1706] Step 5:

[1707] The server compares the current time with the time of the last train to determine whether the user has missed the last train.

[1708] Input: Current location information, last train time.

[1709] Operation: Compares the current time with the time of the last train to determine if the last train has been missed.

[1710] Output: The result of determining whether you missed the last train.

[1711] Step 6:

[1712] The server calculates possible travel routes available if the last train is missed.

[1713] Input: Current location information, destination information, emotional state.

[1714] Function: Calculates the time, cost, and distance traveled for each travel route (walking, train + walking, train + taxi, direct taxi).

[1715] Output: Candidate travel routes.

[1716] Step 7:

[1717] The server adjusts and suggests the optimal travel route based on the user's emotional state.

[1718] Input: Candidate travel routes, emotional state.

[1719] Operation: Depending on the user's emotional state, for example, if they are fatigued, the system will prioritize displaying comfortable routes. Other parameters (such as travel time, cost, and distance) will also be considered and adjusted accordingly.

[1720] Output: Optimal travel route.

[1721] Step 8:

[1722] The server presents the user with multiple optimal travel routes.

[1723] Input: Optimal travel route.

[1724] Operation: Visually displays detailed route information (estimated time, cost, distance, comfort level, etc.) on the app screen.

[1725] Output: A visually presented travel route.

[1726] Step 9:

[1727] The system performs the necessary actions based on the travel route selected by the user.

[1728] Input: The route selected by the user.

[1729] Operation: Depending on the user's selection, it will perform actions such as calling an autonomous vehicle, arranging a taxi, or booking accommodation.

[1730] Output: Actions performed (e.g., arranging an autonomous vehicle, booking a taxi, booking accommodation).

[1731] Step 10:

[1732] The device notifies the user that the action has been completed.

[1733] Input: The result of the action performed.

[1734] Action: The app notifies the user that the action is complete.

[1735] Output: Notification of action completion.

[1736] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1737] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1738] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1739] [Fourth Embodiment]

[1740] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1741] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1742] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1743] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1744] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1745] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1746] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1747] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1748] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1749] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1750] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1751] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1752] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1753] ---

[1754] This invention is a system that suggests an efficient and economical route home for users who have missed the last train. Embodiments of this invention will be described in detail below.

[1755] ---

[1756] System Overview

[1757] This system suggests the most efficient way to get home based on the user's current location and destination information. Specifically, it calculates and presents candidate routes that combine public transport information, taxis, and walking options to the user.

[1758] ---

[1759] Program Processing Description

[1760] 1. Obtaining user information

[1761] User: Launch the app and use the automatic location acquisition function to determine your current location. If GPS functionality is unavailable, manually enter your address.

[1762] User: Enter your home address.

[1763] 2. Obtain the nearest station and the last train time.

[1764] Terminal: Searches for the nearest station from the current location and retrieves the last train time for that station from the public transport database.

[1765] 3. Proposed solutions for when you miss the last train.

[1766] Server: Check if the time for the last train has passed the current time.

[1767] Server: If you miss the last train, calculate the following route.

[1768] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[1769] Server: Searches for the nearest other train station from the current location to the home address and calculates the walking route from there.

[1770] Route 2: Go as far as you can by train, then take a taxi home.

[1771] Server: Calculates the furthest point reachable by train and estimates the taxi fare from there to home.

[1772] Route 3: Walk home entirely.

[1773] Server: Calculates the walking route from the current location to home and displays the estimated time.

[1774] Route 4: Go home entirely by taxi.

[1775] Server: Estimate the taxi fare from the current location to home.

[1776] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[1777] Server: Searches for entertainment and hotels near the current location and displays reservation information.

[1778] 4. Route Optimization and Suggestion

[1779] Server: Calculates the required time, cost, and distance for each route, and calculates the optimal route based on the conditions set by the user.

[1780] Server: Generates a route list in order of priority and sends it to the terminal along with detailed information.

[1781] 5. Providing directions to the user

[1782] Terminal: Displays a list of suggested routes and visually shows the details of each route (estimated time, cost, distance).

[1783] 6. User selection and action execution

[1784] User: Select the most suitable route from the displayed options and press the "Select" button on the screen.

[1785] Terminal: Based on the user's selection, it displays options for performing necessary actions (calling a taxi, making a reservation for entertainment).

[1786] User: Select the appropriate action (e.g., call a taxi) and follow the on-screen instructions.

[1787] Terminal: Performs the specified action, for example, automatically contacting a taxi company.

[1788] ---

[1789] Specific example:

[1790] scenario:

[1791] User: I'm at Shinjuku Station.

[1792] Destination: My home in Naka Ward, Yokohama City.

[1793] Terminal: Confirm that Shinjuku Station is the nearest station and check the last train time (e.g., 23:30) obtained from the public transport database.

[1794] Server: Confirming that you have missed the last train at the current time (e.g., 23:45), the following route is suggested.

[1795] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[1796] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[1797] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[1798] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[1799] Route 5: Stay at a hotel near Shinjuku and return home on the first train the next day (hotel fee: 3000 yen).

[1800] User: Select "Route 2" and initiate a taxi call on the device.

[1801] Terminal: Contact a taxi company and arrange for a taxi to your current location.

[1802] Thus, the present invention provides users who have missed the last train with the most suitable way to get home, thereby reducing the burden on users.

[1803] The following describes the processing flow.

[1804] Step 1:

[1805] The user launches the app and uses the automatic location acquisition function to determine their current location. If GPS functionality is unavailable, the user manually enters their current address.

[1806] Step 2:

[1807] The user enters their home address. This allows the destination information to be identified.

[1808] Step 3:

[1809] The device searches for the nearest public transport station based on the user's current latitude and longitude. This identifies the user's nearest station.

[1810] Step 4:

[1811] The terminal retrieves the last train time for the nearest station from the public transport database. This database contains operational information for each station.

[1812] Step 5:

[1813] The server checks the current time and compares it to the last train time of the nearest station. It then determines whether the current time is past the last train time.

[1814] Step 6:

[1815] If the server determines that you have missed the last train, it will calculate several possible routes home. Specifically, these include the following routes:

[1816] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[1817] The server searches for the nearest other train station from your current location to your home address and calculates the walking route from there.

[1818] Route 2: Go as far as you can by train, then take a taxi home.

[1819] The server calculates the furthest point you can reach by train and then estimates the taxi fare from there to your home.

[1820] Route 3: Walk home entirely.

[1821] The server calculates the walking route from the current location to home and displays the estimated time.

[1822] Route 4: Go home entirely by taxi.

[1823] The server estimates the taxi fare from your current location to your home.

[1824] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[1825] The server searches for entertainment and hotels near the current location and displays reservation information.

[1826] Step 7:

[1827] The server calculates the time, cost, and distance for each route, and then calculates the optimal route based on the conditions set by the user (e.g., "fastest" or "most economical").

[1828] Step 8:

[1829] The server generates a route list in order of priority and sends it to the terminal, along with detailed information.

[1830] Step 9:

[1831] The device displays a list of suggested routes and visually shows the details of each route (estimated time, cost, and distance).

[1832] Step 10:

[1833] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen.

[1834] Step 11:

[1835] The device displays options to perform the necessary action (call a taxi, make a reservation for entertainment) based on the user's selection.

[1836] Step 12:

[1837] The user selects the appropriate action (for example, calling a taxi) and proceeds by following the on-screen instructions.

[1838] Step 13:

[1839] The device performs a specified action, for example, automatically contacting a taxi company.

[1840] Step 14:

[1841] The server monitors the route and actions selected by the user, and updates route information or suggests alternative routes as needed.

[1842] Step 15:

[1843] The device tracks the user's current location and displays an alert if they get lost or need further assistance.

[1844] Step 16:

[1845] The user finally arrives home and chooses to close the app.

[1846] (Example 1)

[1847] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1848] One challenge is that it is difficult for users who miss the last train to find an efficient and economical way to get home. Another challenge is the lack of a system that simultaneously evaluates travel time, cost, and distance to provide the optimal route when users are looking for a way to get home.

[1849] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1850] In this invention, the server includes means for acquiring the user's current location information, means for acquiring the user's destination information, means for acquiring information on the nearest public transportation based on the current location, means for calculating candidate routes available when the last train has departed, means for calculating the time, cost, and distance for each route, means for presenting multiple routes to the user, means for performing necessary actions based on the route selected by the user, means for calculating walking routes based on the proposed routes, means for estimating taxi fares based on the proposed routes, and means for searching for and displaying accommodation information based on the proposed routes. This makes it possible to efficiently and economically provide the optimal route home to a user who has missed the last train.

[1851] A "user" refers to an individual who uses the system to search for a route home.

[1852] "Current location information" refers to geographical data indicating the user's current location.

[1853] "Destination information" refers to geographical data about the location where the user is aiming to return home.

[1854] "Public transportation information" refers to timetables and operating status information for modes of transport such as trains and buses.

[1855] "Last train time" refers to the departure time and service information of the last train running on that day.

[1856] "Possible travel routes" refer to multiple routes that a user can choose to take to get home.

[1857] A "walking route" refers to the path a user takes to reach their destination on foot.

[1858] "Taxi fare estimate" refers to a predicted value of the fare incurred when using a taxi.

[1859] "Accommodation facilities" refer to facilities that users can use for temporary lodging (such as hotels and entertainment venues).

[1860] "Action" refers to a specific action taken as part of the means of getting home (e.g., calling a taxi, booking accommodation).

[1861] A "server" refers to a computer system that processes data and provides information based on requests from users.

[1862] "Terminal" refers to a device (such as a smartphone or tablet) that a user uses to access the system.

[1863] This invention is a system that suggests an efficient and economical route home for users who have missed the last train. The embodiments of this invention will be described in detail below.

[1864] This system is implemented through various operations performed by servers, terminals, and users. The hardware used includes terminals such as smartphones and tablets, while small computers and servers perform the main processing. The software used includes GPS functionality for acquiring location information and APIs (e.g., Google Maps API, NAVITIME, etc.) for acquiring public transportation information.

[1865] User information retrieval

[1866] The user launches the app on their device and enables automatic location acquisition. The device uses its GPS function to determine its current location and sends the location information to the server. If GPS is unavailable, the user can also manually enter their address. Furthermore, the user enters their destination information (where they will return home) into their device.

[1867] Obtain the nearest station and the last train time.

[1868] The device sends its current location information to the server. The server uses an API to retrieve public transportation information, specifically the nearest station and its last train time. This retrieved information is sent to the device and displayed to the user.

[1869] Suggestions for what to do if you miss the last train.

[1870] The server checks if the user has missed the last train by comparing the retrieved last train time with the current time. If the user has missed the last train, it calculates the following possible travel routes.

[1871] Route 1: Take the train to a station that is not the closest but is nearby, and then walk home from there. The server searches for the nearest other station from your current location to your destination and calculates the walking route.

[1872] Route 2: Go as far as you can by train, then take a taxi home. The server calculates the furthest point you can reach by train and estimates the taxi fare from that point to your destination.

[1873] Route 3: Walk home entirely. The server calculates the walking route from the current location to the destination and compiles the estimated time.

[1874] Route 4: Take a taxi home. The server estimates the taxi fare from your current location to your destination.

[1875] Route 5: Stay at a nearby accommodation and return home the next morning. The server searches for accommodations near your current location and displays reservation information.

[1876] Route optimization and suggestions

[1877] The server comprehensively calculates the time, cost, and distance for each potential travel route and selects the optimal route based on user-defined conditions (e.g., minimizing costs, prioritizing time). Finally, the server generates a route list in order of priority and sends it to the terminal along with detailed information.

[1878] Providing route guidance to users

[1879] The device displays the route list received from the server on the app screen. Details of each route (estimated time, cost, distance) are displayed visually, allowing the user to scroll and review them.

[1880] User selection and action execution

[1881] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen. Based on this selection, the terminal suggests and executes the necessary actions (e.g., calling a taxi, making a hotel reservation). Once the user completes the operation, the terminal automatically performs the instructed actions and contacts the taxi company or accommodation.

[1882] Specific example

[1883] scenario:

[1884] User: I'm at Shinjuku Station.

[1885] Destination: My home in Naka Ward, Yokohama City.

[1886] Terminal:

[1887] The nearest station is Shinjuku Station, and the last train time (e.g., 23:30) is checked using the public transport database. The server confirms that the last train has been missed from the current time (e.g., 23:45) and suggests the following route.

[1888] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[1889] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[1890] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[1891] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[1892] Route 5: Stay at an accommodation near Shinjuku and return home on the first train the next day (accommodation fee: 3000 yen).

[1893] user:

[1894] Select "Route 2" and then use the terminal to call a taxi.

[1895] Terminal:

[1896] The system contacts a taxi company and arranges for a taxi to be dispatched to the user's current location. This ensures that users have an efficient and economical way to get home, even if they miss the last train.

[1897] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1898] System program processing flow

[1899] Step 1:

[1900] The user launches the app on their smartphone or tablet. When the user presses the "Automatically acquire current location" button, the device enables its GPS function and acquires the current location. GPS data is input, and the current location information is output. Specifically, the device acquires data from the GPS sensor and displays it as the current location.

[1901] Step 2:

[1902] The user manually enters their current address (if GPS is unavailable). Once the user enters the address and presses the "Confirm" button, the manually entered address is output as the current location information. Specifically, the user's input is retrieved in text format and saved to an internal database.

[1903] Step 3:

[1904] The user enters their home address. Once the address is entered into the terminal's input form and the "Next" button is pressed, destination information is output. Specifically, the user's input is retrieved in text format and saved to an internal database.

[1905] Step 4:

[1906] The device sends its current location and destination information to the server. The current location and destination information are used as input, and the server receives this information as output. Specifically, the device sends data to the server via the internet.

[1907] Step 5:

[1908] The server accesses a public transport API (e.g., Google Maps API) to retrieve the nearest station and last train time. Current location information is used as input, and the output is data on the nearest station and last train time. Specifically, the server sends an API request and saves the received data to an internal database.

[1909] Step 6:

[1910] The server compares the current time with the last train time to determine if the user has missed the last train. The current time and last train time are used as input, and the output is a determination of whether the user missed the last train. Specifically, the server executes an algorithm to compare the current time with the last train time.

[1911] Step 7:

[1912] If you miss the last train, the server calculates possible travel routes. Current location information, destination information, and public transport information are used as input, and multiple travel routes are returned as output. Specifically, the following routes are calculated:

[1913] Route 1: Take the train to a station that is not the closest but is nearby from your current location, and then walk home from there.

[1914] The server searches for the nearest station from the current location and calculates the walking route. The output provides information about the station and the walking route.

[1915] Route 2: Go as far as you can by train, then take a taxi home.

[1916] The server calculates the final destination reachable by train and estimates the taxi fare. The output provides the final destination and the estimated taxi fare.

[1917] Route 3: Walk home entirely.

[1918] The server calculates the walking route from the current location to the destination. The output includes the walking route and the estimated time.

[1919] Route 4: Go home entirely by taxi.

[1920] The server estimates the taxi fare from the current location to the destination. The taxi fare is returned as output.

[1921] Route 5: Stay at a nearby accommodation and return home the next day.

[1922] The server searches for accommodations near the current location and displays a list of accommodations and reservation information. Accommodation information is provided as output.

[1923] Step 8:

[1924] The server calculates the time, cost, and distance for each route and selects the optimal route based on user-defined conditions. Route candidate data is used as input, and the output is a list of optimal routes. Specifically, the server executes a route evaluation algorithm and calculates the evaluation results.

[1925] Step 9:

[1926] The server sends the optimal route list to the terminal. The route list is used as input, and the terminal receives the route list as output. Specifically, the server sends data over the internet, and the terminal receives it.

[1927] Step 10:

[1928] The app displays the route list received by the device on the app screen. It visually displays the estimated time, cost, and distance for each route so the user can review the route details. Specifically, the device generates the display data and renders it on the screen.

[1929] Step 11:

[1930] The user selects a route and presses the "Select" button. The selected route information is used as input, and the selection result is obtained as output. Specifically, the user's selection is received and sent to the server.

[1931] Step 12:

[1932] The device performs actions based on the user's selections (e.g., calling a taxi, making a hotel reservation). User selection information is used as input, and the output is the result of executing a specific action. Specifically, the device calls the corresponding API to contact the taxi or accommodation provider.

[1933] (Application Example 1)

[1934] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1935] While systems already exist that suggest efficient and economical routes home for users who miss the last train, there is a similar need for technology that calculates efficient and economical delivery routes in the food delivery industry. In particular, since reducing time and costs is a critical issue in the food delivery industry, a system that can provide efficient delivery routes is essential.

[1936] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1937] In this invention, the server includes means for acquiring the user's current location information, means for acquiring destination information, means for acquiring information on the nearest public transport, means for calculating candidate available travel routes, means for calculating the time, cost, and distance for each travel route, means for presenting multiple travel routes, means for performing necessary actions based on the travel route selected by the user, and means for calculating an efficient delivery route for food delivery. This makes it possible to provide the optimal travel route in terms of both time and cost.

[1938] "User" refers to an individual or group that uses the system.

[1939] "Current location information" refers to the latitude and longitude data of the user's current location.

[1940] "Destination information" refers to data such as the address and latitude / longitude of the place the user is heading to.

[1941] "Public transportation" refers to mass transportation methods such as buses, trains, and subways.

[1942] "Possible travel routes" refer to multiple options for routes and means of transportation to reach a destination.

[1943] "Travel time" refers to the time required to travel from your current location to your destination.

[1944] "Cost" refers to the economic cost required to travel from your current location to your destination.

[1945] "Distance traveled" refers to the physical distance from your current location to your destination.

[1946] "Food delivery" refers to a service that delivers food and beverages to a specified location.

[1947] An "efficient delivery route" is a route optimized to minimize delivery time and costs.

[1948] "Means of performing actions" refer to methods such as calling a car or taxi or booking accommodation based on the travel route selected by the user.

[1949] This invention is a system that proposes efficient and economical delivery routes for food delivery. Based on the user's current location and destination information, this system calculates and presents a delivery route combining multiple modes of transportation to the user. Embodiments of this invention will be described in detail below.

[1950] Server Processing

[1951] The server first obtains the user's current location and destination information. Current location information is obtained using GPS or an IP geolocation service. Destination information is obtained using the Google Maps API from the address entered by the user, providing latitude and longitude. This ensures the server has accurate location information for both the current and destination locations.

[1952] Next, the server retrieves information about the nearest public transport. To do this, it consults a public transport database to obtain the nearest train station or bus stop and the time of the last train. If the last train has already departed, the server calculates route options for each mode of transportation (bicycle, public transport, walking).

[1953] Route options for each mode of transport are calculated based on their average speed, determining the travel time, cost, and distance. For example, a standard of 15 km / h is used for cycling, 30 km / h for public transport, and 5 km / h for walking. In this way, the server can obtain detailed information for all included routes.

[1954] Terminal processing

[1955] The terminal visually displays multiple route options sent from the server to the user. Each route clearly indicates the estimated travel time, cost, and distance, and presents the user with its advantages and disadvantages. Based on this information, the user can choose the most suitable mode of transportation.

[1956] Based on the route selected by the user, the device performs the necessary actions. For example, if the user requests a taxi or makes a hotel reservation, the device automatically arranges these using the corresponding API.

[1957] Specific example

[1958] User's current location: Shibuya Station

[1959] Destination: Restaurants near Ebisu Station

[1960] Optimal route: Travel by bicycle (approximately 10 minutes)

[1961] In this specific example, if the user is at Shibuya Station, the server obtains location information for both the user's current location and destination, and selects a bicycle as the most efficient route. This bicycle route is calculated to take approximately 10 minutes and is then presented to the user.

[1962] Example of a prompt

[1963] An example of a prompt message to input to a generative AI model is as follows:

[1964] text

[1965] Develop an application that suggests the most efficient route for delivery drivers when a user orders food delivery. Please use the following information as a basis.

[1966] Current location: Shibuya Station

[1967] Destination: Restaurants near Ebisu Station

[1968] Available modes of transportation: bicycle, public transport, walking

[1969] Calculate the travel time and distance for each mode of transportation and suggest the optimal route.

[1970] This allows users to obtain efficient routes in real time and reach their destinations quickly and economically.

[1971] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1972] Step 1:

[1973] The server obtains the user's current location information. Using GPS functionality or an IP geolocation service, it obtains the latitude and longitude of the user's current location from the user's device. This allows the server to obtain the user's precise location information as input data. The output data is the latitude and longitude information of the current location.

[1974] Step 2:

[1975] The user enters the destination address. The device retrieves this entered address data and uses the Google Maps API to obtain the corresponding latitude and longitude. The server receives this latitude and longitude information as input data and stores it as destination information. The output data is the latitude and longitude information of the destination.

[1976] Step 3:

[1977] The server retrieves information about the nearest public transport based on the user's current location. It consults a public transport database to obtain the nearest train station or bus stop, as well as the time of the last train or bus. The server receives this information as input data and stores it as information about the nearest public transport. The output data includes the name, location, and time of the last train or bus of the nearest public transport.

[1978] Step 4:

[1979] The server calculates possible travel routes. It calculates route options for each mode of transport (bicycle, public transport, walking) from the current location to the destination, and calculates the estimated time, cost, and distance for each. The server receives these calculation results as input data and stores them as travel route options. The output data consists of the estimated time, cost, and distance for each mode of transport.

[1980] Step 5:

[1981] The server presents the user with the results of each calculated travel route. The terminal visually displays the detailed information (estimated time, cost, distance) of each travel route received from the server. The user selects the optimal route based on this information. The input data is the route candidate information received from the server, and the output data is the route details displayed on the terminal.

[1982] Step 6:

[1983] The user selects the optimal route from several presented travel routes. The terminal receives the user's selection and sends it to the server. The input data is the route information selected by the user, and the output data is the selected route information sent to the server.

[1984] Step 7:

[1985] The server performs the necessary actions based on the travel route selected by the user. For example, it automatically arranges things like calling a taxi or booking accommodation using APIs corresponding to each route. The input data is the route information selected by the user, and the output data is the result of each arrangement action.

[1986] This allows users to obtain efficient and economical travel routes and take appropriate actions quickly.

[1987] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1988] ---

[1989] This invention combines a system that suggests efficient and economical routes home when a user misses the last train with an emotion engine that recognizes the user's emotions. Based on the user's current location and destination information, the system suggests the most suitable way home. By utilizing the emotion engine, the suggested travel route can be adjusted according to the user's emotional state.

[1990] ---

[1991] System Overview

[1992] This system acquires the user's current location and destination information and suggests the optimal route to get home efficiently and economically, even if the user misses the last train. It also recognizes the user's emotional state and adjusts the travel route based on those emotions.

[1993] ---

[1994] Program Processing Description

[1995] 1. Obtaining user information

[1996] User: Launch the app and use the automatic location acquisition function to determine your current location. If GPS functionality is unavailable, manually enter your address.

[1997] User: Enter your home address. This will identify your destination.

[1998] 2. Obtain the nearest station and the last train time.

[1999] Terminal: Searches for the nearest station from the current location and retrieves the last train time for that station from the public transport database.

[2000] 3. Recognizing the user's emotional state

[2001] Device: The app uses its in-app camera and microphone to recognize emotions from the user's facial expressions and tone of voice.

[2002] Device: The emotion engine analyzes the user's emotional state, identifying things like fatigue, stress, and relaxation.

[2003] 4. Proposed solutions for when you miss the last train.

[2004] Server: Check if the time for the last train has passed the current time.

[2005] Server: If you miss the last train, calculate the following route.

[2006] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[2007] Server: Searches for the nearest other train station from the current location to the home address and calculates the walking route from there.

[2008] Route 2: Go as far as you can by train, then take a taxi home.

[2009] Server: Calculates the furthest point reachable by train and estimates the taxi fare from there to home.

[2010] Route 3: Walk home entirely.

[2011] Server: Calculates the walking route from the current location to home and displays the estimated time.

[2012] Route 4: Go home entirely by taxi.

[2013] Server: Estimate the taxi fare from the current location to home.

[2014] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[2015] Server: Searches for entertainment and hotels near the current location and displays reservation information.

[2016] 5. Route Optimization and Suggestions

[2017] Server: Calculates the time, cost, and distance for each route, and adjusts the optimal route based on the user's emotional state.

[2018] Server: If the user is tired, prioritize suggesting a relaxing route; if they prioritize economical options, suggest a low-cost route.

[2019] Server: Generates a route list in order of priority and sends it to the terminal along with detailed information.

[2020] 6. Providing directions to the user

[2021] Terminal: Displays a list of each proposed route and visually shows the details of each route (estimated time, cost, distance, and emotional considerations).

[2022] 7. User selection and action execution

[2023] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen.

[2024] The device displays options to perform the necessary action (call a taxi, make a reservation for entertainment) based on the user's selection.

[2025] The user selects the appropriate action (for example, calling a taxi) and proceeds by following the on-screen instructions.

[2026] The device performs a specified action, for example, automatically contacting a taxi company.

[2027] ---

[2028] Specific example:

[2029] scenario:

[2030] User: I'm at Shinjuku Station.

[2031] Destination: My home in Naka Ward, Yokohama City.

[2032] User's emotions: Analysis of facial expressions and voice indicates that the user is tired.

[2033] Terminal: Confirm that Shinjuku Station is the nearest station and retrieve the last train time (e.g., 23:30) from the public transport database.

[2034] Server: Confirming that you have missed the last train at the current time (e.g., 23:45), the following route is suggested.

[2035] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[2036] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[2037] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[2038] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[2039] Route 5: Stay at a hotel near Shinjuku and return home on the first train the next day (hotel fee: 3000 yen).

[2040] Server: Since the server recognizes the user is tired, it prioritizes displaying the most comfortable and fastest "Route 4".

[2041] User: Select "Route 4" and initiate a taxi call on the device.

[2042] Terminal: Contact a taxi company and arrange for a taxi to your current location.

[2043] In this way, by utilizing the emotion engine, it is possible to provide the optimal way for users to return home according to their emotions, further reducing the burden on users.

[2044] The following describes the processing flow.

[2045] Step 1:

[2046] The user launches the app and uses the automatic location acquisition function to determine their current location. If GPS functionality is unavailable, the user manually enters their current address.

[2047] Step 2:

[2048] The user enters their home address. This allows the destination information to be identified.

[2049] Step 3:

[2050] The device searches for the nearest public transport station based on the user's current latitude and longitude. This identifies the user's nearest station.

[2051] Step 4:

[2052] The terminal retrieves the last train time for the nearest station from the public transport database. This database contains operational information for each station.

[2053] Step 5:

[2054] The server checks the current time and compares it to the last train time of the nearest station. It then determines whether the current time is past the last train time.

[2055] Step 6:

[2056] The device activates the user's facial recognition function and uses its built-in camera to capture the user's facial expressions. It also uses its voice recognition function to record what the user says and transfers this recording to emotion recognition software.

[2057] Step 7:

[2058] The server's emotion engine analyzes captured facial expressions and voice data to identify the user's emotional state (e.g., fatigue, stress, relaxation).

[2059] Step 8:

[2060] The server takes your emotional state into account and calculates the following return route. Specifically:

[2061] Route 1: Take the train to a station that's not the closest but is nearby, and then walk home from there.

[2062] The server searches for the nearest other train station from your current location to your home address and calculates the walking route from there.

[2063] Route 2: Go as far as you can by train, then take a taxi home.

[2064] The server calculates the furthest point you can reach by train and then estimates the taxi fare from there to your home.

[2065] Route 3: Walk home entirely.

[2066] The server calculates the walking route from the current location to home and displays the estimated time.

[2067] Route 4: Go home entirely by taxi.

[2068] The server estimates the taxi fare from your current location to your home.

[2069] Route 5: Stay at a nearby entertainment venue or hotel and return home the next day.

[2070] The server searches for entertainment and hotels near the current location and displays reservation information.

[2071] Step 9:

[2072] The server calculates the time, cost, and distance for each route and adjusts the optimal route based on the user's emotional state. If the user is highly fatigued, it prioritizes a comfortable and fast route.

[2073] Step 10:

[2074] The server generates a route list in order of priority and sends it to the terminal, along with detailed information.

[2075] Step 11:

[2076] The device displays a list of suggested routes and visually shows the details of each route (estimated time, cost, distance, and emotional considerations).

[2077] Step 12:

[2078] The user selects the most suitable route from the displayed options and presses the "Select" button on the screen.

[2079] Step 13:

[2080] The device displays options to perform the necessary action (call a taxi, make a reservation for entertainment) based on the user's selection.

[2081] Step 14:

[2082] The user selects the appropriate action (for example, calling a taxi) and proceeds by following the on-screen instructions.

[2083] Step 15:

[2084] The device performs the specified action, for example, automatically contacting a taxi company. It also completes the booking process if a reservation is required.

[2085] Step 16:

[2086] The server monitors the route and actions selected by the user, and updates route information or suggests alternative routes as needed.

[2087] Step 17:

[2088] The device tracks the user's current location and displays an alert if they get lost or need further assistance.

[2089] Step 18:

[2090] After the user finally arrives home, they can choose to exit the app and then it will close.

[2091] (Example 2)

[2092] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2093] In modern urban life, missing the last train is a common occurrence, and finding an efficient and economical way to get home is not easy. Furthermore, the lack of consideration for the user's emotional state can increase fatigue and stress. Conventional systems have been unable to suggest appropriate routes that take the user's emotional state into account.

[2094] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[2095] In this invention, the server includes means for acquiring the user's current location information, means for acquiring the user's destination information, and means for calculating multiple candidate travel routes. This makes it possible to propose the optimal travel route while taking into account the user's emotional state.

[2096] "Means of obtaining the user's current location information" refers to a function that determines the user's current location through GPS functionality or manual input by the user.

[2097] "Means for obtaining user destination information" refers to the function of inputting and obtaining information about the address or location set by the user as their destination.

[2098] "Means of obtaining information on the nearest public transport" refers to a function that searches for and retrieves the location and timetable of the nearest public transport based on the user's current location.

[2099] "Method for calculating multiple travel route options" refers to a function that calculates and suggests multiple travel routes that allow a user to reach their destination efficiently and economically, even if they miss the last train.

[2100] "Means for calculating the time, cost, and distance of each travel route" refers to a function that calculates in detail the time, cost, distance, etc., for each proposed travel route.

[2101] "Means of presenting multiple travel routes to the user" refers to a function that visually displays multiple calculated travel routes to the user, making it easier to compare and consider them.

[2102] "Means of performing necessary actions based on the travel route selected by the user" refers to a function that performs specific actions based on the travel route selected by the user, such as arranging a taxi or making a hotel reservation.

[2103] "A means of recognizing the user's emotional state and adjusting the travel route based on those emotions" refers to a function that uses cameras and microphones to analyze the user's emotions from their facial expressions and tone of voice, and then adjusts and suggests the optimal travel route according to those emotions.

[2104]

[2105] This invention is a system that suggests an efficient and economical route home when a user misses the last train, and further combines it with an emotion engine that recognizes the user's emotions to provide the optimal route according to the user's state. Specific embodiments of this invention are described below.

[2106] System Configuration

[2107] Hardware and software

[2108] 1. User's device:

[2109] Smartphone or tablet

[2110] Features include GPS, camera, and microphone.

[2111] 2. Server:

[2112] Servers with high-speed data processing capabilities

[2113] API that integrates with public transport databases

[2114] Emotion analysis engine

[2115] Specific names to use

[2116] GPS function: A standard geolocation API for obtaining the user's current location information.

[2117] Emotion analysis engine: OpenCV is used for facial expression analysis, and Azure Cognitive Services is used for speech analysis.

[2118] Public transport APIs: Google Maps API and public transport databases

[2119] Overview of program processing

[2120] 1. User's device:

[2121] Launch the app and use the GPS function to obtain your current location.

[2122] If the current location cannot be determined, the user will manually enter the address.

[2123] 2. User's device:

[2124] The user enters their home address as destination information.

[2125] 3. Terminal:

[2126] Use the public transport API to retrieve information about the nearest public transport.

[2127] Calculate the distance and travel time from your current location to the nearest station.

[2128] 4. Device:

[2129] The camera and microphone are activated to collect data on the user's face and voice.

[2130] The data is sent to an emotion analysis engine to analyze the user's emotional state (e.g., fatigue, stress, relaxation).

[2131] 5. Server:

[2132] Based on data transmitted from the device, it receives current location, destination, nearest station, last train time, and sentiment data.

[2133] The last train times are retrieved from the public transport database and compared with the current time.

[2134] 6. Server:

[2135] If you miss the last train, calculate multiple routes home.

[2136] For each route, calculate the required time, cost, and distance traveled.

[2137] 7. Server:

[2138] Based on the results of emotion analysis, the optimal route is selected. For example, if the user is tired, taking a taxi home is prioritized.

[2139] 8. Terminal:

[2140] Visually display multiple calculated routes. Detailed information for each route (estimated time, cost, distance, and emotional considerations) is displayed in a list.

[2141] 9. User:

[2142] Select the best route from the available options and press the "Select" button.

[2143] 10. Terminal:

[2144] Based on the selected route, the system will perform the necessary actions (such as calling a taxi or making a reservation for entertainment).

[2145] Specific example

[2146] scenario

[2147] User: I'm at Shinjuku Station.

[2148] Destination: My home in Naka Ward, Yokohama City.

[2149] User's emotions: Analysis of facial expressions and voice indicates that the user is tired.

[2150] Flow of operations

[2151] 1. Terminal: Confirm that Shinjuku Station is the nearest station and retrieve the last train time (e.g., 23:30) from the public transport database.

[2152] 2. Server: Confirms that the user has missed the last train from the current time (e.g., 23:45) and suggests the following route.

[2153] Route 1: Take the train to Shibuya Station and walk home from there (20-minute walk).

[2154] Route 2: Travel by train from Shinjuku Station to Yokohama, then take a taxi home (estimated taxi fare: 5000 yen).

[2155] Route 3: Walk home from Shinjuku Station to Naka Ward, Yokohama City (4 hours on foot).

[2156] Route 4: Take a taxi from Shinjuku and go directly home (estimated taxi fare: 8000 yen).

[2157] Route 5: Stay at a hotel near Shinjuku and return home on the first train the next day (hotel fee: 3000 yen).

[2158] 3. Server: Since the server recognizes that the user is tired, it prioritizes displaying the most comfortable and fastest "Route 4".

[2159] 4. User: Select "Route 4" and initiate a taxi call on the device.

[2160] 5. Terminal: Contact a taxi company and arrange for a taxi to your current location.

[2161] Example of a prompt

[2162] The following prompt statements can be used to generate a detailed explanation of the program's processing using a generative AI model.

[2163] Point down

[2164] We have developed a system that suggests efficient and economical routes home for users who have missed the last train. This system incorporates an emotion engine that recognizes the user's emotions. Please explain in detail how this system works, including specific processing steps and actions. Also, please include specific user actions and system responses in your explanation.

[2165] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2166] Step 1:

[2167] User: Launch the app and press the "Find Current Location" button using the GPS function. If GPS is enabled, your current location will be automatically acquired. If GPS is unavailable, you will be taken to the "Manual Address Input" screen and can manually enter your address.

[2168] Input: Location data from GPS, or manually entered address.

[2169] Output: User's current location information.

[2170] Specific operation: The app performs an operation to obtain the current location. If GPS is enabled, location information is obtained automatically. If GPS is disabled, the user enters an address.

[2171] Step 2:

[2172] User: Enter your home address in the address input field within the app and press the "Set Destination" button.

[2173] Input: Address data entered by the user.

[2174] Output: Destination information.

[2175] Specific operation: The user enters their home address in the address input field within the app and presses the submit button, registering the destination information in the system.

[2176] Step 3:

[2177] Terminal: Based on the current location information, it queries the public transport API to obtain information about the nearest public transport. At this time, it also calculates the distance and travel time from the current location to the nearest station.

[2178] Input: Current location information.

[2179] Output: Information on the nearest public transport, distance, and travel time.

[2180] Specific operation: The device sends its current location information to the public transport API to obtain information about the nearest train station or bus stop. Furthermore, it calculates the travel time and distance from the current location to the nearest public transport.

[2181] Step 4:

[2182] Device: Activates the camera and microphone and performs "facial expression analysis" and "voice analysis." Captures the user's face with the camera and sends the data to the emotion analysis engine.

[2183] Input: Face image captured by camera, voice recorded by microphone.

[2184] Output: User sentiment data.

[2185] Specific operation: The camera and microphone are used to collect the user's facial expressions and voice, and send them to an emotion analysis engine. As a result of the analysis, the user's emotional state is determined, such as "fatigue," "stress," or "relaxation."

[2186] Step 5:

[2187] Server: Receives current location information, destination information, nearest station information, last train time, and sentiment data sent from the terminal.

[2188] Input: Current location information, destination information, nearest station information, last train time, sentiment data.

[2189] Output: Integrated data of various types of information.

[2190] Specific operation: Integrates all the information sent from the terminal and stores it in a database. It also retrieves the last train time from the public transport database and compares it with the current time.

[2191] Step 6:

[2192] Server: If you have missed the last train, calculate several possible routes home. For each route, calculate the time required, cost, and distance traveled.

[2193] Input: Last train time, current time, destination information.

[2194] Output: Multiple travel route options, estimated travel time, cost, and distance for each route.

[2195] Specific actions: Compare the current time with the last train time, and if the last train has been missed, suggest multiple routes home. Calculate the travel time, cost, and distance for each route.

[2196] Step 7:

[2197] Server: Based on the results of sentiment analysis, the server selects the optimal route considering the user's emotional state. If the user is tired, priority is given to taking a taxi home.

[2198] Input: Candidate travel routes, sentiment analysis data.

[2199] Output: An optimal travel route based on emotions.

[2200] Specific operation: Based on the emotion analysis results, the system selects and adjusts the optimal travel route that reflects the user's emotional state.

[2201] Step 8:

[2202] Terminal: Displays a list of multiple routes sent from the server. Visually displays the estimated time, cost, distance, and sentiment considerations for each route.

[2203] Input: Route data sent from the server.

[2204] Output: A list of routes presented to the user.

[2205] Specific operation: Visually display detailed data for each route and present it in a list format for easy comparison by the user.

[2206] Step 9:

[2207] User: Select the best route from the multiple routes presented and press the "Select" button on the screen.

[2208] Input: User-selected route.

[2209] Output: Selected travel route.

[2210] Specific operation: The user selects the optimal route from the list of routes on the screen and presses the "Select" button.

[2211] Step 10:

[2212] Terminal: Performs necessary actions (such as calling a taxi or making a reservation) based on the travel route selected by the user. If calling a taxi, it contacts a taxi company and arranges for a taxi to be dispatched to the user's current location.

[2213] Input: The travel route selected by the user.

[2214] Output: Action performed (e.g., arranging a taxi).

[2215] Specific operation: Based on the route selected by the user, the device automatically performs the necessary actions. If a taxi is requested, it will automatically contact a taxi company and arrange for a taxi to be dispatched to the user's current location.

[2216] (Application Example 2)

[2217] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2218] In modern society, it is particularly difficult to return home efficiently and economically if one misses the last train. However, there is a need not only to provide transportation, but also to propose the optimal way to return home that takes into account the user's emotional state. This invention aims to solve this problem by proposing a system that provides a comfortable and efficient way to return home, taking into account the emotional state of the user when they miss the last train.

[2219] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[2220] In this invention, the server includes means for acquiring the user's current location information, means for acquiring the user's destination information, means for acquiring information on the nearest public transportation based on the current location, means for calculating candidate travel routes available when the time for the last train has passed, means for calculating the time, cost, and distance for each travel route, means for recognizing the user's emotional state and adjusting the travel route based on the emotional state, means for prioritizing the presentation of comfortable travel options when the user is fatigued based on the emotional state, means for presenting the user with multiple travel routes, means for performing necessary actions based on the travel route selected by the user, and means for summoning an autonomous vehicle and providing a means of transportation from the user's current location to their destination. This enables the provision of an optimal way to return home that takes the user's emotional state into consideration, allowing for a comfortable and efficient return home.

[2221] A "user" is an individual who uses the system to input their current location and destination information and receive suggestions for the optimal route home.

[2222] "Current location information" refers to the location information of the user's current location when using the system.

[2223] "Destination information" refers to the location information of the place the user wants to return to using the system.

[2224] "Public transport" refers to means of transportation such as trains, buses, and ferries that operate according to specific routes and schedules.

[2225] "Last train" refers to the time when public transportation services cease operations for the day.

[2226] A "travel route" refers to the path or method a user takes to travel from their current location to their destination.

[2227] "Travel time" refers to the time it takes to reach a destination using a specific travel route.

[2228] "Expenses" refer to the financial expenditure necessary to use a particular travel route.

[2229] "Distance traveled" refers to the actual distance traveled along a specific route.

[2230] "Emotional state" refers to the user's psychological or physical condition, including, for example, fatigue, stress, and relaxation.

[2231] An "emotion engine" is a program that recognizes and analyzes a user's emotional state.

[2232] An "action" is a specific operation or procedure performed based on the travel route selected by the user.

[2233] An "autonomous vehicle" is a vehicle that does not require a driver and is operated automatically by a computer.

[2234] This invention is a system that suggests an efficient and economical route home when a user misses the last train, and it recognizes the user's emotional state to provide the optimal way to get home. This system uses a smartphone as its main interface and combines an emotion recognition engine with an autonomous vehicle API to suggest a route home that meets the user's individual needs.

[2235] The system consists of the following elements:

[2236] 1. Obtaining user information

[2237] The user launches a smartphone application and enters their current location and destination information. Current location information is either automatically obtained using GPS or entered manually. Destination information is also entered manually by the user.

[2238] 2. Recognition of emotional states

[2239] The system uses the smartphone's camera and microphone to analyze the user's facial expressions and voice tone to recognize their emotional state. This process utilizes software called EmotionRecognition.

[2240] 3. Calculation of the optimal route

[2241] Based on your current location, destination, and emotional state, the system calculates the optimal route home. The calculated route includes factors such as travel time, cost, distance, and comfort level.

[2242] 4. Provision of transportation

[2243] Based on the route calculated and determined to be the most suitable by the user, an autonomous vehicle is dispatched. The SelfDrivingCarAPI is used to select the appropriate autonomous vehicle and dispatch it to the user's specified current location.

[2244] Specifically, the system operates in the following scenario:

[2245] scenario:

[2246] If a user is at Shinjuku Station, has missed the last train, and is trying to return home to Naka Ward, Yokohama City, the user's smartphone app is launched, retrieves "Shinjuku Station" as the current location, and inputs "Naka ​​Ward, Yokohama City" as the destination. If the emotion recognition engine determines from the user's facial expression and voice that they are "tired," the system prioritizes displaying a comfortable route home.

[2247] For example, routes such as "traveling by train from Shinjuku to Yokohama and then taking a taxi from there" or "taking a taxi directly home from Shinjuku" are suggested. If the user chooses the latter, an autonomous vehicle will pick them up at Shinjuku Station and drop them off at their home in Naka Ward, Yokohama City.

[2248] The specific software and hardware used to perform the above processes are as follows:

[2249] Software used:

[2250] EmotionRecognition (emotion recognition engine)

[2251] SelfDrivingCarAPI (Autonomous Vehicle API)

[2252] Hardware used:

[2253] Smartphone (acquisition of user's current location information and sentiment recognition)

[2254] Autonomous vehicles (providing transportation to the user's destination)

[2255] Prompt example:

[2256] Create an app that, when a user misses the last train, will summon an autonomous vehicle and suggest an efficient and economical route home. The app will obtain the user's current location and destination, and calculate the optimal route based on the user's emotional state. Specifically, it will prioritize displaying comfortable routes that take into account the user's fatigue level, and summon an autonomous taxi if necessary.

[2257] In this way, the present invention can provide an optimal method of returning home that takes into account the user's emotional state, making it possible to return home comfortably and efficiently.

[2258] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2259] Step 1:

[2260] The user launches the application on their smartphone and enters their current location and destination information.

[2261] Input: User's current location (GPS information or manual input) and destination (manual input).

[2262] Operation: The app automatically obtains the current location using GPS, or the user manually enters the current location and destination.

[2263] Step 2:

[2264] The device retrieves information about the nearest public transportation based on the user's current location.

[2265] Input: Current location information.

[2266] Function: Searches the public transport database for the nearest train station or bus stop and retrieves that information.

[2267] Output: Information on the nearest public transport.

[2268] Step 3: 【22...

Claims

1. A means of obtaining the user's current location information, A means of obtaining the user's destination information, A means of obtaining information about the nearest public transport based on the current location, A method for calculating possible travel routes available after the last train has departed, A means of calculating the time required, cost, and distance traveled for each travel route, A means of presenting the user with multiple travel routes, A means of performing the necessary actions based on the travel route selected by the user, A system that includes this.

2. The system according to claim 1, which includes means for calculating a route that involves using public transportation to a specific point and then walking from there, as a candidate for the aforementioned travel route.

3. The system according to claim 1, which includes means for calculating a route that uses public transportation to a specific point and then takes a taxi from there as a candidate for the aforementioned travel route.

Images