System

Abstract

A system is provided.SOLUTION: A system comprising: location information obtaining means for obtaining a current location; input means for receiving input information relating to a destination; traffic information obtaining means for obtaining traffic information in real time based on the current location obtained by the location information obtaining means and the destination received by the input means; route calculation means for calculating an optimum route based on the traffic information obtained by the traffic information obtaining means; and route providing means for providing the optimum route calculated by the route calculation means to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When using conventional public transportation, users had to refer to timetables and multiple information sources and determine the optimal route themselves. This process was time-consuming and inconvenient, especially when delays or route changes occurred. Furthermore, users were unable to obtain real-time traffic conditions and operation information in a timely manner, which resulted in inefficient travel to their destinations. Therefore, there was a need to improve the convenience of public transportation for users. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides the following means: a system including: location information acquisition means for acquiring a current location; input means for receiving input information related to a destination; traffic information acquisition means for acquiring traffic information in real time based on the current location acquired by the location information acquisition means and the destination received by the input means; route calculation means for calculating an optimal route based on the traffic information acquired by the traffic information acquisition means; and route provision means for providing a user with the optimal route calculated by the route calculation means. This system enables a user to quickly and accurately determine the optimal route to their destination based on the real-time traffic information.

[0006] "Location information acquisition means" refers to a device or function for acquiring the user's current location. Specifically, it includes a GPS sensor.

[0007] "Input means" refers to a device or function that allows a user to input information such as a destination. Specifically, it includes the input interface of a smartphone or a computer.

[0008] "Traffic information acquisition means" refers to a device or function for acquiring traffic information in real time based on the current location and destination. Specifically, this includes API access via the Internet.

[0009] The "route calculation means" is a device or function for calculating the optimum route based on the acquired traffic information. Specifically, it includes processing using an algorithm.

[0010] The "route providing means" refers to a device or function for providing the user with the calculated optimal route. Specifically, this includes screen display and voice guidance. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0012] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0013] First, the terms used in the following description will be explained.

[0014] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0015] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0016] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0017] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0019] [First embodiment]

[0020] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0021] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0024] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0027] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0031] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0032] The present invention relates to a system that suggests public transportation options in real time that will allow a user to arrive at their destination from their current location in the shortest time. This system includes a location information acquisition means, an input means, a traffic information acquisition means, a route calculation means, and a route provision means. Details of each means and the operation of the system are explained below.

[0033] 1. Obtaining location information

[0034] 1.1 Getting your current location

[0035] Device: Uses the GPS sensor to obtain current location. The device's GPS chip receives signals from satellites and obtains latitude and longitude information. This information is passed to the application.

[0036] 1.2 Entering your destination

[0037] User: Enters a destination through the application interface, for example, "Shinjuku Station."

[0038] Terminal: Passes the destination entered by the user to the application.

[0039] 2. Traffic information inquiry

[0040] 2.1 Making an API request

[0041] Device: Creates data for requesting the real-time traffic information API based on the current location and destination information, including latitude and longitude information and text information for the destination.

[0042] 2.2 Submitting a Request

[0043] Terminal: Sends the created request to the traffic information API. Sends data securely using the HTTPS protocol.

[0044] 3. Calculating the optimal route

[0045] 3.1 Receiving a Request

[0046] Server: Receives requests from the terminal via the traffic information API.

[0047] 3.2 Querying the Database

[0048] Server: Based on the received request, queries the transport company's database, which contains information on timetables, operation status, delays, etc.

[0049] 3.3 Obtaining real-time information

[0050] Server: Obtain real-time information on transportation status, such as delays on the Yamanote Line.

[0051] 3.4 Optimal Route Calculation

[0052] Server: Based on the acquired information, the server calculates the optimal route from the current location to the destination. The calculation takes into account the travel time, number of transfers, and whether or not there are any delays. Specifically, route search algorithms such as the Dijkstra algorithm and the A algorithm are used.

[0053] 4. Route provision

[0054] 4.1 Generating calculation results

[0055] Server: Generates the results of the calculation in a data format such as JSON, which contains the following information:

[0056] Departure and arrival points

[0057] Route name to be used

[0058] How to transfer

[0059] Estimated arrival time

[0060] 4.2 Sending a Response

[0061] Server: Sends the calculation results to the terminal.

[0062] 4.3 Receiving and Displaying Data

[0063] Terminal: Parses the calculation results received from the server and displays them in a user-friendly format. Route guidance, timetables, transfer instructions, etc. are displayed on the screen.

[0064] 4.4 User Visibility

[0065] User: Looks at the device screen to see the optimal route displayed, and receives instructions on which train or bus to take next, where to transfer, etc.

[0066] Example: From Shibuya Station to Shinjuku Station

[0067] 1. Obtaining location information

[0068] Device: Obtain current location (Shibuya Station) using GPS.

[0069] User: Enter "Shinjuku Station" in the app's search box.

[0070] 2. Traffic information inquiry

[0071] Terminal: Generate API request data including latitude and longitude information (Shibuya Station) and destination "Shinjuku Station".

[0072] Terminal: Sends the generated request to the API.

[0073] 3. Calculating the optimal route

[0074] Server: Receives API requests and sends queries to the database.

[0075] Server: Obtain information about delays on the Yamanote Line.

[0076] Server: Calculates a route that uses the Keio Line instead of the Yamanote Line.

[0077] 4. Route provision

[0078] Server: Generate calculation results (Shibuya Station to Shinjuku Station, using the Keio Line) in JSON format.

[0079] Server: Sends the calculation results to the terminal.

[0080] Terminal: Parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take approximately 8 minutes."

[0081] User: Follow the instructions displayed and board the Keio Line.

[0082] In this way, the user can receive optimal route instructions utilizing real-time information. The specific processing steps will be explained separately.

[0083] The processing flow will be explained below.

[0084] Step 1:

[0085] The device uses the GPS sensor to obtain the latitude and longitude of the current location, and passes this information to the application.

[0086] Step 2:

[0087] The user uses the application interface to enter a destination, for example, "Shinjuku Station."

[0088] Step 3:

[0089] The device creates request data based on the current location information acquired and the destination entered by the user. This data includes the latitude and longitude of the current location and the name of the destination.

[0090] Step 4:

[0091] The request data created by the device is sent to the traffic information acquisition API. The data is sent securely using protocols such as HTTPS.

[0092] Step 5:

[0093] The server receives the request sent from the terminal.

[0094] Step 6:

[0095] Based on the request received by the server, the server queries the transport company database to obtain information such as transport schedules, operation status, and delay information.

[0096] Step 7:

[0097] The server obtains real-time information about the operation status of public transport, such as delays on the Yamanote Line.

[0098] Step 8:

[0099] Based on the information acquired by the server, the optimal route from the current location to the destination is calculated, taking into account the required time, number of transfers, and information on train delays.

[0100] Step 9:

[0101] The server generates the calculation results in data format such as JSON, which includes the departure point, arrival point, route name, transfer method, estimated arrival time, etc.

[0102] Step 10:

[0103] The server sends the generated calculation results to the terminal.

[0104] Step 11:

[0105] The terminal receives the calculation results sent from the server, parses the received data, and displays it in a format that is easy for the user to view.

[0106] Step 12:

[0107] The user looks at the device screen to see the optimal route displayed, and then receives instructions on which public transport to use next, where to transfer, etc.

[0108] Example 1

[0109] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0110] Currently, when users reach their destination using public transportation, it is difficult to determine the optimal route that takes into account real-time traffic information and service delays. Furthermore, when transferring between multiple modes of transportation, there are few systems that provide integrated transfer information, which often makes users' travel complicated. Therefore, there is a need for a system that instantly provides the optimal route based on real-time information.

[0111] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0112] In this invention, the server includes a location information acquisition means for acquiring the current location, an input means for receiving input information about the destination, and a means for acquiring traffic information in real time, thereby making it possible to provide an optimal route taking into account real-time traffic conditions.

[0113] "Location information acquisition means" refers to devices or functions for acquiring the user's current location.

[0114] "Input means" refers to an interface or device for receiving destination-related information from a user.

[0115] "Traffic information acquisition means" refers to functions and devices for acquiring traffic information in real time.

[0116] "Optimal route calculation means" refers to a function or device for calculating the optimal travel route based on acquired traffic information and transfer information.

[0117] "Route providing means" refers to a means for providing information on the calculated optimal route to a user.

[0118] "Satellite Positioning System Sensor" means a positioning system that uses signals from satellites to determine location on Earth.

[0119] "Web Application Programming Interface" refers to a means for communicating with external software or applications through web services.

[0120] "Transportation information" refers to information such as public transportation schedules, operation status, and delay information.

[0121] The present invention relates to a system that suggests public transportation options in real time that will allow a user to arrive at their destination from their current location in the shortest time. This system is composed of the following means, including location information acquisition means, input means, traffic information acquisition means, optimal route calculation means, and route provision means.

[0122] Obtaining location information

[0123] Get current location

[0124] Device: The GPS sensor is used to obtain the user's current location. The GPS chip in the device receives signals from satellites, extracts latitude and longitude information, and passes it to the application.

[0125] Enter your destination

[0126] User: Enters a destination through the application interface. For example, the user enters "Shinjuku Station."

[0127] Terminal: Passes the destination information entered by the user to the application.

[0128] Traffic information inquiry

[0129] Making API Requests

[0130] Device: Creates request data for the real-time traffic information API based on the current location and destination information. This data includes the acquired latitude and longitude information and the destination information entered by the user.

[0131] Submitting a Request

[0132] Terminal: Sends the created request data to the API using the HTTPS protocol.

[0133] Calculating the best route

[0134] Receiving a request

[0135] Server: Queries the transit agency database based on requests received via the API.

[0136] Server: The database contains information on transportation schedules, operation status, delays, etc.

[0137] Obtaining real-time information

[0138] Server: Obtains real-time information about the operation of public transport. For example, obtains information such as "The Yamanote Line is currently delayed by 5 minutes."

[0139] Calculating the best route

[0140] Server: Based on the acquired data, the server calculates the optimal route from the user's current location to the destination. The calculation takes into account the travel time, number of transfers, and real-time delay information. The specific algorithms used for route search are the Dijkstra algorithm and the A algorithm.

[0141] Route provision

[0142] Generate calculation results

[0143] Server: Generates the optimal route calculation results in data format such as JSON. This data includes information such as the departure point, arrival point, route name, transfer method, and estimated arrival time.

[0144] Sending a response

[0145] Server: Sends the generated calculation results to the terminal.

[0146] Receiving and displaying data

[0147] Terminal: Parses the calculation results received from the server and displays them in a user-friendly format. For example, route guidance, timetables, transfer procedures, etc. are displayed on the screen.

[0148] User Visibility

[0149] User: Check the information displayed on the device and follow the instructions to begin traveling. For example, follow the instructions such as "Take the Keio Line to Shinjuku Station, it will take about 8 minutes."

[0150] Example: From Shibuya Station to Shinjuku Station

[0151] Get current location:

[0152] Device: Obtain current location (Shibuya Station) using GPS.

[0153] Enter your destination:

[0154] User: Enters "Shinjuku Station" into the app's search box.

[0155] Terminal: Obtain the entered destination information.

[0156] Make an API request:

[0157] Terminal: Generate API request data including latitude and longitude information (Shibuya Station) and destination "Shinjuku Station".

[0158] Sending an API request:

[0159] Terminal: Sends the generated request data to the API.

[0160] Query the database:

[0161] Server: Receives API requests and sends queries to the transit agency database.

[0162] Get real-time information:

[0163] Server: Obtain delay information for the Yamanote Line.

[0164] Calculating the best route:

[0165] Server: Calculate a route that avoids delays on the Yamanote Line and uses the Keio Line.

[0166] Generate calculation results:

[0167] Server: Generate the calculation result (Shibuya Station to Shinjuku Station, using the Keio Line) in JSON format.

[0168] Sending a response:

[0169] Server: Sends the calculation results to the terminal.

[0170] Receiving and displaying data:

[0171] Terminal: Parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take approximately 8 minutes."

[0172] Check the results and follow the instructions:

[0173] User: Follow the displayed instructions and use the Keio Line.

[0174] Example prompt sentence:

[0175] "I'm currently at Shibuya Station and my destination is Shinjuku Station. Please tell me the best route taking real-time traffic information into account."

[0176] This system allows users to receive optimal route instructions based on real-time information, enabling them to use public transportation efficiently.

[0177] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0178] Step 1: Obtaining location information

[0179] Device: The device uses a GPS sensor to obtain the user's current location. Specifically, the device's GPS chip receives signals from satellites and generates latitude and longitude information. This location information is passed to the application. The input is the signal from the GPS, and the output is the user's latitude and longitude information.

[0180] Step 2: Enter your destination

[0181] User: The user inputs a destination through the application interface. For example, they input "Shinjuku Station." The input is text input by the user, and the output is the destination information "Shinjuku Station."

[0182] Terminal: The terminal passes the destination information entered by the user to the application. The input is text data from the user, and the output is the destination information passed to the application.

[0183] Step 3: Making an API Request

[0184] Terminal: The terminal creates request data for the real-time traffic information API based on the current location (latitude and longitude information) and destination information. Specifically, this information is combined and placed in a request format. The input is latitude and longitude information and destination information, and the output is the API request data.

[0185] Step 4: Sending an API Request

[0186] Terminal: The terminal uses the HTTPS protocol to send the created request data to the API. The input is the API request data, and the output is communication to the API server.

[0187] Step 5: Receiving the request

[0188] Server: The API server receives the request data sent from the terminal. The input is the request data, and the output is a confirmation of the received request data.

[0189] Step 6: Query the database

[0190] Server: The server queries the traffic information database based on the received request. Specifically, it searches for schedule information, operation status, and delay information. The input is the request data, and the output is the traffic information retrieved from the database.

[0191] Step 7: Get real-time information

[0192] Server: The server obtains real-time operational status of transportation facilities. For example, it obtains information such as "The Yamanote Line is currently delayed by 5 minutes." The input is real-time data of transportation facilities, and the output is the latest operational status.

[0193] Step 8: Calculate the optimal route

[0194] Server: The server calculates the optimal route from the current location to the destination based on the acquired data. It uses the Dijkstra algorithm and the A algorithm to take into account the travel time, number of transfers, and real-time delay information. The input is traffic information and real-time operation status, and the output is the optimal route information.

[0195] Step 9: Generate calculation results

[0196] Server: The server generates the optimal route calculation results in data format such as JSON. This data includes the departure point, arrival point, route name to be used, transfer method, and estimated arrival time. The input is the optimal route information, and the output is data in JSON format.

[0197] Step 10: Sending the Response

[0198] Server: The server sends the generated calculation results to the terminal. The input is JSON format data, and the output is the response to the terminal.

[0199] Step 11: Receiving and displaying data

[0200] Terminal: The terminal parses the calculation results received from the server and displays them in a user-friendly format. Route guidance, timetables, and transfer procedures are displayed on the screen. The input is the response data from the server, and the output is the information displayed on the user interface.

[0201] Step 12: Review the results and follow the instructions

[0202] User: The user checks the information displayed on the device and starts moving according to the instructions. For example, they follow instructions such as "Take the Keio Line to Shinjuku Station, it will take about 8 minutes." The input is the information displayed on the device, and the output is the user's actions.

[0203] (Application example 1)

[0204] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0205] While many conventional public transportation guidance systems provide the optimal route from a user's current location to their destination, they are inadequate when it comes to providing destination guidance within shopping malls and large facilities. Providing adequate guidance is particularly difficult when a user has multiple destinations or when congestion needs to be taken into consideration. There is also a growing need for intuitive operation using voice input in addition to visual guidance information. The present invention aims to solve these problems by providing a system that provides real-time guidance on the shortest route to a destination within a shopping mall.

[0206] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0207] In this invention, the server includes location information acquisition means for acquiring location information, input means for receiving input information related to a destination, information acquisition means for acquiring traffic information in real time, route calculation means for calculating an optimal route, route provision means for providing the optimal route to a user, display means for visually guiding the optimal route based on real-time congestion status, and voice input means for inputting a destination using voice recognition, thereby making it possible to provide the shortest route to a destination within a shopping mall in real time by visual and voice input.

[0208] "Location information acquisition means" refers to a device or system for acquiring the user's current location, and utilizes a satellite system or GPS sensor.

[0209] "Input means" refers to a device or system for receiving information about a destination from a user, and includes a touch panel, a keyboard, a voice recognition system, and the like.

[0210] The "information acquisition means" is a device or system for acquiring data such as traffic information and congestion status in real time, and acquires data via a network.

[0211] The "route calculation means" is a device or system for calculating the optimum route based on the acquired information, and uses a route search algorithm that takes into account the required travel time and traffic congestion.

[0212] "Route providing means" refers to a device or system for providing the user with the calculated optimum route, and conveys information through a screen display, voice guidance, etc.

[0213] "Display means" refers to a device or system that visually guides passengers along optimal routes based on real-time congestion information, and uses smart glasses or a head-mounted display.

[0214] "Voice input means" refers to a device or system that utilizes voice recognition technology to receive destination information from a user.

[0215] The embodiment of the present invention is directed to constructing a system that guides users so that they can reach their destination from their current location in the shortest time possible. This system is particularly effective for providing guidance within large facilities such as shopping malls. Detailed embodiments of this system are described below.

[0216] 1. System Configuration

[0217] The system consists of the following major components:

[0218] Location information acquisition means

[0219] Input Method

[0220] Information acquisition means

[0221] Route calculation method

[0222] Route provision method

[0223] Display means

[0224] Voice input means

[0225] Location information acquisition means

[0226] Location information acquisition means has the function of acquiring the user's current location. It mainly uses satellite systems and GPS sensors to acquire the current latitude and longitude information. For example, a GPS chip installed in smart glasses measures the user's location in real time.

[0227] Input Method

[0228] The input means has the function of receiving information about the destination from the user. This can be done using a touch panel, keyboard, or voice recognition system. Specifically, it is possible to use the voice recognition system built into the smart glasses to input destinations such as "supermarket" or "toilet" by voice.

[0229] Information acquisition means

[0230] The information acquisition means has the function of acquiring data such as traffic information and congestion status in real time. This data is acquired via a network, for example, using the Google (registered trademark) Maps API or a shopping mall's dedicated API.

[0231] Route calculation method

[0232] The route calculation means has the function of calculating the optimal route based on the acquired information. For this, a route search algorithm (such as the Dijkstra algorithm or the A algorithm) is used that takes into account the required time and congestion status.

[0233] Route provision method

[0234] The route providing means has the function of providing the user with the calculated optimal route, and displays visual guidance information on the smart glasses display or the smartphone screen.

[0235] Display means

[0236] The display function visually guides users to the optimal route based on real-time congestion information, primarily using smart glasses or head-mounted displays to visually show the user the next direction to go and the route to their destination.

[0237] Voice input means

[0238] The voice input means has the function of receiving destination information from the user using voice recognition technology, using a microphone and voice recognition software built into the smart glasses.

[0239] Program processing explanation

[0240] This system works in conjunction with each mode of transportation to provide users with optimal route guidance. The specific processing flow is explained below.

[0241] 1. Location information acquisition means

[0242] It uses a GPS sensor to obtain the user's current location and transmits that information to a server. For example, the GPS chip in smart glasses measures the user's location in real time and passes the latitude and longitude data to the application.

[0243] 2. Input Method

[0244] The user inputs their destination through a voice input system, for example by saying "I'm going to the supermarket," and the voice recognition software recognizes the destination and sends that information to the server.

[0245] 3. Information acquisition means

[0246] The server makes a request to a real-time congestion information API based on the acquired current location and destination information. Specifically, it accesses the Google Maps API or a congestion information API dedicated to shopping malls.

[0247] 4. Route calculation method

[0248] The server calculates the shortest route using the Dijkstra algorithm or A algorithm based on the acquired congestion information. For example, it calculates a route that avoids congested areas.

[0249] 5. Route provision method

[0250] The server sends the calculation results in JSON format to the smart glasses, which then display arrows and text instructions to the user based on the route information received.

[0251] Examples of concrete examples and prompts

[0252] Hardware / Software used

[0253] Reference Hardware: Smart glasses, e.g. Vuzix Blade or North Focals

[0254] Real-time location information acquisition API: Google Maps API

[0255] Server: AWS (registered trademark) EC2

[0256] Real-time congestion information API: SmartMall API (tentative)

[0257] Prompt Sentence Examples

[0258] Generative AI model used: GPT-4 (registered trademark)

[0259] Example prompt: "Code the process flow for a smart glasses app that provides visual guidance on the shortest route from a user's current location to a destination. The destination is specified via voice input, and the route display takes real-time information into account."

[0260] This makes it possible to provide the shortest route to a destination within a shopping mall in real time through visual and voice input.

[0261] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0262] Step 1:

[0263] Get current location

[0264] Subject: Device

[0265] What it does: Uses the device's GPS sensor to obtain the user's current location.

[0266] Input: The device's GPS chip receives signals from satellites and obtains latitude and longitude information.

[0267] Output: Latitude and longitude information (e.g., current location coordinates). The device passes this information to the next processing step.

[0268] Step 2:

[0269] Enter your destination

[0270] Subject: User

[0271] How it works: The user enters destination information through the voice recognition system.

[0272] Input: The user's spoken command (e.g., "Go to the supermarket").

[0273] Output: Textual destination information (e.g., destination name) is generated and passed to the terminal.

[0274] Step 3:

[0275] Obtaining real-time information

[0276] Subject: Device

[0277] How it works: The device sends a request to a real-time congestion information API based on its current location and destination.

[0278] Input: Latitude and longitude information of current location, text information of destination.

[0279] Output: Real-time congestion data (e.g., congestion and traffic information obtained from an API) is returned to the device.

[0280] Step 4:

[0281] Calculating the best route

[0282] Subject: Server

[0283] How it works: The server calculates the optimal route based on the real-time data it receives.

[0284] Input: current location latitude and longitude, destination text, real-time congestion data.

[0285] Data processing / calculation: Using a path-finding algorithm (e.g., Dijkstra algorithm or A algorithm), calculate the route that can be reached in the shortest time.

[0286] Output: The calculated optimal route (e.g., a list of route information) is generated.

[0287] Step 5:

[0288] Route information format conversion

[0289] Subject: Server

[0290] How it works: The server converts the optimal route information into a data format such as JSON.

[0291] Input: The calculated optimal route information.

[0292] Output: Route information (e.g., navigation data) in JSON format is generated and sent to the device.

[0293] Step 6:

[0294] View route information

[0295] Subject: Device

[0296] How it works: The device parses the route information it receives and displays it visually to the user.

[0297] Input: Route information in JSON format.

[0298] Data processing / computation: The device parses the JSON data and generates a visual navigation display.

[0299] Output: Route guidance is displayed on the smart glasses display (e.g., arrows and text instructions showing the direction of travel).

[0300] Step 7:

[0301] User movement guidance

[0302] Subject: User

[0303] How it works: The user follows the guidance displayed on the smart glasses to reach their destination.

[0304] Input: Visual navigation information (e.g., arrows and text instructions).

[0305] Output: The user arrives at the destination.

[0306] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0307] The present invention combines an emotion engine with a system that suggests in real time the public transportation that will take a user from their current location to their destination in the shortest time. This system includes a location information acquisition means, an input means, a traffic information acquisition means, a route calculation means, a route provision means, and an emotion recognition means. Details of each means and the operation of the system are described below.

[0308] 1. Obtaining location information

[0309] 1.1 Getting your current location

[0310] The device uses the GPS sensor to obtain the latitude and longitude of the current location, and passes this information to the application.

[0311] 1.2 Entering your destination

[0312] The user uses the application interface to enter a destination, for example, "Shinjuku Station."

[0313] The device passes the destination entered by the user to the application.

[0314] 2. Emotional Recognition

[0315] 2.1 Acquiring Emotion Data

[0316] The device uses a microphone and camera to capture the user's voice and facial images.

[0317] The terminal passes the voice and image data to the emotion recognition means.

[0318] 2.2 Emotion Data Analysis

[0319] The server analyzes the received audio or image data to recognize the user's emotions. For example, it can identify whether the user is anxious, impatient, calm, etc. based on changes in voice tone or facial expression.

[0320] 3. Traffic information inquiry

[0321] 3.1 Making an API request

[0322] Request data is created based on the current location information acquired by the device, the destination entered by the user, and the recognized emotion data.

[0323] 3.2 Sending a Request

[0324] The request data created by the device is sent to the traffic information acquisition API. The data is sent securely using protocols such as HTTPS.

[0325] 4. Calculating the optimal route

[0326] 4.1 Receiving a Request

[0327] The server receives the request sent from the terminal.

[0328] 4.2 Querying the Database

[0329] Based on the request received by the server, the server queries the transport company database to obtain information such as transport schedules, operation status, and delay information.

[0330] 4.3 Obtaining real-time information

[0331] The server obtains real-time information about the operation status of public transport, such as delays on the Yamanote Line.

[0332] 4.4 Optimal Route Calculation

[0333] The server calculates the optimal route from the current location to the destination based on the information acquired and the user's emotional state. The calculation takes into account the required travel time, number of transfers, and service delay information, as well as the user's emotional state. Specifically, if the user is in a hurry, the system will choose a route with fewer transfers, and if the user has time, it will choose a route with better scenery.

[0334] 5. Route provision

[0335] 5.1 Generating Calculation Results

[0336] The server generates the calculation results in data format such as JSON, which includes the departure point, arrival point, route name, transfer method, and estimated arrival time.

[0337] 5.2 Sending a Response

[0338] The server sends the generated calculation results to the terminal.

[0339] 5.3 Receiving and Displaying Data

[0340] The terminal receives the calculation results sent from the server, parses the received data, and displays it in a format that is easy for the user to view.

[0341] 5.4 User Visibility

[0342] The user looks at the device screen to see the optimal route displayed, and then receives instructions on which public transport to use next, where to transfer, etc.

[0343] Example: From Shibuya Station to Shinjuku Station

[0344] 1. Obtaining location information

[0345] Device: Obtain current location (Shibuya Station) using GPS.

[0346] User: Enter "Shinjuku Station" in the app's search box.

[0347] 2. Emotional Recognition

[0348] Device: Captures your face and voice with a camera or microphone.

[0349] Server: Recognizes the user's emotions using the emotion engine. For example, it detects that the user is in a hurry.

[0350] 3. Traffic information inquiry

[0351] Device: Generate API request data including latitude and longitude information (Shibuya Station), destination "Shinjuku Station", and emotion data.

[0352] Terminal: Sends the generated request to the API.

[0353] 4. Calculating the optimal route

[0354] Server: Receives API requests and sends queries to the database.

[0355] Server: Obtain information about delays on the Yamanote Line.

[0356] Server: Because the user is in a hurry, calculate a route using the Keio Line with fewer transfers.

[0357] 5. Route provision

[0358] Server: Generate calculation results (Shibuya Station to Shinjuku Station, using the Keio Line) in JSON format.

[0359] Server: Sends the calculation results to the terminal.

[0360] Terminal: Parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take approximately 8 minutes."

[0361] User: Follow the instructions displayed and board the Keio Line.

[0362] In this way, the user can receive optimal route instructions that utilize real-time information and take into account their emotional state. The specific processing steps will be described later.

[0363] The processing flow will be explained below.

[0364] Step 1:

[0365] The device uses the GPS sensor to obtain the latitude and longitude of the current location, and passes this information to the application.

[0366] Step 2:

[0367] The user uses the application interface to enter a destination, for example, "Shinjuku Station."

[0368] Step 3:

[0369] The device uses a microphone or camera to capture the user's voice and facial images, and passes this emotional data to the application.

[0370] Step 4:

[0371] The request data is created based on the current location information acquired by the device, the destination entered by the user, and emotion data. This data includes latitude and longitude information, the name of the destination, and emotion data.

[0372] Step 5:

[0373] The request data created by the device is sent to the traffic information acquisition API. The data is sent securely using the HTTPS protocol.

[0374] Step 6:

[0375] The server receives the request sent from the terminal.

[0376] Step 7:

[0377] Based on the request received by the server, the server queries the transport company database to obtain information such as transport schedules, operation status, and delay information.

[0378] Step 8:

[0379] The server obtains real-time information about the operation status of public transport, such as delays on the Yamanote Line.

[0380] Step 9:

[0381] The server uses an emotion engine to analyze the user's emotional information, such as whether the user is in a hurry or relaxed, based on changes in voice tone and facial expressions.

[0382] Step 10:

[0383] The server calculates the optimal route from the current location to the destination based on traffic information, timetable information, operation status, and user emotion information, taking into account travel time, number of transfers, operation delays, and route adjustments according to the user's emotion.

[0384] Step 11:

[0385] The server generates the calculation results in data format such as JSON, which includes the departure point, arrival point, route name, transfer method, and estimated arrival time.

[0386] Step 12:

[0387] The server sends the generated calculation results to the terminal.

[0388] Step 13:

[0389] The terminal receives the calculation results sent from the server, parses the received data, and displays it in a format that is easy for the user to view.

[0390] Step 14:

[0391] The user looks at the device screen to see the optimal route displayed, and then receives instructions on which public transport to use next, where to transfer, etc.

[0392] Example 2

[0393] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0394] Conventional public transport guidance systems provide the optimal route based on the user's current location and destination, but they do not take into account the user's emotional state, making it difficult to respond flexibly to the user's situation. Furthermore, simply providing the shortest route does not necessarily provide the optimal method for the user, as it cannot respond to sudden delays or congestion.

[0395] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: location information acquisition means for acquiring a current location; input means for receiving input information related to a destination; information acquisition means for acquiring traffic information in real time based on the current location acquired by the location information acquisition means and the destination received by the input means; calculation means for calculating an optimal route based on the traffic information acquired by the information acquisition means; emotion recognition means for acquiring a user's voice or image data and recognizing emotions; calculation means for calculating an optimal route taking into account the user's emotion recognized by the emotion recognition means; and provision means for providing the user with the optimal route calculated by the calculation means. This enables a more flexible and adaptable optimal route to be proposed taking into account the user's emotional state. Furthermore, by acquiring and analyzing traffic information in real time, a route that can accommodate unexpected service delays and congestion can be provided.

[0396] "Location information acquisition means" refers to a device or function for acquiring the user's current location using a satellite positioning system sensor.

[0397] The "input means" is an interface through which the user inputs information about the destination.

[0398] The "information acquisition means" is a device or function for acquiring traffic information in real time based on the current location information acquired by the position information acquisition means and the destination information received by the input means.

[0399] The "calculation means" is a device or function for calculating an optimum route based on the traffic information acquired by the information acquisition means.

[0400] The "emotion recognition means" is a device or function for acquiring the user's voice and image data, analyzing them, and recognizing the user's emotions.

[0401] "Providing means" refers to a device or function for providing the calculated optimum route to the user.

[0402] "User's emotional state" is the user's mental or emotional state recognized by the emotion recognition means.

[0403] A "satellite positioning system sensor" is a sensor that acquires location information using a global positioning system (GPS) or similar.

[0404] A "data communication interface" is a device or function for obtaining external information using a communication protocol.

[0405] "Getting traffic information in real time" means getting current traffic conditions and operation information instantly.

[0406] The present invention is a system that suggests public transportation options in real time that will get a user from their current location to their destination in the shortest time, and also combines an emotion engine. This system includes a location information acquisition means, an input means, an information acquisition means, a calculation means, an emotion recognition means, and a provision means. Details of each means and the operation of the system are explained below.

[0407] Specific hardware and software

[0408] Location information acquisition means: A device that acquires the user's current location using a satellite positioning system (GPS) sensor.

[0409] Input method: The interface of the application through which the user enters their destination.

[0410] Information acquisition means: A device for acquiring traffic information through a data communication interface.

[0411] Calculation means: The calculation module in the server calculates the optimal route based on the data obtained from the information acquisition means.

[0412] Emotion recognition means: An emotion recognition module that acquires and analyzes the user's voice and image data.

[0413] Provision means: A terminal display and application for providing the user with the optimal route calculated by the calculation means.

[0414] System Operation

[0415] First, the device uses the GPS sensor to acquire the user's current location. The user inputs the destination through the application interface and the information is sent to the system. Then, the location information acquisition means identifies the user's current location, and the input means receives the destination information.

[0416] Next, the device captures the user's voice and facial images using a microphone and camera, and passes them to the emotion recognition means. The emotion recognition means runs on the server and analyzes the received voice and image data to recognize the user's emotions. As a result of the analysis, the user's emotional state, such as impatience, anxiety, or calmness, is identified.

[0417] Based on these data, the terminal compiles the current location information, destination information, and the user's emotional state into one request data, and acquires real-time traffic information through the information acquisition means. This data acquisition is performed via the data communication interface.

[0418] The server receives the request data sent from the information acquisition means and queries an internal database to obtain information on the operation status of public transport, delay information, transfer information, etc. It also obtains the latest public transport information using an external Web API. Based on this data, the calculation means calculates the optimal route.

[0419] In the calculation, the server takes into account the user's emotional state. For example, if the user is in a hurry, it will prioritize routes with fewer transfers. On the other hand, if the user has time, it will suggest scenic or comfortable routes.

[0420] The calculation results are sent from the server to the terminal and provided to the user. The application on the terminal displays the calculation results to the user through the means of provision, and gives specific instructions such as "If you take the Keio Line to Shinjuku Station, the journey will take about 8 minutes."

[0421] Specific examples

[0422] For example, consider a case where a user wants to travel from Shibuya Station to Shinjuku Station. First, the device's GPS acquires the current location, Shibuya Station, and the user enters "Shinjuku Station" into the application. Next, the device uses the camera and microphone to capture the user's facial expressions and voice, and the server recognizes the emotion of "I'm in a hurry."

[0423] The device then generates a request containing the current location (Shibuya Station), the destination (Shinjuku Station), and the user's emotion data, and sends it to the traffic information API. The server receives the request and queries the database and external APIs to collect traffic information. If the Yamanote Line is delayed, it is perceived that the user is in a hurry, and it is determined that the Keio Line, which has fewer transfers, is the optimal route.

[0424] Finally, the server generates the calculation result in JSON format and sends it to the terminal. The terminal parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take about 8 minutes." The user follows this and boards the Keio Line to Shinjuku Station.

[0425] Prompt Sentence Examples

[0426] Examples of prompts for generating emotion recognition and route calculation instructions using generative AI models are as follows:

[0427] "Please explain how a system that suggests the public transport option that will get a user from their current location to their destination in real time works by combining it with an emotion engine. Specifically, please explain how the current location is obtained using GPS, the user inputs their destination, emotion recognition is performed, the procedure for collecting traffic information, how the optimal route is calculated, and the process for providing the route to the user."

[0428] The above is a specific embodiment of the present invention. This system provides users with real-time recommendations for optimal public transport routes, and even tailored to their individual emotional state, greatly improving convenience.

[0429] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0430] Step 1: Get your current location

[0431] The device uses the GPS sensor to obtain the latitude and longitude of the user's current location, which is then sent to the application backend.

[0432] Input: Location data from the GPS sensor

[0433] Data processing: Convert location data into application format

[0434] Output: Latitude and longitude information (e.g., latitude: 35.6581, longitude: 139.7414)

[0435] Step 2: Enter your destination

[0436] The user enters a destination into the application's search box, and the device sends the entered data to the application's backend.

[0437] Input: Destination information entered by the user (text format)

[0438] Data processing: Save the entered text as destination information

[0439] Output: Destination information (e.g., destination: "Shinjuku Station")

[0440] Step 3: Obtaining emotion data

[0441] The device uses a microphone and camera to capture the user's voice and facial images, and the captured data is passed to the emotion recognition means.

[0442] Input: Audio and image data obtained from microphones and cameras

[0443] Data processing: Converting audio and image data into a format that can be analyzed by emotion recognition algorithms

[0444] Output: Audio data and image data (e.g., image_data: [binary data], audio_data: [binary data])

[0445] Step 4: Analyze the sentiment data

[0446] The server analyzes the acquired voice and image data to recognize the user's emotions. This analysis is performed by the emotion recognition module.

[0447] Input: Audio and image data

[0448] Data processing: Analyzing voice tone and facial expression changes

[0449] Output: User's emotional state (e.g. emotion: "anxious")

[0450] Step 5: Check traffic information

[0451] The terminal creates request data based on the current location, destination, and emotion data, and acquires traffic information in real time through the information acquisition means.

[0452] Input: current location information, destination information, emotional state

[0453] Data processing: Assemble these data into a single request object

[0454] Output: Request data (e.g., { currentLocation: {latitude: 35.6581, longitude: 139.7414}, destination: "Shinjuku Station", emotion: "Impatience"})

[0455] Step 6: Obtaining real-time traffic information

[0456] The terminal sends the request data to the traffic information acquisition API to obtain real-time traffic information.

[0457] Input: Request data

[0458] Data processing: Communication with external traffic information databases via API

[0459] Output: Real-time traffic information (e.g. { line: "Yamanote Line", status: "Delay"})

[0460] Step 7: Calculate the optimal route

[0461] The optimal route is calculated based on real-time traffic information obtained by the server and the user's emotional state.

[0462] Input: Real-time traffic information, emotional state, current location information, destination information

[0463] Data processing: This information is input into an algorithm to calculate the optimal route.

[0464] Output: Optimal route data (e.g., Optimal route calculated: Shibuya Station → Keio Line → Shinjuku Station)

[0465] Step 8: Generate route calculation results

[0466] The server generates the optimal route calculation results in JSON format and sends them to the terminal.

[0467] Input: Optimal route data

[0468] Data processing: Convert calculation results into JSON format

[0469] Output: Calculation result in JSON format (e.g., {"origin": "Shibuya Station", "destination": "Shinjuku Station", "route": "Keio Line", "estimated_time": "8 minutes"})

[0470] Step 9: Receive and display the calculation results

[0471] The terminal receives the calculation results sent from the server, analyzes them, and provides them to the user.

[0472] Input: Calculation result in JSON format

[0473] Data processing: Parse JSON data and format it for display

[0474] Output: Display data (e.g. "The journey to Shinjuku Station on the Keio Line takes approximately 8 minutes.")

[0475] Step 10: Deliver to users

[0476] The device displays the formatted calculation results to the user and gives instructions on what to do next.

[0477] Input: Display data

[0478] Data processing: Display on screen

[0479] Output: Displayed instructions (e.g., the user looks at the device and performs the action "take the Keio Line")

[0480] (Application example 2)

[0481] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0482] Conventional navigation systems provide an optimal route based on a user's current location and destination, but do not consider the user's emotional state. As a result, they are unable to provide an appropriate route based on the user's emotional state, such as when the user is in a hurry or relaxed. For example, a user may want to be presented with the shortest route when in a hurry, or with a scenic route when relaxed. The present invention aims to realize a more comfortable and stress-free journey by providing route suggestions based on the user's emotional state.

[0483] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0484] In this invention, the server includes emotion recognition means for analyzing the emotional state of the user, route optimization means for adjusting an optimal route taking into consideration the emotional state of the user acquired by the emotion recognition means, and information presentation means for presenting the route information provided by the route provision means through an in-vehicle display device and voice guidance system, thereby making it possible to propose an optimal route based on the emotional state of the user.

[0485] "Location information acquisition means" is a device for acquiring latitude and longitude information of the current location. It mainly has the function of identifying the current location using a global positioning system sensor, etc.

[0486] An "input means" is a device through which a user inputs destinations and other information. This typically includes a touchscreen or voice input device.

[0487] "Traffic information acquisition means" refers to a device for acquiring traffic conditions and operation information in real time. This includes methods of acquiring information via web APIs.

[0488] The "route calculation means" is a device that calculates the optimal route based on the current location, destination, and traffic information. It has the function of using an algorithm to calculate the optimal route based on conditions such as the shortest time and the fewest transfers.

[0489] "Route providing means" refers to a device that provides optimal route information to users. It has the function of displaying or notifying route information through a display, voice guidance system, etc.

[0490] The "emotion recognition means" is a device for analyzing the user's emotional state. It has the function of analyzing the user's emotions based on data acquired using a camera or microphone.

[0491] The "route optimization means" is a device that adjusts the optimal route taking into consideration the emotional state of the user acquired by the emotion recognition means. For example, it has the function of suggesting the shortest route if the user is in a hurry, or the scenic route if the user is relaxed, depending on the user's emotions.

[0492] The "information presentation means" is a device for presenting the route information provided by the route provision means through a display device and a voice guidance system in the vehicle, and has the function of conveying the route information to the user visually and audibly.

[0493] The present invention is a navigation system for autonomous vehicles that proposes the optimal route from a user's current location to their destination, and is characterized by its consideration of the user's emotional state. This system optimizes the route based on the user's emotional state to ensure a comfortable journey for the user.

[0494] System Configuration

[0495] A system for implementing the present invention includes the following major hardware and software components:

[0496] Location information acquisition method: Use a global positioning system sensor or similar to acquire latitude and longitude information of the current location.

[0497] Input method: The user inputs destination information using a touchscreen or voice input device.

[0498] Traffic information acquisition method: Use a web API (e.g., Google Maps API) to acquire real-time traffic information.

[0499] Route calculation method: An algorithm is used to calculate the optimal route based on the acquired traffic information.

[0500] Route provision means: Calculated route information is provided to the user through a display or voice guidance system.

[0501] Emotion recognition means: The user's emotional state is analyzed using a camera and microphone. The analysis is performed using an emotion recognition engine such as Microsoft® Azure® Cognitive Services.

[0502] Route optimization means: The emotion data obtained by emotion recognition is taken into account to adjust the optimal route.

[0503] Information presentation means: Presents the route information provided by the route provision means through a display device and a voice guidance system in the vehicle.

[0504] Program processing and explanation

[0505] The whole system works as follows:

[0506] 1. Acquisition and analysis of emotion data

[0507] Cameras and microphones installed in the vehicle capture the user's facial images and voice. This data is passed to an emotion recognition engine for analysis. The analysis results identify the user's emotional state, such as whether they are in a hurry or relaxed.

[0508] 2. Data collection for route calculation

[0509] The server obtains the current location information and the destination information set by the user, and generates an API request that includes the emotional state along with the obtained current location and destination information.

[0510] 3. Obtaining traffic information and calculating the optimal route

[0511] The server retrieves real-time traffic information from a traffic information database via API requests. Traffic information includes operation status and delay information. The optimal route is calculated based on the retrieved traffic information and the user's emotional state.

[0512] 4. Providing route information

[0513] The calculated optimal route is sent from the server to the terminal inside the vehicle. The terminal analyzes the route information and displays it on the vehicle's display device. The user is then guided through the voice guidance system.

[0514] Specific examples

[0515] Below is a specific example where a user wants to go from Shibuya Station to Shinjuku Station. At this time, the user is a little impatient.

[0516] Acquiring and analyzing emotion data

[0517] A camera inside the vehicle captures the user's face, and an emotion recognition engine determines that the user is anxious.

[0518] Data collection for route calculation

[0519] The server obtains the location information of the current location (Shibuya Station) and the destination information (Shinjuku Station) entered by the user.

[0520] Obtaining traffic information and calculating the optimal route

[0521] Traffic information is obtained through a web API and the shortest route using the Keio Line is calculated.

[0522] Providing route information

[0523] The calculated route to Shinjuku Station using the Keio Line is sent to the terminal, which then displays this information on its display and announces via a voice guidance system that "If you take the Keio Line to Shinjuku Station, the journey will take approximately eight minutes."

[0524] Prompt Sentence Examples

[0525] An example of a prompt sentence is, "I'm going to Shinjuku Station. I'm currently at Shibuya Station and I'm a little anxious. Please tell me the best route."

[0526] In this way, a system can be realized that suggests optimal routes based on real-time traffic conditions while taking into account the user's emotional state.

[0527] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0528] Step 1:

[0529] The device uses cameras and microphones installed in the vehicle to capture the user's facial images and voice, which then generates data reflecting the user's emotional state. The acquired data is then sent to emotion recognition means.

[0530] Input: face image, voice data

[0531] Output: Raw data passed to the emotion recognition engine

[0532] Step 2:

[0533] The server uses an emotion recognition engine (e.g., Microsoft Azure Cognitive Services) to analyze the acquired facial images and voice data. As a result of the analysis, the user's emotional state (e.g., anxious, relaxed, etc.) is identified.

[0534] Input: face image, voice data

[0535] Output: User's emotional state (e.g., anxious, relaxed)

[0536] Step 3:

[0537] The user inputs the destination into the device. In this process, the destination information, such as "Shinjuku Station," is entered using a touch screen or voice input device. The device then sends the entered destination information along with the current location information to the server.

[0538] Input: Destination information (e.g. Shinjuku Station)

[0539] Output: Destination information is sent to the server

[0540] Step 4:

[0541] The device uses the GPS sensor to obtain the user's current location. The latitude and longitude information of the current location is acquired and sent to the server.

[0542] Input: GPS data

[0543] Output: Current location information (latitude and longitude) is sent to the server

[0544] Step 5:

[0545] The server generates API request data including the current location information, destination information, and emotional state, and sends the request to the web API for traffic information acquisition. The request includes a query to find out the current traffic situation.

[0546] Input: current location information, destination information, emotional state

[0547] Output: API request data sent

[0548] Step 6:

[0549] The server receives real-time traffic information from a web API for traffic information acquisition. The received data includes real-time traffic information such as operation status and delay information.

[0550] Input: API request data

[0551] Output: Real-time traffic information

[0552] Step 7:

[0553] The server calculates the optimal route based on traffic information and the user's emotional state. If the user is in a hurry, the shortest route will be selected, and if the user is relaxed, the route with the best scenery will be selected. The calculated route information is generated in JSON format.

[0554] Input: Traffic information, emotional state

[0555] Output: Optimal route information (JSON format)

[0556] Step 8:

[0557] The server sends the calculated optimal route information to the terminal.

[0558] Input: Optimal route information (JSON format)

[0559] Output: Route information is sent to the device

[0560] Step 9:

[0561] The device analyzes the received route information and displays it in a user-friendly format (e.g., displaying the route on the vehicle's display). It also provides route instructions to the user through a voice guidance system.

[0562] Input: Optimal route information (JSON format)

[0563] Output: The route is displayed on the display and voice guidance is provided.

[0564] In this way, an optimal route is provided based on real-time traffic information while taking into account the user's emotional state.

[0565] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0566] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0567] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0568] [Second embodiment]

[0569] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0570] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0571] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0572] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0573] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0574] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0575] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0576] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0577] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0578] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0579] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0580] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0581] The present invention relates to a system that suggests public transportation options in real time that will allow a user to arrive at their destination from their current location in the shortest time. This system includes a location information acquisition means, an input means, a traffic information acquisition means, a route calculation means, and a route provision means. Details of each means and the operation of the system are explained below.

[0582] 1. Obtaining location information

[0583] 1.1 Getting your current location

[0584] Device: Uses the GPS sensor to obtain current location. The device's GPS chip receives signals from satellites and obtains latitude and longitude information. This information is passed to the application.

[0585] 1.2 Entering your destination

[0586] User: Enters a destination through the application interface, for example, "Shinjuku Station."

[0587] Terminal: Passes the destination entered by the user to the application.

[0588] 2. Traffic information inquiry

[0589] 2.1 Making an API request

[0590] Device: Creates data for requesting the real-time traffic information API based on the current location and destination information, including latitude and longitude information and text information for the destination.

[0591] 2.2 Submitting a Request

[0592] Terminal: Sends the created request to the traffic information API. Sends data securely using the HTTPS protocol.

[0593] 3. Calculating the optimal route

[0594] 3.1 Receiving a Request

[0595] Server: Receives requests from the terminal via the traffic information API.

[0596] 3.2 Querying the Database

[0597] Server: Based on the received request, queries the transport company's database, which contains information on timetables, operation status, delays, etc.

[0598] 3.3 Obtaining real-time information

[0599] Server: Obtain real-time information on transportation status, such as delays on the Yamanote Line.

[0600] 3.4 Optimal Route Calculation

[0601] Server: Based on the acquired information, the server calculates the optimal route from the current location to the destination. The calculation takes into account the travel time, number of transfers, and whether or not there are any delays. Specifically, route search algorithms such as the Dijkstra algorithm and the A algorithm are used.

[0602] 4. Route provision

[0603] 4.1 Generating calculation results

[0604] Server: Generates the results of the calculation in a data format such as JSON, which contains the following information:

[0605] Departure and arrival points

[0606] Route name to be used

[0607] How to transfer

[0608] Estimated arrival time

[0609] 4.2 Sending a Response

[0610] Server: Sends the calculation results to the terminal.

[0611] 4.3 Receiving and Displaying Data

[0612] Terminal: Parses the calculation results received from the server and displays them in a user-friendly format. Route guidance, timetables, transfer instructions, etc. are displayed on the screen.

[0613] 4.4 User Visibility

[0614] User: Looks at the device screen to see the optimal route displayed, and receives instructions on which train or bus to take next, where to transfer, etc.

[0615] Example: From Shibuya Station to Shinjuku Station

[0616] 1. Obtaining location information

[0617] Device: Obtain current location (Shibuya Station) using GPS.

[0618] User: Enter "Shinjuku Station" in the app's search box.

[0619] 2. Traffic information inquiry

[0620] Terminal: Generate API request data including latitude and longitude information (Shibuya Station) and destination "Shinjuku Station".

[0621] Terminal: Sends the generated request to the API.

[0622] 3. Calculating the optimal route

[0623] Server: Receives API requests and sends queries to the database.

[0624] Server: Obtain information about delays on the Yamanote Line.

[0625] Server: Calculates a route that uses the Keio Line instead of the Yamanote Line.

[0626] 4. Route provision

[0627] Server: Generate calculation results (Shibuya Station to Shinjuku Station, using the Keio Line) in JSON format.

[0628] Server: Sends the calculation results to the terminal.

[0629] Terminal: Parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take approximately 8 minutes."

[0630] User: Follow the instructions displayed and board the Keio Line.

[0631] In this way, the user can receive optimal route instructions utilizing real-time information. The specific processing steps will be explained separately.

[0632] The processing flow will be explained below.

[0633] Step 1:

[0634] The device uses the GPS sensor to obtain the latitude and longitude of the current location, and passes this information to the application.

[0635] Step 2:

[0636] The user uses the application interface to enter a destination, for example, "Shinjuku Station."

[0637] Step 3:

[0638] The device creates request data based on the current location information acquired and the destination entered by the user. This data includes the latitude and longitude of the current location and the name of the destination.

[0639] Step 4:

[0640] The request data created by the device is sent to the traffic information acquisition API. The data is sent securely using protocols such as HTTPS.

[0641] Step 5:

[0642] The server receives the request sent from the terminal.

[0643] Step 6:

[0644] Based on the request received by the server, the server queries the transport company database to obtain information such as transport schedules, operation status, and delay information.

[0645] Step 7:

[0646] The server obtains real-time information about the operation status of public transport, such as delays on the Yamanote Line.

[0647] Step 8:

[0648] Based on the information acquired by the server, the optimal route from the current location to the destination is calculated, taking into account the required time, number of transfers, and information on train delays.

[0649] Step 9:

[0650] The server generates the calculation results in data format such as JSON, which includes the departure point, arrival point, route name, transfer method, estimated arrival time, etc.

[0651] Step 10:

[0652] The server sends the generated calculation results to the terminal.

[0653] Step 11:

[0654] The terminal receives the calculation results sent from the server, parses the received data, and displays it in a format that is easy for the user to view.

[0655] Step 12:

[0656] The user looks at the device screen to see the optimal route displayed, and then receives instructions on which public transport to use next, where to transfer, etc.

[0657] Example 1

[0658] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0659] Currently, when users reach their destination using public transportation, it is difficult to determine the optimal route that takes into account real-time traffic information and service delays. Furthermore, when transferring between multiple modes of transportation, there are few systems that provide integrated transfer information, which often makes users' travel complicated. Therefore, there is a need for a system that instantly provides the optimal route based on real-time information.

[0660] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0661] In this invention, the server includes a location information acquisition means for acquiring the current location, an input means for receiving input information about the destination, and a means for acquiring traffic information in real time, thereby making it possible to provide an optimal route taking into account real-time traffic conditions.

[0662] "Location information acquisition means" refers to devices or functions for acquiring the user's current location.

[0663] "Input means" refers to an interface or device for receiving destination-related information from a user.

[0664] "Traffic information acquisition means" refers to functions and devices for acquiring traffic information in real time.

[0665] "Optimal route calculation means" refers to a function or device for calculating the optimal travel route based on acquired traffic information and transfer information.

[0666] "Route providing means" refers to a means for providing information on the calculated optimal route to a user.

[0667] "Satellite Positioning System Sensor" means a positioning system that uses signals from satellites to determine location on Earth.

[0668] "Web Application Programming Interface" refers to a means for communicating with external software or applications through web services.

[0669] "Transportation information" refers to information such as public transportation schedules, operation status, and delay information.

[0670] The present invention relates to a system that suggests public transportation options in real time that will allow a user to arrive at their destination from their current location in the shortest time. This system is composed of the following means, including location information acquisition means, input means, traffic information acquisition means, optimal route calculation means, and route provision means.

[0671] Obtaining location information

[0672] Get current location

[0673] Device: The GPS sensor is used to obtain the user's current location. The GPS chip in the device receives signals from satellites, extracts latitude and longitude information, and passes it to the application.

[0674] Enter your destination

[0675] User: Enters a destination through the application interface. For example, the user enters "Shinjuku Station."

[0676] Terminal: Passes the destination information entered by the user to the application.

[0677] Traffic information inquiry

[0678] Making API Requests

[0679] Device: Creates request data for the real-time traffic information API based on the current location and destination information. This data includes the acquired latitude and longitude information and the destination information entered by the user.

[0680] Submitting a Request

[0681] Terminal: Sends the created request data to the API using the HTTPS protocol.

[0682] Calculating the best route

[0683] Receiving a request

[0684] Server: Queries the transit agency database based on requests received via the API.

[0685] Server: The database contains information on transportation schedules, operation status, delays, etc.

[0686] Obtaining real-time information

[0687] Server: Obtains real-time information about the operation of public transport. For example, obtains information such as "The Yamanote Line is currently delayed by 5 minutes."

[0688] Calculating the best route

[0689] Server: Based on the acquired data, the server calculates the optimal route from the user's current location to the destination. The calculation takes into account the travel time, number of transfers, and real-time delay information. The specific algorithms used for route search are the Dijkstra algorithm and the A algorithm.

[0690] Route provision

[0691] Generate calculation results

[0692] Server: Generates the optimal route calculation results in data format such as JSON. This data includes information such as the departure point, arrival point, route name, transfer method, and estimated arrival time.

[0693] Sending a response

[0694] Server: Sends the generated calculation results to the terminal.

[0695] Receiving and displaying data

[0696] Terminal: Parses the calculation results received from the server and displays them in a user-friendly format. For example, route guidance, timetables, transfer procedures, etc. are displayed on the screen.

[0697] User Visibility

[0698] User: Check the information displayed on the device and follow the instructions to begin traveling. For example, follow the instructions such as "Take the Keio Line to Shinjuku Station, it will take about 8 minutes."

[0699] Example: From Shibuya Station to Shinjuku Station

[0700] Get current location:

[0701] Device: Obtain current location (Shibuya Station) using GPS.

[0702] Enter your destination:

[0703] User: Enters "Shinjuku Station" into the app's search box.

[0704] Terminal: Obtain the entered destination information.

[0705] Make an API request:

[0706] Terminal: Generate API request data including latitude and longitude information (Shibuya Station) and destination "Shinjuku Station".

[0707] Sending an API request:

[0708] Terminal: Sends the generated request data to the API.

[0709] Query the database:

[0710] Server: Receives API requests and sends queries to the transit agency database.

[0711] Get real-time information:

[0712] Server: Obtain delay information for the Yamanote Line.

[0713] Calculating the best route:

[0714] Server: Calculate a route that avoids delays on the Yamanote Line and uses the Keio Line.

[0715] Generate calculation results:

[0716] Server: Generate the calculation result (Shibuya Station to Shinjuku Station, using the Keio Line) in JSON format.

[0717] Sending a response:

[0718] Server: Sends the calculation results to the terminal.

[0719] Receiving and displaying data:

[0720] Terminal: Parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take approximately 8 minutes."

[0721] Check the results and follow the instructions:

[0722] User: Follow the displayed instructions and use the Keio Line.

[0723] Example prompt sentence:

[0724] "I'm currently at Shibuya Station and my destination is Shinjuku Station. Please tell me the best route taking real-time traffic information into account."

[0725] This system allows users to receive optimal route instructions based on real-time information, enabling them to use public transportation efficiently.

[0726] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0727] Step 1: Obtaining location information

[0728] Device: The device uses a GPS sensor to obtain the user's current location. Specifically, the device's GPS chip receives signals from satellites and generates latitude and longitude information. This location information is passed to the application. The input is the signal from the GPS, and the output is the user's latitude and longitude information.

[0729] Step 2: Enter your destination

[0730] User: The user inputs a destination through the application interface. For example, they input "Shinjuku Station." The input is text input by the user, and the output is the destination information "Shinjuku Station."

[0731] Terminal: The terminal passes the destination information entered by the user to the application. The input is text data from the user, and the output is the destination information passed to the application.

[0732] Step 3: Making an API Request

[0733] Terminal: The terminal creates request data for the real-time traffic information API based on the current location (latitude and longitude information) and destination information. Specifically, this information is combined and placed in a request format. The input is latitude and longitude information and destination information, and the output is the API request data.

[0734] Step 4: Sending an API Request

[0735] Terminal: The terminal uses the HTTPS protocol to send the created request data to the API. The input is the API request data, and the output is communication to the API server.

[0736] Step 5: Receiving the request

[0737] Server: The API server receives the request data sent from the terminal. The input is the request data, and the output is a confirmation of the received request data.

[0738] Step 6: Query the database

[0739] Server: The server queries the traffic information database based on the received request. Specifically, it searches for schedule information, operation status, and delay information. The input is the request data, and the output is the traffic information retrieved from the database.

[0740] Step 7: Get real-time information

[0741] Server: The server obtains real-time operational status of transportation facilities. For example, it obtains information such as "The Yamanote Line is currently delayed by 5 minutes." The input is real-time data of transportation facilities, and the output is the latest operational status.

[0742] Step 8: Calculate the optimal route

[0743] Server: The server calculates the optimal route from the current location to the destination based on the acquired data. It uses the Dijkstra algorithm and the A algorithm to take into account the travel time, number of transfers, and real-time delay information. The input is traffic information and real-time operation status, and the output is the optimal route information.

[0744] Step 9: Generate calculation results

[0745] Server: The server generates the optimal route calculation results in data format such as JSON. This data includes the departure point, arrival point, route name to be used, transfer method, and estimated arrival time. The input is the optimal route information, and the output is data in JSON format.

[0746] Step 10: Sending the Response

[0747] Server: The server sends the generated calculation results to the terminal. The input is JSON format data, and the output is the response to the terminal.

[0748] Step 11: Receiving and displaying data

[0749] Terminal: The terminal parses the calculation results received from the server and displays them in a user-friendly format. Route guidance, timetables, and transfer procedures are displayed on the screen. The input is the response data from the server, and the output is the information displayed on the user interface.

[0750] Step 12: Review the results and follow the instructions

[0751] User: The user checks the information displayed on the device and starts moving according to the instructions. For example, they follow instructions such as "Take the Keio Line to Shinjuku Station, it will take about 8 minutes." The input is the information displayed on the device, and the output is the user's actions.

[0752] (Application example 1)

[0753] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0754] While many conventional public transportation guidance systems provide the optimal route from a user's current location to their destination, they are inadequate when it comes to providing destination guidance within shopping malls and large facilities. Providing adequate guidance is particularly difficult when a user has multiple destinations or when congestion needs to be taken into consideration. There is also a growing need for intuitive operation using voice input in addition to visual guidance information. The present invention aims to solve these problems by providing a system that provides real-time guidance on the shortest route to a destination within a shopping mall.

[0755] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0756] In this invention, the server includes location information acquisition means for acquiring location information, input means for receiving input information related to a destination, information acquisition means for acquiring traffic information in real time, route calculation means for calculating an optimal route, route provision means for providing the optimal route to a user, display means for visually guiding the optimal route based on real-time congestion status, and voice input means for inputting a destination using voice recognition, thereby making it possible to provide the shortest route to a destination within a shopping mall in real time by visual and voice input.

[0757] "Location information acquisition means" refers to a device or system for acquiring the user's current location, and utilizes a satellite system or GPS sensor.

[0758] "Input means" refers to a device or system for receiving information about a destination from a user, and includes a touch panel, a keyboard, a voice recognition system, and the like.

[0759] The "information acquisition means" is a device or system for acquiring data such as traffic information and congestion status in real time, and acquires data via a network.

[0760] The "route calculation means" is a device or system for calculating the optimum route based on the acquired information, and uses a route search algorithm that takes into account the required travel time and traffic congestion.

[0761] "Route providing means" refers to a device or system for providing the user with the calculated optimum route, and conveys information through a screen display, voice guidance, etc.

[0762] "Display means" refers to a device or system that visually guides passengers along optimal routes based on real-time congestion information, and uses smart glasses or a head-mounted display.

[0763] "Voice input means" refers to a device or system that utilizes voice recognition technology to receive destination information from a user.

[0764] The embodiment of the present invention is directed to constructing a system that guides users so that they can reach their destination from their current location in the shortest time possible. This system is particularly effective for providing guidance within large facilities such as shopping malls. Detailed embodiments of this system are described below.

[0765] 1. System Configuration

[0766] The system consists of the following major components:

[0767] Location information acquisition means

[0768] Input Method

[0769] Information acquisition means

[0770] Route calculation method

[0771] Route provision method

[0772] Display means

[0773] Voice input means

[0774] Location information acquisition means

[0775] Location information acquisition means has the function of acquiring the user's current location. It mainly uses satellite systems and GPS sensors to acquire the current latitude and longitude information. For example, a GPS chip installed in smart glasses measures the user's location in real time.

[0776] Input Method

[0777] The input means has the function of receiving information about the destination from the user. This can be done using a touch panel, keyboard, or voice recognition system. Specifically, it is possible to use the voice recognition system built into the smart glasses to input destinations such as "supermarket" or "toilet" by voice.

[0778] Information acquisition means

[0779] The information acquisition means has the function of acquiring data such as traffic information and congestion status in real time. This data is acquired via the network, for example, using the Google Maps API or a shopping mall's dedicated API.

[0780] Route calculation method

[0781] The route calculation means has the function of calculating the optimal route based on the acquired information. For this, a route search algorithm (such as the Dijkstra algorithm or the A algorithm) is used that takes into account the required time and congestion status.

[0782] Route provision method

[0783] The route providing means has the function of providing the user with the calculated optimal route, and displays visual guidance information on the smart glasses display or the smartphone screen.

[0784] Display means

[0785] The display function visually guides users to the optimal route based on real-time congestion information, primarily using smart glasses or head-mounted displays to visually show the user the next direction to go and the route to their destination.

[0786] Voice input means

[0787] The voice input means has the function of receiving destination information from the user using voice recognition technology, using a microphone and voice recognition software built into the smart glasses.

[0788] Program processing explanation

[0789] This system works in conjunction with each mode of transportation to provide users with optimal route guidance. The specific processing flow is explained below.

[0790] 1. Location information acquisition means

[0791] It uses a GPS sensor to obtain the user's current location and transmits that information to a server. For example, the GPS chip in smart glasses measures the user's location in real time and passes the latitude and longitude data to the application.

[0792] 2. Input Method

[0793] The user inputs their destination through a voice input system, for example by saying "I'm going to the supermarket," and the voice recognition software recognizes the destination and sends that information to the server.

[0794] 3. Information acquisition means

[0795] The server makes a request to a real-time congestion information API based on the acquired current location and destination information. Specifically, it accesses the Google Maps API or a congestion information API dedicated to shopping malls.

[0796] 4. Route calculation method

[0797] The server calculates the shortest route using the Dijkstra algorithm or A algorithm based on the acquired congestion information. For example, it calculates a route that avoids congested areas.

[0798] 5. Route provision method

[0799] The server sends the calculation results in JSON format to the smart glasses, which then display arrows and text instructions to the user based on the route information received.

[0800] Examples of concrete examples and prompts

[0801] Hardware / Software used

[0802] Reference Hardware: Smart glasses, e.g. Vuzix Blade or North Focals

[0803] Real-time location information acquisition API: Google Maps API

[0804] Server: AWS EC2

[0805] Real-time congestion information API: SmartMall API (tentative)

[0806] Prompt Sentence Examples

[0807] Generative AI model used: GPT-4

[0808] Example prompt: "Code the process flow for a smart glasses app that provides visual guidance on the shortest route from a user's current location to a destination. The destination is specified via voice input, and the route display takes real-time information into account."

[0809] This makes it possible to provide the shortest route to a destination within a shopping mall in real time through visual and voice input.

[0810] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0811] Step 1:

[0812] Get current location

[0813] Subject: Device

[0814] What it does: Uses the device's GPS sensor to obtain the user's current location.

[0815] Input: The device's GPS chip receives signals from satellites and obtains latitude and longitude information.

[0816] Output: Latitude and longitude information (e.g., current location coordinates). The device passes this information to the next processing step.

[0817] Step 2:

[0818] Enter your destination

[0819] Subject: User

[0820] How it works: The user enters destination information through the voice recognition system.

[0821] Input: The user's spoken command (e.g., "Go to the supermarket").

[0822] Output: Textual destination information (e.g., destination name) is generated and passed to the terminal.

[0823] Step 3:

[0824] Obtaining real-time information

[0825] Subject: Device

[0826] How it works: The device sends a request to a real-time congestion information API based on its current location and destination.

[0827] Input: Latitude and longitude information of current location, text information of destination.

[0828] Output: Real-time congestion data (e.g., congestion and traffic information obtained from an API) is returned to the device.

[0829] Step 4:

[0830] Calculating the best route

[0831] Subject: Server

[0832] How it works: The server calculates the optimal route based on the real-time data it receives.

[0833] Input: current location latitude and longitude, destination text, real-time congestion data.

[0834] Data processing / calculation: Using a path-finding algorithm (e.g., Dijkstra algorithm or A algorithm), calculate the route that can be reached in the shortest time.

[0835] Output: The calculated optimal route (e.g., a list of route information) is generated.

[0836] Step 5:

[0837] Route information format conversion

[0838] Subject: Server

[0839] How it works: The server converts the optimal route information into a data format such as JSON.

[0840] Input: The calculated optimal route information.

[0841] Output: Route information (e.g., navigation data) in JSON format is generated and sent to the device.

[0842] Step 6:

[0843] View route information

[0844] Subject: Device

[0845] How it works: The device parses the route information it receives and displays it visually to the user.

[0846] Input: Route information in JSON format.

[0847] Data processing / computation: The device parses the JSON data and generates a visual navigation display.

[0848] Output: Route guidance is displayed on the smart glasses display (e.g., arrows and text instructions showing the direction of travel).

[0849] Step 7:

[0850] User movement guidance

[0851] Subject: User

[0852] How it works: The user follows the guidance displayed on the smart glasses to reach their destination.

[0853] Input: Visual navigation information (e.g., arrows and text instructions).

[0854] Output: The user arrives at the destination.

[0855] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0856] The present invention combines an emotion engine with a system that suggests in real time the public transportation that will take a user from their current location to their destination in the shortest time. This system includes a location information acquisition means, an input means, a traffic information acquisition means, a route calculation means, a route provision means, and an emotion recognition means. Details of each means and the operation of the system are described below.

[0857] 1. Obtaining location information

[0858] 1.1 Getting your current location

[0859] The device uses the GPS sensor to obtain the latitude and longitude of the current location, and passes this information to the application.

[0860] 1.2 Entering your destination

[0861] The user uses the application interface to enter a destination, for example, "Shinjuku Station."

[0862] The device passes the destination entered by the user to the application.

[0863] 2. Emotional Recognition

[0864] 2.1 Acquiring Emotion Data

[0865] The device uses a microphone and camera to capture the user's voice and facial images.

[0866] The terminal passes the voice and image data to the emotion recognition means.

[0867] 2.2 Emotion Data Analysis

[0868] The server analyzes the received audio or image data to recognize the user's emotions. For example, it can identify whether the user is anxious, impatient, calm, etc. based on changes in voice tone or facial expression.

[0869] 3. Traffic information inquiry

[0870] 3.1 Making an API request

[0871] Request data is created based on the current location information acquired by the device, the destination entered by the user, and the recognized emotion data.

[0872] 3.2 Sending a Request

[0873] The request data created by the device is sent to the traffic information acquisition API. The data is sent securely using protocols such as HTTPS.

[0874] 4. Calculating the optimal route

[0875] 4.1 Receiving a Request

[0876] The server receives the request sent from the terminal.

[0877] 4.2 Querying the Database

[0878] Based on the request received by the server, the server queries the transport company database to obtain information such as transport schedules, operation status, and delay information.

[0879] 4.3 Obtaining real-time information

[0880] The server obtains real-time information about the operation status of public transport, such as delays on the Yamanote Line.

[0881] 4.4 Optimal Route Calculation

[0882] The server calculates the optimal route from the current location to the destination based on the information acquired and the user's emotional state. The calculation takes into account the required travel time, number of transfers, and service delay information, as well as the user's emotional state. Specifically, if the user is in a hurry, the system will choose a route with fewer transfers, and if the user has time, it will choose a route with better scenery.

[0883] 5. Route provision

[0884] 5.1 Generating Calculation Results

[0885] The server generates the calculation results in data format such as JSON, which includes the departure point, arrival point, route name, transfer method, and estimated arrival time.

[0886] 5.2 Sending a Response

[0887] The server sends the generated calculation results to the terminal.

[0888] 5.3 Receiving and Displaying Data

[0889] The terminal receives the calculation results sent from the server, parses the received data, and displays it in a format that is easy for the user to view.

[0890] 5.4 User Visibility

[0891] The user looks at the device screen to see the optimal route displayed, and then receives instructions on which public transport to use next, where to transfer, etc.

[0892] Example: From Shibuya Station to Shinjuku Station

[0893] 1. Obtaining location information

[0894] Device: Obtain current location (Shibuya Station) using GPS.

[0895] User: Enter "Shinjuku Station" in the app's search box.

[0896] 2. Emotional Recognition

[0897] Device: Captures your face and voice with a camera or microphone.

[0898] Server: Recognizes the user's emotions using the emotion engine. For example, it detects that the user is in a hurry.

[0899] 3. Traffic information inquiry

[0900] Device: Generate API request data including latitude and longitude information (Shibuya Station), destination "Shinjuku Station", and emotion data.

[0901] Terminal: Sends the generated request to the API.

[0902] 4. Calculating the optimal route

[0903] Server: Receives API requests and sends queries to the database.

[0904] Server: Obtain information about delays on the Yamanote Line.

[0905] Server: Because the user is in a hurry, calculate a route using the Keio Line with fewer transfers.

[0906] 5. Route provision

[0907] Server: Generate calculation results (Shibuya Station to Shinjuku Station, using the Keio Line) in JSON format.

[0908] Server: Sends the calculation results to the terminal.

[0909] Terminal: Parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take approximately 8 minutes."

[0910] User: Follow the instructions displayed and board the Keio Line.

[0911] In this way, the user can receive optimal route instructions that utilize real-time information and take into account their emotional state. The specific processing steps will be described later.

[0912] The processing flow will be explained below.

[0913] Step 1:

[0914] The device uses the GPS sensor to obtain the latitude and longitude of the current location, and passes this information to the application.

[0915] Step 2:

[0916] The user uses the application interface to enter a destination, for example, "Shinjuku Station."

[0917] Step 3:

[0918] The device uses a microphone or camera to capture the user's voice and facial images, and passes this emotional data to the application.

[0919] Step 4:

[0920] The request data is created based on the current location information acquired by the device, the destination entered by the user, and emotion data. This data includes latitude and longitude information, the name of the destination, and emotion data.

[0921] Step 5:

[0922] The request data created by the device is sent to the traffic information acquisition API. The data is sent securely using the HTTPS protocol.

[0923] Step 6:

[0924] The server receives the request sent from the terminal.

[0925] Step 7:

[0926] Based on the request received by the server, the server queries the transport company database to obtain information such as transport schedules, operation status, and delay information.

[0927] Step 8:

[0928] The server obtains real-time information about the operation status of public transport, such as delays on the Yamanote Line.

[0929] Step 9:

[0930] The server uses an emotion engine to analyze the user's emotional information, such as whether the user is in a hurry or relaxed, based on changes in voice tone and facial expressions.

[0931] Step 10:

[0932] The server calculates the optimal route from the current location to the destination based on traffic information, timetable information, operation status, and user emotion information, taking into account travel time, number of transfers, operation delays, and route adjustments according to the user's emotion.

[0933] Step 11:

[0934] The server generates the calculation results in data format such as JSON, which includes the departure point, arrival point, route name, transfer method, and estimated arrival time.

[0935] Step 12:

[0936] The server sends the generated calculation results to the terminal.

[0937] Step 13:

[0938] The terminal receives the calculation results sent from the server, parses the received data, and displays it in a format that is easy for the user to view.

[0939] Step 14:

[0940] The user looks at the device screen to see the optimal route displayed, and then receives instructions on which public transport to use next, where to transfer, etc.

[0941] Example 2

[0942] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0943] Conventional public transport guidance systems provide the optimal route based on the user's current location and destination, but they do not take into account the user's emotional state, making it difficult to respond flexibly to the user's situation. Furthermore, simply providing the shortest route does not necessarily provide the optimal method for the user, as it cannot respond to sudden delays or congestion.

[0944] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: location information acquisition means for acquiring a current location; input means for receiving input information related to a destination; information acquisition means for acquiring traffic information in real time based on the current location acquired by the location information acquisition means and the destination received by the input means; calculation means for calculating an optimal route based on the traffic information acquired by the information acquisition means; emotion recognition means for acquiring a user's voice or image data and recognizing emotions; calculation means for calculating an optimal route taking into account the user's emotion recognized by the emotion recognition means; and provision means for providing the user with the optimal route calculated by the calculation means. This enables a more flexible and adaptable optimal route to be proposed taking into account the user's emotional state. Furthermore, by acquiring and analyzing traffic information in real time, a route that can accommodate unexpected service delays and congestion can be provided.

[0945] "Location information acquisition means" refers to a device or function for acquiring the user's current location using a satellite positioning system sensor.

[0946] The "input means" is an interface through which the user inputs information about the destination.

[0947] The "information acquisition means" is a device or function for acquiring traffic information in real time based on the current location information acquired by the position information acquisition means and the destination information received by the input means.

[0948] The "calculation means" is a device or function for calculating an optimum route based on the traffic information acquired by the information acquisition means.

[0949] The "emotion recognition means" is a device or function for acquiring the user's voice and image data, analyzing them, and recognizing the user's emotions.

[0950] "Providing means" refers to a device or function for providing the calculated optimum route to the user.

[0951] "User's emotional state" is the user's mental or emotional state recognized by the emotion recognition means.

[0952] A "satellite positioning system sensor" is a sensor that acquires location information using a global positioning system (GPS) or similar.

[0953] A "data communication interface" is a device or function for obtaining external information using a communication protocol.

[0954] "Getting traffic information in real time" means getting current traffic conditions and operation information instantly.

[0955] The present invention is a system that suggests public transportation options in real time that will get a user from their current location to their destination in the shortest time, and also combines an emotion engine. This system includes a location information acquisition means, an input means, an information acquisition means, a calculation means, an emotion recognition means, and a provision means. Details of each means and the operation of the system are explained below.

[0956] Specific hardware and software

[0957] Location information acquisition means: A device that acquires the user's current location using a satellite positioning system (GPS) sensor.

[0958] Input method: The interface of the application through which the user enters their destination.

[0959] Information acquisition means: A device for acquiring traffic information through a data communication interface.

[0960] Calculation means: The calculation module in the server calculates the optimal route based on the data obtained from the information acquisition means.

[0961] Emotion recognition means: An emotion recognition module that acquires and analyzes the user's voice and image data.

[0962] Provision means: A terminal display and application for providing the user with the optimal route calculated by the calculation means.

[0963] System Operation

[0964] First, the device uses the GPS sensor to acquire the user's current location. The user inputs the destination through the application interface and the information is sent to the system. Then, the location information acquisition means identifies the user's current location, and the input means receives the destination information.

[0965] Next, the device captures the user's voice and facial images using a microphone and camera, and passes them to the emotion recognition means. The emotion recognition means runs on the server and analyzes the received voice and image data to recognize the user's emotions. As a result of the analysis, the user's emotional state, such as impatience, anxiety, or calmness, is identified.

[0966] Based on these data, the terminal compiles the current location information, destination information, and the user's emotional state into one request data, and acquires real-time traffic information through the information acquisition means. This data acquisition is performed via the data communication interface.

[0967] The server receives the request data sent from the information acquisition means and queries an internal database to obtain information on the operation status of public transport, delay information, transfer information, etc. It also obtains the latest public transport information using an external Web API. Based on this data, the calculation means calculates the optimal route.

[0968] In the calculation, the server takes into account the user's emotional state. For example, if the user is in a hurry, it will prioritize routes with fewer transfers. On the other hand, if the user has time, it will suggest scenic or comfortable routes.

[0969] The calculation results are sent from the server to the terminal and provided to the user. The application on the terminal displays the calculation results to the user through the means of provision, and gives specific instructions such as "If you take the Keio Line to Shinjuku Station, the journey will take about 8 minutes."

[0970] Specific examples

[0971] For example, consider a case where a user wants to travel from Shibuya Station to Shinjuku Station. First, the device's GPS acquires the current location, Shibuya Station, and the user enters "Shinjuku Station" into the application. Next, the device uses the camera and microphone to capture the user's facial expressions and voice, and the server recognizes the emotion of "I'm in a hurry."

[0972] The device then generates a request containing the current location (Shibuya Station), the destination (Shinjuku Station), and the user's emotion data, and sends it to the traffic information API. The server receives the request and queries the database and external APIs to collect traffic information. If the Yamanote Line is delayed, it is perceived that the user is in a hurry, and it is determined that the Keio Line, which has fewer transfers, is the optimal route.

[0973] Finally, the server generates the calculation result in JSON format and sends it to the terminal. The terminal parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take about 8 minutes." The user follows this and boards the Keio Line to Shinjuku Station.

[0974] Prompt Sentence Examples

[0975] Examples of prompts for generating emotion recognition and route calculation instructions using generative AI models are as follows:

[0976] "Please explain how a system that suggests the public transport option that will get a user from their current location to their destination in real time works by combining it with an emotion engine. Specifically, please explain how the current location is obtained using GPS, the user inputs their destination, emotion recognition is performed, the procedure for collecting traffic information, how the optimal route is calculated, and the process for providing the route to the user."

[0977] The above is a specific embodiment of the present invention. This system provides users with real-time recommendations for optimal public transport routes, and even tailored to their individual emotional state, greatly improving convenience.

[0978] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0979] Step 1: Get your current location

[0980] The device uses the GPS sensor to obtain the latitude and longitude of the user's current location, which is then sent to the application backend.

[0981] Input: Location data from the GPS sensor

[0982] Data processing: Convert location data into application format

[0983] Output: Latitude and longitude information (e.g., latitude: 35.6581, longitude: 139.7414)

[0984] Step 2: Enter your destination

[0985] The user enters a destination into the application's search box, and the device sends the entered data to the application's backend.

[0986] Input: Destination information entered by the user (text format)

[0987] Data processing: Save the entered text as destination information

[0988] Output: Destination information (e.g., destination: "Shinjuku Station")

[0989] Step 3: Obtaining emotion data

[0990] The device uses a microphone and camera to capture the user's voice and facial images, and the captured data is passed to the emotion recognition means.

[0991] Input: Audio and image data obtained from microphones and cameras

[0992] Data processing: Converting audio and image data into a format that can be analyzed by emotion recognition algorithms

[0993] Output: Audio data and image data (e.g., image_data: [binary data], audio_data: [binary data])

[0994] Step 4: Analyze the sentiment data

[0995] The server analyzes the acquired voice and image data to recognize the user's emotions. This analysis is performed by the emotion recognition module.

[0996] Input: Audio and image data

[0997] Data processing: Analyzing voice tone and facial expression changes

[0998] Output: User's emotional state (e.g. emotion: "anxious")

[0999] Step 5: Check traffic information

[1000] The terminal creates request data based on the current location, destination, and emotion data, and acquires traffic information in real time through the information acquisition means.

[1001] Input: current location information, destination information, emotional state

[1002] Data processing: Assemble these data into a single request object

[1003] Output: Request data (e.g., { currentLocation: {latitude: 35.6581, longitude: 139.7414}, destination: "Shinjuku Station", emotion: "Impatience"})

[1004] Step 6: Obtaining real-time traffic information

[1005] The terminal sends the request data to the traffic information acquisition API to obtain real-time traffic information.

[1006] Input: Request data

[1007] Data processing: Communication with external traffic information databases via API

[1008] Output: Real-time traffic information (e.g. { line: "Yamanote Line", status: "Delay"})

[1009] Step 7: Calculate the optimal route

[1010] The optimal route is calculated based on real-time traffic information obtained by the server and the user's emotional state.

[1011] Input: Real-time traffic information, emotional state, current location information, destination information

[1012] Data processing: This information is input into an algorithm to calculate the optimal route.

[1013] Output: Optimal route data (e.g., Optimal route calculated: Shibuya Station → Keio Line → Shinjuku Station)

[1014] Step 8: Generate route calculation results

[1015] The server generates the optimal route calculation results in JSON format and sends them to the terminal.

[1016] Input: Optimal route data

[1017] Data processing: Convert calculation results into JSON format

[1018] Output: Calculation result in JSON format (e.g., {"origin": "Shibuya Station", "destination": "Shinjuku Station", "route": "Keio Line", "estimated_time": "8 minutes"})

[1019] Step 9: Receive and display the calculation results

[1020] The terminal receives the calculation results sent from the server, analyzes them, and provides them to the user.

[1021] Input: Calculation result in JSON format

[1022] Data processing: Parse JSON data and format it for display

[1023] Output: Display data (e.g. "The journey to Shinjuku Station on the Keio Line takes approximately 8 minutes.")

[1024] Step 10: Deliver to users

[1025] The device displays the formatted calculation results to the user and gives instructions on what to do next.

[1026] Input: Display data

[1027] Data processing: Display on screen

[1028] Output: Displayed instructions (e.g., the user looks at the device and performs the action "take the Keio Line")

[1029] (Application example 2)

[1030] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1031] Conventional navigation systems provide an optimal route based on a user's current location and destination, but do not consider the user's emotional state. As a result, they are unable to provide an appropriate route based on the user's emotional state, such as when the user is in a hurry or relaxed. For example, a user may want to be presented with the shortest route when in a hurry, or with a scenic route when relaxed. The present invention aims to realize a more comfortable and stress-free journey by providing route suggestions based on the user's emotional state.

[1032] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1033] In this invention, the server includes emotion recognition means for analyzing the emotional state of the user, route optimization means for adjusting an optimal route taking into consideration the emotional state of the user acquired by the emotion recognition means, and information presentation means for presenting the route information provided by the route provision means through an in-vehicle display device and voice guidance system, thereby making it possible to propose an optimal route based on the emotional state of the user.

[1034] "Location information acquisition means" is a device for acquiring latitude and longitude information of the current location. It mainly has the function of identifying the current location using a global positioning system sensor, etc.

[1035] An "input means" is a device through which a user inputs destinations and other information. This typically includes a touchscreen or voice input device.

[1036] "Traffic information acquisition means" refers to a device for acquiring traffic conditions and operation information in real time. This includes methods of acquiring information via web APIs.

[1037] The "route calculation means" is a device that calculates the optimal route based on the current location, destination, and traffic information. It has the function of using an algorithm to calculate the optimal route based on conditions such as the shortest time and the fewest transfers.

[1038] "Route providing means" refers to a device that provides optimal route information to users. It has the function of displaying or notifying route information through a display, voice guidance system, etc.

[1039] The "emotion recognition means" is a device for analyzing the user's emotional state. It has the function of analyzing the user's emotions based on data acquired using a camera or microphone.

[1040] The "route optimization means" is a device that adjusts the optimal route taking into consideration the emotional state of the user acquired by the emotion recognition means. For example, it has the function of suggesting the shortest route if the user is in a hurry, or the scenic route if the user is relaxed, depending on the user's emotions.

[1041] The "information presentation means" is a device for presenting the route information provided by the route provision means through a display device and a voice guidance system in the vehicle, and has the function of conveying the route information to the user visually and audibly.

[1042] The present invention is a navigation system for autonomous vehicles that proposes the optimal route from a user's current location to their destination, and is characterized by its consideration of the user's emotional state. This system optimizes the route based on the user's emotional state to ensure a comfortable journey for the user.

[1043] System Configuration

[1044] A system for implementing the present invention includes the following major hardware and software components:

[1045] Location information acquisition method: Use a global positioning system sensor or similar to acquire latitude and longitude information of the current location.

[1046] Input method: The user inputs destination information using a touchscreen or voice input device.

[1047] Traffic information acquisition method: Use a web API (e.g., Google Maps API) to acquire real-time traffic information.

[1048] Route calculation method: An algorithm is used to calculate the optimal route based on the acquired traffic information.

[1049] Route provision means: Calculated route information is provided to the user through a display or voice guidance system.

[1050] Emotion recognition: The user's emotional state is analyzed using a camera and microphone. The analysis is performed using an emotion recognition engine such as Microsoft Azure Cognitive Services.

[1051] Route optimization means: The emotion data obtained by emotion recognition is taken into account to adjust the optimal route.

[1052] Information presentation means: Presents the route information provided by the route provision means through a display device and a voice guidance system in the vehicle.

[1053] Program processing and explanation

[1054] The whole system works as follows:

[1055] 1. Acquisition and analysis of emotion data

[1056] Cameras and microphones installed in the vehicle capture the user's facial images and voice. This data is passed to an emotion recognition engine for analysis. The analysis results identify the user's emotional state, such as whether they are in a hurry or relaxed.

[1057] 2. Data collection for route calculation

[1058] The server obtains the current location information and the destination information set by the user, and generates an API request that includes the emotional state along with the obtained current location and destination information.

[1059] 3. Obtaining traffic information and calculating the optimal route

[1060] The server retrieves real-time traffic information from a traffic information database via API requests. Traffic information includes operation status and delay information. The optimal route is calculated based on the retrieved traffic information and the user's emotional state.

[1061] 4. Providing route information

[1062] The calculated optimal route is sent from the server to the terminal inside the vehicle. The terminal analyzes the route information and displays it on the vehicle's display device. The user is then guided through the voice guidance system.

[1063] Specific examples

[1064] Below is a specific example where a user wants to go from Shibuya Station to Shinjuku Station. At this time, the user is a little impatient.

[1065] Acquiring and analyzing emotion data

[1066] A camera inside the vehicle captures the user's face, and an emotion recognition engine determines that the user is anxious.

[1067] Data collection for route calculation

[1068] The server obtains the location information of the current location (Shibuya Station) and the destination information (Shinjuku Station) entered by the user.

[1069] Obtaining traffic information and calculating the optimal route

[1070] Traffic information is obtained through a web API and the shortest route using the Keio Line is calculated.

[1071] Providing route information

[1072] The calculated route to Shinjuku Station using the Keio Line is sent to the terminal, which then displays this information on its display and announces via a voice guidance system that "If you take the Keio Line to Shinjuku Station, the journey will take approximately eight minutes."

[1073] Prompt Sentence Examples

[1074] An example of a prompt sentence is, "I'm going to Shinjuku Station. I'm currently at Shibuya Station and I'm a little anxious. Please tell me the best route."

[1075] In this way, a system can be realized that suggests optimal routes based on real-time traffic conditions while taking into account the user's emotional state.

[1076] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1077] Step 1:

[1078] The device uses cameras and microphones installed in the vehicle to capture the user's facial images and voice, which then generates data reflecting the user's emotional state. The acquired data is then sent to emotion recognition means.

[1079] Input: face image, voice data

[1080] Output: Raw data passed to the emotion recognition engine

[1081] Step 2:

[1082] The server uses an emotion recognition engine (e.g., Microsoft Azure Cognitive Services) to analyze the acquired facial images and voice data. As a result of the analysis, the user's emotional state (e.g., anxious, relaxed, etc.) is identified.

[1083] Input: face image, voice data

[1084] Output: User's emotional state (e.g., anxious, relaxed)

[1085] Step 3:

[1086] The user inputs the destination into the device. In this process, the destination information, such as "Shinjuku Station," is entered using a touch screen or voice input device. The device then sends the entered destination information along with the current location information to the server.

[1087] Input: Destination information (e.g. Shinjuku Station)

[1088] Output: Destination information is sent to the server

[1089] Step 4:

[1090] The device uses the GPS sensor to obtain the user's current location. The latitude and longitude information of the current location is acquired and sent to the server.

[1091] Input: GPS data

[1092] Output: Current location information (latitude and longitude) is sent to the server

[1093] Step 5:

[1094] The server generates API request data including the current location information, destination information, and emotional state, and sends the request to the web API for traffic information acquisition. The request includes a query to find out the current traffic situation.

[1095] Input: current location information, destination information, emotional state

[1096] Output: API request data sent

[1097] Step 6:

[1098] The server receives real-time traffic information from a web API for traffic information acquisition. The received data includes real-time traffic information such as operation status and delay information.

[1099] Input: API request data

[1100] Output: Real-time traffic information

[1101] Step 7:

[1102] The server calculates the optimal route based on traffic information and the user's emotional state. If the user is in a hurry, the shortest route will be selected, and if the user is relaxed, the route with the best scenery will be selected. The calculated route information is generated in JSON format.

[1103] Input: Traffic information, emotional state

[1104] Output: Optimal route information (JSON format)

[1105] Step 8:

[1106] The server sends the calculated optimal route information to the terminal.

[1107] Input: Optimal route information (JSON format)

[1108] Output: Route information is sent to the device

[1109] Step 9:

[1110] The device analyzes the received route information and displays it in a user-friendly format (e.g., displaying the route on the vehicle's display). It also provides route instructions to the user through a voice guidance system.

[1111] Input: Optimal route information (JSON format)

[1112] Output: The route is displayed on the display and voice guidance is provided.

[1113] In this way, an optimal route is provided based on real-time traffic information while taking into account the user's emotional state.

[1114] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1115] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1116] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1117] [Third embodiment]

[1118] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1119] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1120] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1121] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1122] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1123] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1124] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1125] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1126] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1127] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1128] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1129] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1130] The present invention relates to a system that suggests public transportation options in real time that will allow a user to arrive at their destination from their current location in the shortest time. This system includes a location information acquisition means, an input means, a traffic information acquisition means, a route calculation means, and a route provision means. Details of each means and the operation of the system are explained below.

[1131] 1. Obtaining location information

[1132] 1.1 Getting your current location

[1133] Device: Uses the GPS sensor to obtain current location. The device's GPS chip receives signals from satellites and obtains latitude and longitude information. This information is passed to the application.

[1134] 1.2 Entering your destination

[1135] User: Enters a destination through the application interface, for example, "Shinjuku Station."

[1136] Terminal: Passes the destination entered by the user to the application.

[1137] 2. Traffic information inquiry

[1138] 2.1 Making an API request

[1139] Device: Creates data for requesting the real-time traffic information API based on the current location and destination information, including latitude and longitude information and text information for the destination.

[1140] 2.2 Submitting a Request

[1141] Terminal: Sends the created request to the traffic information API. Sends data securely using the HTTPS protocol.

[1142] 3. Calculating the optimal route

[1143] 3.1 Receiving a Request

[1144] Server: Receives requests from the terminal via the traffic information API.

[1145] 3.2 Querying the Database

[1146] Server: Based on the received request, queries the transport company's database, which contains information on timetables, operation status, delays, etc.

[1147] 3.3 Obtaining real-time information

[1148] Server: Obtain real-time information on transportation status, such as delays on the Yamanote Line.

[1149] 3.4 Optimal Route Calculation

[1150] Server: Based on the acquired information, the server calculates the optimal route from the current location to the destination. The calculation takes into account the travel time, number of transfers, and whether or not there are any delays. Specifically, route search algorithms such as the Dijkstra algorithm and the A algorithm are used.

[1151] 4. Route provision

[1152] 4.1 Generating calculation results

[1153] Server: Generates the results of the calculation in a data format such as JSON, which contains the following information:

[1154] Departure and arrival points

[1155] Route name to be used

[1156] How to transfer

[1157] Estimated arrival time

[1158] 4.2 Sending a Response

[1159] Server: Sends the calculation results to the terminal.

[1160] 4.3 Receiving and Displaying Data

[1161] Terminal: Parses the calculation results received from the server and displays them in a user-friendly format. Route guidance, timetables, transfer instructions, etc. are displayed on the screen.

[1162] 4.4 User Visibility

[1163] User: Looks at the device screen to see the optimal route displayed, and receives instructions on which train or bus to take next, where to transfer, etc.

[1164] Example: From Shibuya Station to Shinjuku Station

[1165] 1. Obtaining location information

[1166] Device: Obtain current location (Shibuya Station) using GPS.

[1167] User: Enter "Shinjuku Station" in the app's search box.

[1168] 2. Traffic information inquiry

[1169] Terminal: Generate API request data including latitude and longitude information (Shibuya Station) and destination "Shinjuku Station".

[1170] Terminal: Sends the generated request to the API.

[1171] 3. Calculating the optimal route

[1172] Server: Receives API requests and sends queries to the database.

[1173] Server: Obtain information about delays on the Yamanote Line.

[1174] Server: Calculates a route that uses the Keio Line instead of the Yamanote Line.

[1175] 4. Route provision

[1176] Server: Generate calculation results (Shibuya Station to Shinjuku Station, using the Keio Line) in JSON format.

[1177] Server: Sends the calculation results to the terminal.

[1178] Terminal: Parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take approximately 8 minutes."

[1179] User: Follow the instructions displayed and board the Keio Line.

[1180] In this way, the user can receive optimal route instructions utilizing real-time information. The specific processing steps will be explained separately.

[1181] The processing flow will be explained below.

[1182] Step 1:

[1183] The device uses the GPS sensor to obtain the latitude and longitude of the current location, and passes this information to the application.

[1184] Step 2:

[1185] The user uses the application interface to enter a destination, for example, "Shinjuku Station."

[1186] Step 3:

[1187] The device creates request data based on the current location information acquired and the destination entered by the user. This data includes the latitude and longitude of the current location and the name of the destination.

[1188] Step 4:

[1189] The request data created by the device is sent to the traffic information acquisition API. The data is sent securely using protocols such as HTTPS.

[1190] Step 5:

[1191] The server receives the request sent from the terminal.

[1192] Step 6:

[1193] Based on the request received by the server, the server queries the transport company database to obtain information such as transport schedules, operation status, and delay information.

[1194] Step 7:

[1195] The server obtains real-time information about the operation status of public transport, such as delays on the Yamanote Line.

[1196] Step 8:

[1197] Based on the information acquired by the server, the optimal route from the current location to the destination is calculated, taking into account the required time, number of transfers, and information on train delays.

[1198] Step 9:

[1199] The server generates the calculation results in data format such as JSON, which includes the departure point, arrival point, route name, transfer method, estimated arrival time, etc.

[1200] Step 10:

[1201] The server sends the generated calculation results to the terminal.

[1202] Step 11:

[1203] The terminal receives the calculation results sent from the server, parses the received data, and displays it in a format that is easy for the user to view.

[1204] Step 12:

[1205] The user looks at the device screen to see the optimal route displayed, and then receives instructions on which public transport to use next, where to transfer, etc.

[1206] Example 1

[1207] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1208] Currently, when users reach their destination using public transportation, it is difficult to determine the optimal route that takes into account real-time traffic information and service delays. Furthermore, when transferring between multiple modes of transportation, there are few systems that provide integrated transfer information, which often makes users' travel complicated. Therefore, there is a need for a system that instantly provides the optimal route based on real-time information.

[1209] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1210] In this invention, the server includes a location information acquisition means for acquiring the current location, an input means for receiving input information about the destination, and a means for acquiring traffic information in real time, thereby making it possible to provide an optimal route taking into account real-time traffic conditions.

[1211] "Location information acquisition means" refers to devices or functions for acquiring the user's current location.

[1212] "Input means" refers to an interface or device for receiving destination-related information from a user.

[1213] "Traffic information acquisition means" refers to functions and devices for acquiring traffic information in real time.

[1214] "Optimal route calculation means" refers to a function or device for calculating the optimal travel route based on acquired traffic information and transfer information.

[1215] "Route providing means" refers to a means for providing information on the calculated optimal route to a user.

[1216] "Satellite Positioning System Sensor" means a positioning system that uses signals from satellites to determine location on Earth.

[1217] "Web Application Programming Interface" refers to a means for communicating with external software or applications through web services.

[1218] "Transportation information" refers to information such as public transportation schedules, operation status, and delay information.

[1219] The present invention relates to a system that suggests public transportation options in real time that will allow a user to arrive at their destination from their current location in the shortest time. This system is composed of the following means, including location information acquisition means, input means, traffic information acquisition means, optimal route calculation means, and route provision means.

[1220] Obtaining location information

[1221] Get current location

[1222] Device: The GPS sensor is used to obtain the user's current location. The GPS chip in the device receives signals from satellites, extracts latitude and longitude information, and passes it to the application.

[1223] Enter your destination

[1224] User: Enters a destination through the application interface. For example, the user enters "Shinjuku Station."

[1225] Terminal: Passes the destination information entered by the user to the application.

[1226] Traffic information inquiry

[1227] Making API Requests

[1228] Device: Creates request data for the real-time traffic information API based on the current location and destination information. This data includes the acquired latitude and longitude information and the destination information entered by the user.

[1229] Submitting a Request

[1230] Terminal: Sends the created request data to the API using the HTTPS protocol.

[1231] Calculating the best route

[1232] Receiving a request

[1233] Server: Queries the transit agency database based on requests received via the API.

[1234] Server: The database contains information on transportation schedules, operation status, delays, etc.

[1235] Obtaining real-time information

[1236] Server: Obtains real-time information about the operation of public transport. For example, obtains information such as "The Yamanote Line is currently delayed by 5 minutes."

[1237] Calculating the best route

[1238] Server: Based on the acquired data, the server calculates the optimal route from the user's current location to the destination. The calculation takes into account the travel time, number of transfers, and real-time delay information. The specific algorithms used for route search are the Dijkstra algorithm and the A algorithm.

[1239] Route provision

[1240] Generate calculation results

[1241] Server: Generates the optimal route calculation results in data format such as JSON. This data includes information such as the departure point, arrival point, route name, transfer method, and estimated arrival time.

[1242] Sending a response

[1243] Server: Sends the generated calculation results to the terminal.

[1244] Receiving and displaying data

[1245] Terminal: Parses the calculation results received from the server and displays them in a user-friendly format. For example, route guidance, timetables, transfer procedures, etc. are displayed on the screen.

[1246] User Visibility

[1247] User: Check the information displayed on the device and follow the instructions to begin traveling. For example, follow the instructions such as "Take the Keio Line to Shinjuku Station, it will take about 8 minutes."

[1248] Example: From Shibuya Station to Shinjuku Station

[1249] Get current location:

[1250] Device: Obtain current location (Shibuya Station) using GPS.

[1251] Enter your destination:

[1252] User: Enters "Shinjuku Station" into the app's search box.

[1253] Terminal: Obtain the entered destination information.

[1254] Make an API request:

[1255] Terminal: Generate API request data including latitude and longitude information (Shibuya Station) and destination "Shinjuku Station".

[1256] Sending an API request:

[1257] Terminal: Sends the generated request data to the API.

[1258] Query the database:

[1259] Server: Receives API requests and sends queries to the transit agency database.

[1260] Get real-time information:

[1261] Server: Obtain delay information for the Yamanote Line.

[1262] Calculating the best route:

[1263] Server: Calculate a route that avoids delays on the Yamanote Line and uses the Keio Line.

[1264] Generate calculation results:

[1265] Server: Generate the calculation result (Shibuya Station to Shinjuku Station, using the Keio Line) in JSON format.

[1266] Sending a response:

[1267] Server: Sends the calculation results to the terminal.

[1268] Receiving and displaying data:

[1269] Terminal: Parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take approximately 8 minutes."

[1270] Check the results and follow the instructions:

[1271] User: Follow the displayed instructions and use the Keio Line.

[1272] Example prompt sentence:

[1273] "I'm currently at Shibuya Station and my destination is Shinjuku Station. Please tell me the best route taking real-time traffic information into account."

[1274] This system allows users to receive optimal route instructions based on real-time information, enabling them to use public transportation efficiently.

[1275] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1276] Step 1: Obtaining location information

[1277] Device: The device uses a GPS sensor to obtain the user's current location. Specifically, the device's GPS chip receives signals from satellites and generates latitude and longitude information. This location information is passed to the application. The input is the signal from the GPS, and the output is the user's latitude and longitude information.

[1278] Step 2: Enter your destination

[1279] User: The user inputs a destination through the application interface. For example, they input "Shinjuku Station." The input is text input by the user, and the output is the destination information "Shinjuku Station."

[1280] Terminal: The terminal passes the destination information entered by the user to the application. The input is text data from the user, and the output is the destination information passed to the application.

[1281] Step 3: Making an API Request

[1282] Terminal: The terminal creates request data for the real-time traffic information API based on the current location (latitude and longitude information) and destination information. Specifically, this information is combined and placed in a request format. The input is latitude and longitude information and destination information, and the output is the API request data.

[1283] Step 4: Sending an API Request

[1284] Terminal: The terminal uses the HTTPS protocol to send the created request data to the API. The input is the API request data, and the output is communication to the API server.

[1285] Step 5: Receiving the request

[1286] Server: The API server receives the request data sent from the terminal. The input is the request data, and the output is a confirmation of the received request data.

[1287] Step 6: Query the database

[1288] Server: The server queries the traffic information database based on the received request. Specifically, it searches for schedule information, operation status, and delay information. The input is the request data, and the output is the traffic information retrieved from the database.

[1289] Step 7: Get real-time information

[1290] Server: The server obtains real-time operational status of transportation facilities. For example, it obtains information such as "The Yamanote Line is currently delayed by 5 minutes." The input is real-time data of transportation facilities, and the output is the latest operational status.

[1291] Step 8: Calculate the optimal route

[1292] Server: The server calculates the optimal route from the current location to the destination based on the acquired data. It uses the Dijkstra algorithm and the A algorithm to take into account the travel time, number of transfers, and real-time delay information. The input is traffic information and real-time operation status, and the output is the optimal route information.

[1293] Step 9: Generate calculation results

[1294] Server: The server generates the optimal route calculation results in data format such as JSON. This data includes the departure point, arrival point, route name to be used, transfer method, and estimated arrival time. The input is the optimal route information, and the output is data in JSON format.

[1295] Step 10: Sending the Response

[1296] Server: The server sends the generated calculation results to the terminal. The input is JSON format data, and the output is the response to the terminal.

[1297] Step 11: Receiving and displaying data

[1298] Terminal: The terminal parses the calculation results received from the server and displays them in a user-friendly format. Route guidance, timetables, and transfer procedures are displayed on the screen. The input is the response data from the server, and the output is the information displayed on the user interface.

[1299] Step 12: Review the results and follow the instructions

[1300] User: The user checks the information displayed on the device and starts moving according to the instructions. For example, they follow instructions such as "Take the Keio Line to Shinjuku Station, it will take about 8 minutes." The input is the information displayed on the device, and the output is the user's actions.

[1301] (Application example 1)

[1302] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1303] While many conventional public transportation guidance systems provide the optimal route from a user's current location to their destination, they are inadequate when it comes to providing destination guidance within shopping malls and large facilities. Providing adequate guidance is particularly difficult when a user has multiple destinations or when congestion needs to be taken into consideration. There is also a growing need for intuitive operation using voice input in addition to visual guidance information. The present invention aims to solve these problems by providing a system that provides real-time guidance on the shortest route to a destination within a shopping mall.

[1304] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1305] In this invention, the server includes location information acquisition means for acquiring location information, input means for receiving input information related to a destination, information acquisition means for acquiring traffic information in real time, route calculation means for calculating an optimal route, route provision means for providing the optimal route to a user, display means for visually guiding the optimal route based on real-time congestion status, and voice input means for inputting a destination using voice recognition, thereby making it possible to provide the shortest route to a destination within a shopping mall in real time by visual and voice input.

[1306] "Location information acquisition means" refers to a device or system for acquiring the user's current location, and utilizes a satellite system or GPS sensor.

[1307] "Input means" refers to a device or system for receiving information about a destination from a user, and includes a touch panel, a keyboard, a voice recognition system, and the like.

[1308] The "information acquisition means" is a device or system for acquiring data such as traffic information and congestion status in real time, and acquires data via a network.

[1309] The "route calculation means" is a device or system for calculating the optimum route based on the acquired information, and uses a route search algorithm that takes into account the required travel time and traffic congestion.

[1310] "Route providing means" refers to a device or system for providing the user with the calculated optimum route, and conveys information through a screen display, voice guidance, etc.

[1311] "Display means" refers to a device or system that visually guides passengers along optimal routes based on real-time congestion information, and uses smart glasses or a head-mounted display.

[1312] "Voice input means" refers to a device or system that utilizes voice recognition technology to receive destination information from a user.

[1313] The embodiment of the present invention is directed to constructing a system that guides users so that they can reach their destination from their current location in the shortest time possible. This system is particularly effective for providing guidance within large facilities such as shopping malls. Detailed embodiments of this system are described below.

[1314] 1. System Configuration

[1315] The system consists of the following major components:

[1316] Location information acquisition means

[1317] Input Method

[1318] Information acquisition means

[1319] Route calculation method

[1320] Route provision method

[1321] Display means

[1322] Voice input means

[1323] Location information acquisition means

[1324] Location information acquisition means has the function of acquiring the user's current location. It mainly uses satellite systems and GPS sensors to acquire the current latitude and longitude information. For example, a GPS chip installed in smart glasses measures the user's location in real time.

[1325] Input Method

[1326] The input means has the function of receiving information about the destination from the user. This can be done using a touch panel, keyboard, or voice recognition system. Specifically, it is possible to use the voice recognition system built into the smart glasses to input destinations such as "supermarket" or "toilet" by voice.

[1327] Information acquisition means

[1328] The information acquisition means has the function of acquiring data such as traffic information and congestion status in real time. This data is acquired via the network, for example, using the Google Maps API or a shopping mall's dedicated API.

[1329] Route calculation method

[1330] The route calculation means has the function of calculating the optimal route based on the acquired information. For this, a route search algorithm (such as the Dijkstra algorithm or the A algorithm) is used that takes into account the required time and congestion status.

[1331] Route provision method

[1332] The route providing means has the function of providing the user with the calculated optimal route, and displays visual guidance information on the smart glasses display or the smartphone screen.

[1333] Display means

[1334] The display function visually guides users to the optimal route based on real-time congestion information, primarily using smart glasses or head-mounted displays to visually show the user the next direction to go and the route to their destination.

[1335] Voice input means

[1336] The voice input means has the function of receiving destination information from the user using voice recognition technology, using a microphone and voice recognition software built into the smart glasses.

[1337] Program processing explanation

[1338] This system works in conjunction with each mode of transportation to provide users with optimal route guidance. The specific processing flow is explained below.

[1339] 1. Location information acquisition means

[1340] It uses a GPS sensor to obtain the user's current location and transmits that information to a server. For example, the GPS chip in smart glasses measures the user's location in real time and passes the latitude and longitude data to the application.

[1341] 2. Input Method

[1342] The user inputs their destination through a voice input system, for example by saying "I'm going to the supermarket," and the voice recognition software recognizes the destination and sends that information to the server.

[1343] 3. Information acquisition means

[1344] The server makes a request to a real-time congestion information API based on the acquired current location and destination information. Specifically, it accesses the Google Maps API or a congestion information API dedicated to shopping malls.

[1345] 4. Route calculation method

[1346] The server calculates the shortest route using the Dijkstra algorithm or A algorithm based on the acquired congestion information. For example, it calculates a route that avoids congested areas.

[1347] 5. Route provision method

[1348] The server sends the calculation results in JSON format to the smart glasses, which then display arrows and text instructions to the user based on the route information received.

[1349] Examples of concrete examples and prompts

[1350] Hardware / Software used

[1351] Reference Hardware: Smart glasses, e.g. Vuzix Blade or North Focals

[1352] Real-time location information acquisition API: Google Maps API

[1353] Server: AWS EC2

[1354] Real-time congestion information API: SmartMall API (tentative)

[1355] Prompt Sentence Examples

[1356] Generative AI model used: GPT-4

[1357] Example prompt: "Code the process flow for a smart glasses app that provides visual guidance on the shortest route from a user's current location to a destination. The destination is specified via voice input, and the route display takes real-time information into account."

[1358] This makes it possible to provide the shortest route to a destination within a shopping mall in real time through visual and voice input.

[1359] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1360] Step 1:

[1361] Get current location

[1362] Subject: Device

[1363] What it does: Uses the device's GPS sensor to obtain the user's current location.

[1364] Input: The device's GPS chip receives signals from satellites and obtains latitude and longitude information.

[1365] Output: Latitude and longitude information (e.g., current location coordinates). The device passes this information to the next processing step.

[1366] Step 2:

[1367] Enter your destination

[1368] Subject: User

[1369] How it works: The user enters destination information through the voice recognition system.

[1370] Input: The user's spoken command (e.g., "Go to the supermarket").

[1371] Output: Textual destination information (e.g., destination name) is generated and passed to the terminal.

[1372] Step 3:

[1373] Obtaining real-time information

[1374] Subject: Device

[1375] How it works: The device sends a request to a real-time congestion information API based on its current location and destination.

[1376] Input: Latitude and longitude information of current location, text information of destination.

[1377] Output: Real-time congestion data (e.g., congestion and traffic information obtained from an API) is returned to the device.

[1378] Step 4:

[1379] Calculating the best route

[1380] Subject: Server

[1381] How it works: The server calculates the optimal route based on the real-time data it receives.

[1382] Input: current location latitude and longitude, destination text, real-time congestion data.

[1383] Data processing / calculation: Using a path-finding algorithm (e.g., Dijkstra algorithm or A algorithm), calculate the route that can be reached in the shortest time.

[1384] Output: The calculated optimal route (e.g., a list of route information) is generated.

[1385] Step 5:

[1386] Route information format conversion

[1387] Subject: Server

[1388] How it works: The server converts the optimal route information into a data format such as JSON.

[1389] Input: The calculated optimal route information.

[1390] Output: Route information (e.g., navigation data) in JSON format is generated and sent to the device.

[1391] Step 6:

[1392] View route information

[1393] Subject: Device

[1394] How it works: The device parses the route information it receives and displays it visually to the user.

[1395] Input: Route information in JSON format.

[1396] Data processing / computation: The device parses the JSON data and generates a visual navigation display.

[1397] Output: Route guidance is displayed on the smart glasses display (e.g., arrows and text instructions showing the direction of travel).

[1398] Step 7:

[1399] User movement guidance

[1400] Subject: User

[1401] How it works: The user follows the guidance displayed on the smart glasses to reach their destination.

[1402] Input: Visual navigation information (e.g., arrows and text instructions).

[1403] Output: The user arrives at the destination.

[1404] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1405] The present invention combines an emotion engine with a system that suggests in real time the public transportation that will take a user from their current location to their destination in the shortest time. This system includes a location information acquisition means, an input means, a traffic information acquisition means, a route calculation means, a route provision means, and an emotion recognition means. Details of each means and the operation of the system are described below.

[1406] 1. Obtaining location information

[1407] 1.1 Getting your current location

[1408] The device uses the GPS sensor to obtain the latitude and longitude of the current location, and passes this information to the application.

[1409] 1.2 Entering your destination

[1410] The user uses the application interface to enter a destination, for example, "Shinjuku Station."

[1411] The device passes the destination entered by the user to the application.

[1412] 2. Emotional Recognition

[1413] 2.1 Acquiring Emotion Data

[1414] The device uses a microphone and camera to capture the user's voice and facial images.

[1415] The terminal passes the voice and image data to the emotion recognition means.

[1416] 2.2 Emotion Data Analysis

[1417] The server analyzes the received audio or image data to recognize the user's emotions. For example, it can identify whether the user is anxious, impatient, calm, etc. based on changes in voice tone or facial expression.

[1418] 3. Traffic information inquiry

[1419] 3.1 Making an API request

[1420] Request data is created based on the current location information acquired by the device, the destination entered by the user, and the recognized emotion data.

[1421] 3.2 Sending a Request

[1422] The request data created by the device is sent to the traffic information acquisition API. The data is sent securely using protocols such as HTTPS.

[1423] 4. Calculating the optimal route

[1424] 4.1 Receiving a Request

[1425] The server receives the request sent from the terminal.

[1426] 4.2 Querying the Database

[1427] Based on the request received by the server, the server queries the transport company database to obtain information such as transport schedules, operation status, and delay information.

[1428] 4.3 Obtaining real-time information

[1429] The server obtains real-time information about the operation status of public transport, such as delays on the Yamanote Line.

[1430] 4.4 Optimal Route Calculation

[1431] The server calculates the optimal route from the current location to the destination based on the information acquired and the user's emotional state. The calculation takes into account the required travel time, number of transfers, and service delay information, as well as the user's emotional state. Specifically, if the user is in a hurry, the system will choose a route with fewer transfers, and if the user has time, it will choose a route with better scenery.

[1432] 5. Route provision

[1433] 5.1 Generating Calculation Results

[1434] The server generates the calculation results in data format such as JSON, which includes the departure point, arrival point, route name, transfer method, and estimated arrival time.

[1435] 5.2 Sending a Response

[1436] The server sends the generated calculation results to the terminal.

[1437] 5.3 Receiving and Displaying Data

[1438] The terminal receives the calculation results sent from the server, parses the received data, and displays it in a format that is easy for the user to view.

[1439] 5.4 User Visibility

[1440] The user looks at the device screen to see the optimal route displayed, and then receives instructions on which public transport to use next, where to transfer, etc.

[1441] Example: From Shibuya Station to Shinjuku Station

[1442] 1. Obtaining location information

[1443] Device: Obtain current location (Shibuya Station) using GPS.

[1444] User: Enter "Shinjuku Station" in the app's search box.

[1445] 2. Emotional Recognition

[1446] Device: Captures your face and voice with a camera or microphone.

[1447] Server: Recognizes the user's emotions using the emotion engine. For example, it detects that the user is in a hurry.

[1448] 3. Traffic information inquiry

[1449] Device: Generate API request data including latitude and longitude information (Shibuya Station), destination "Shinjuku Station", and emotion data.

[1450] Terminal: Sends the generated request to the API.

[1451] 4. Calculating the optimal route

[1452] Server: Receives API requests and sends queries to the database.

[1453] Server: Obtain information about delays on the Yamanote Line.

[1454] Server: Because the user is in a hurry, calculate a route using the Keio Line with fewer transfers.

[1455] 5. Route provision

[1456] Server: Generate calculation results (Shibuya Station to Shinjuku Station, using the Keio Line) in JSON format.

[1457] Server: Sends the calculation results to the terminal.

[1458] Terminal: Parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take approximately 8 minutes."

[1459] User: Follow the instructions displayed and board the Keio Line.

[1460] In this way, the user can receive optimal route instructions that utilize real-time information and take into account their emotional state. The specific processing steps will be described later.

[1461] The processing flow will be explained below.

[1462] Step 1:

[1463] The device uses the GPS sensor to obtain the latitude and longitude of the current location, and passes this information to the application.

[1464] Step 2:

[1465] The user uses the application interface to enter a destination, for example, "Shinjuku Station."

[1466] Step 3:

[1467] The device uses a microphone or camera to capture the user's voice and facial images, and passes this emotional data to the application.

[1468] Step 4:

[1469] The request data is created based on the current location information acquired by the device, the destination entered by the user, and emotion data. This data includes latitude and longitude information, the name of the destination, and emotion data.

[1470] Step 5:

[1471] The request data created by the device is sent to the traffic information acquisition API. The data is sent securely using the HTTPS protocol.

[1472] Step 6:

[1473] The server receives the request sent from the terminal.

[1474] Step 7:

[1475] Based on the request received by the server, the server queries the transport company database to obtain information such as transport schedules, operation status, and delay information.

[1476] Step 8:

[1477] The server obtains real-time information about the operation status of public transport, such as delays on the Yamanote Line.

[1478] Step 9:

[1479] The server uses an emotion engine to analyze the user's emotional information, such as whether the user is in a hurry or relaxed, based on changes in voice tone and facial expressions.

[1480] Step 10:

[1481] The server calculates the optimal route from the current location to the destination based on traffic information, timetable information, operation status, and user emotion information, taking into account travel time, number of transfers, operation delays, and route adjustments according to the user's emotion.

[1482] Step 11:

[1483] The server generates the calculation results in data format such as JSON, which includes the departure point, arrival point, route name, transfer method, and estimated arrival time.

[1484] Step 12:

[1485] The server sends the generated calculation results to the terminal.

[1486] Step 13:

[1487] The terminal receives the calculation results sent from the server, parses the received data, and displays it in a format that is easy for the user to view.

[1488] Step 14:

[1489] The user looks at the device screen to see the optimal route displayed, and then receives instructions on which public transport to use next, where to transfer, etc.

[1490] Example 2

[1491] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1492] Conventional public transport guidance systems provide the optimal route based on the user's current location and destination, but they do not take into account the user's emotional state, making it difficult to respond flexibly to the user's situation. Furthermore, simply providing the shortest route does not necessarily provide the optimal method for the user, as it cannot respond to sudden delays or congestion.

[1493] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: location information acquisition means for acquiring a current location; input means for receiving input information related to a destination; information acquisition means for acquiring traffic information in real time based on the current location acquired by the location information acquisition means and the destination received by the input means; calculation means for calculating an optimal route based on the traffic information acquired by the information acquisition means; emotion recognition means for acquiring a user's voice or image data and recognizing emotions; calculation means for calculating an optimal route taking into account the user's emotion recognized by the emotion recognition means; and provision means for providing the user with the optimal route calculated by the calculation means. This enables a more flexible and adaptable optimal route to be proposed taking into account the user's emotional state. Furthermore, by acquiring and analyzing traffic information in real time, a route that can accommodate unexpected service delays and congestion can be provided.

[1494] "Location information acquisition means" refers to a device or function for acquiring the user's current location using a satellite positioning system sensor.

[1495] The "input means" is an interface through which the user inputs information about the destination.

[1496] The "information acquisition means" is a device or function for acquiring traffic information in real time based on the current location information acquired by the position information acquisition means and the destination information received by the input means.

[1497] The "calculation means" is a device or function for calculating an optimum route based on the traffic information acquired by the information acquisition means.

[1498] The "emotion recognition means" is a device or function for acquiring the user's voice and image data, analyzing them, and recognizing the user's emotions.

[1499] "Providing means" refers to a device or function for providing the calculated optimum route to the user.

[1500] "User's emotional state" is the user's mental or emotional state recognized by the emotion recognition means.

[1501] A "satellite positioning system sensor" is a sensor that acquires location information using a global positioning system (GPS) or similar.

[1502] A "data communication interface" is a device or function for obtaining external information using a communication protocol.

[1503] "Getting traffic information in real time" means getting current traffic conditions and operation information instantly.

[1504] The present invention is a system that suggests public transportation options in real time that will get a user from their current location to their destination in the shortest time, and also combines an emotion engine. This system includes a location information acquisition means, an input means, an information acquisition means, a calculation means, an emotion recognition means, and a provision means. Details of each means and the operation of the system are explained below.

[1505] Specific hardware and software

[1506] Location information acquisition means: A device that acquires the user's current location using a satellite positioning system (GPS) sensor.

[1507] Input method: The interface of the application through which the user enters their destination.

[1508] Information acquisition means: A device for acquiring traffic information through a data communication interface.

[1509] Calculation means: The calculation module in the server calculates the optimal route based on the data obtained from the information acquisition means.

[1510] Emotion recognition means: An emotion recognition module that acquires and analyzes the user's voice and image data.

[1511] Provision means: A terminal display and application for providing the user with the optimal route calculated by the calculation means.

[1512] System Operation

[1513] First, the device uses the GPS sensor to acquire the user's current location. The user inputs the destination through the application interface and the information is sent to the system. Then, the location information acquisition means identifies the user's current location, and the input means receives the destination information.

[1514] Next, the device captures the user's voice and facial images using a microphone and camera, and passes them to the emotion recognition means. The emotion recognition means runs on the server and analyzes the received voice and image data to recognize the user's emotions. As a result of the analysis, the user's emotional state, such as impatience, anxiety, or calmness, is identified.

[1515] Based on these data, the terminal compiles the current location information, destination information, and the user's emotional state into one request data, and acquires real-time traffic information through the information acquisition means. This data acquisition is performed via the data communication interface.

[1516] The server receives the request data sent from the information acquisition means and queries an internal database to obtain information on the operation status of public transport, delay information, transfer information, etc. It also obtains the latest public transport information using an external Web API. Based on this data, the calculation means calculates the optimal route.

[1517] In the calculation, the server takes into account the user's emotional state. For example, if the user is in a hurry, it will prioritize routes with fewer transfers. On the other hand, if the user has time, it will suggest scenic or comfortable routes.

[1518] The calculation results are sent from the server to the terminal and provided to the user. The application on the terminal displays the calculation results to the user through the means of provision, and gives specific instructions such as "If you take the Keio Line to Shinjuku Station, the journey will take about 8 minutes."

[1519] Specific examples

[1520] For example, consider a case where a user wants to travel from Shibuya Station to Shinjuku Station. First, the device's GPS acquires the current location, Shibuya Station, and the user enters "Shinjuku Station" into the application. Next, the device uses the camera and microphone to capture the user's facial expressions and voice, and the server recognizes the emotion of "I'm in a hurry."

[1521] The device then generates a request containing the current location (Shibuya Station), the destination (Shinjuku Station), and the user's emotion data, and sends it to the traffic information API. The server receives the request and queries the database and external APIs to collect traffic information. If the Yamanote Line is delayed, it is perceived that the user is in a hurry, and it is determined that the Keio Line, which has fewer transfers, is the optimal route.

[1522] Finally, the server generates the calculation result in JSON format and sends it to the terminal. The terminal parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take about 8 minutes." The user follows this and boards the Keio Line to Shinjuku Station.

[1523] Prompt Sentence Examples

[1524] Examples of prompts for generating emotion recognition and route calculation instructions using generative AI models are as follows:

[1525] "Please explain how a system that suggests the public transport option that will get a user from their current location to their destination in real time works by combining it with an emotion engine. Specifically, please explain how the current location is obtained using GPS, the user inputs their destination, emotion recognition is performed, the procedure for collecting traffic information, how the optimal route is calculated, and the process for providing the route to the user."

[1526] The above is a specific embodiment of the present invention. This system provides users with real-time recommendations for optimal public transport routes, and even tailored to their individual emotional state, greatly improving convenience.

[1527] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1528] Step 1: Get your current location

[1529] The device uses the GPS sensor to obtain the latitude and longitude of the user's current location, which is then sent to the application backend.

[1530] Input: Location data from the GPS sensor

[1531] Data processing: Convert location data into application format

[1532] Output: Latitude and longitude information (e.g., latitude: 35.6581, longitude: 139.7414)

[1533] Step 2: Enter your destination

[1534] The user enters a destination into the application's search box, and the device sends the entered data to the application's backend.

[1535] Input: Destination information entered by the user (text format)

[1536] Data processing: Save the entered text as destination information

[1537] Output: Destination information (e.g., destination: "Shinjuku Station")

[1538] Step 3: Obtaining emotion data

[1539] The device uses a microphone and camera to capture the user's voice and facial images, and the captured data is passed to the emotion recognition means.

[1540] Input: Audio and image data obtained from microphones and cameras

[1541] Data processing: Converting audio and image data into a format that can be analyzed by emotion recognition algorithms

[1542] Output: Audio data and image data (e.g., image_data: [binary data], audio_data: [binary data])

[1543] Step 4: Analyze the sentiment data

[1544] The server analyzes the acquired voice and image data to recognize the user's emotions. This analysis is performed by the emotion recognition module.

[1545] Input: Audio and image data

[1546] Data processing: Analyzing voice tone and facial expression changes

[1547] Output: User's emotional state (e.g. emotion: "anxious")

[1548] Step 5: Check traffic information

[1549] The terminal creates request data based on the current location, destination, and emotion data, and acquires traffic information in real time through the information acquisition means.

[1550] Input: current location information, destination information, emotional state

[1551] Data processing: Assemble these data into a single request object

[1552] Output: Request data (e.g., { currentLocation: {latitude: 35.6581, longitude: 139.7414}, destination: "Shinjuku Station", emotion: "Impatience"})

[1553] Step 6: Obtaining real-time traffic information

[1554] The terminal sends the request data to the traffic information acquisition API to obtain real-time traffic information.

[1555] Input: Request data

[1556] Data processing: Communication with external traffic information databases via API

[1557] Output: Real-time traffic information (e.g. { line: "Yamanote Line", status: "Delay"})

[1558] Step 7: Calculate the optimal route

[1559] The optimal route is calculated based on real-time traffic information obtained by the server and the user's emotional state.

[1560] Input: Real-time traffic information, emotional state, current location information, destination information

[1561] Data processing: This information is input into an algorithm to calculate the optimal route.

[1562] Output: Optimal route data (e.g., Optimal route calculated: Shibuya Station → Keio Line → Shinjuku Station)

[1563] Step 8: Generate route calculation results

[1564] The server generates the optimal route calculation results in JSON format and sends them to the terminal.

[1565] Input: Optimal route data

[1566] Data processing: Convert calculation results into JSON format

[1567] Output: Calculation result in JSON format (e.g., {"origin": "Shibuya Station", "destination": "Shinjuku Station", "route": "Keio Line", "estimated_time": "8 minutes"})

[1568] Step 9: Receive and display the calculation results

[1569] The terminal receives the calculation results sent from the server, analyzes them, and provides them to the user.

[1570] Input: Calculation result in JSON format

[1571] Data processing: Parse JSON data and format it for display

[1572] Output: Display data (e.g. "The journey to Shinjuku Station on the Keio Line takes approximately 8 minutes.")

[1573] Step 10: Deliver to users

[1574] The device displays the formatted calculation results to the user and gives instructions on what to do next.

[1575] Input: Display data

[1576] Data processing: Display on screen

[1577] Output: Displayed instructions (e.g., the user looks at the device and performs the action "take the Keio Line")

[1578] (Application example 2)

[1579] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1580] Conventional navigation systems provide an optimal route based on a user's current location and destination, but do not consider the user's emotional state. As a result, they are unable to provide an appropriate route based on the user's emotional state, such as when the user is in a hurry or relaxed. For example, a user may want to be presented with the shortest route when in a hurry, or with a scenic route when relaxed. The present invention aims to realize a more comfortable and stress-free journey by providing route suggestions based on the user's emotional state.

[1581] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1582] In this invention, the server includes emotion recognition means for analyzing the emotional state of the user, route optimization means for adjusting an optimal route taking into consideration the emotional state of the user acquired by the emotion recognition means, and information presentation means for presenting the route information provided by the route provision means through an in-vehicle display device and voice guidance system, thereby making it possible to propose an optimal route based on the emotional state of the user.

[1583] "Location information acquisition means" is a device for acquiring latitude and longitude information of the current location. It mainly has the function of identifying the current location using a global positioning system sensor, etc.

[1584] An "input means" is a device through which a user inputs destinations and other information. This typically includes a touchscreen or voice input device.

[1585] "Traffic information acquisition means" refers to a device for acquiring traffic conditions and operation information in real time. This includes methods of acquiring information via web APIs.

[1586] The "route calculation means" is a device that calculates the optimal route based on the current location, destination, and traffic information. It has the function of using an algorithm to calculate the optimal route based on conditions such as the shortest time and the fewest transfers.

[1587] "Route providing means" refers to a device that provides optimal route information to users. It has the function of displaying or notifying route information through a display, voice guidance system, etc.

[1588] The "emotion recognition means" is a device for analyzing the user's emotional state. It has the function of analyzing the user's emotions based on data acquired using a camera or microphone.

[1589] The "route optimization means" is a device that adjusts the optimal route taking into consideration the emotional state of the user acquired by the emotion recognition means. For example, it has the function of suggesting the shortest route if the user is in a hurry, or the scenic route if the user is relaxed, depending on the user's emotions.

[1590] The "information presentation means" is a device for presenting the route information provided by the route provision means through a display device and a voice guidance system in the vehicle, and has the function of conveying the route information to the user visually and audibly.

[1591] The present invention is a navigation system for autonomous vehicles that proposes the optimal route from a user's current location to their destination, and is characterized by its consideration of the user's emotional state. This system optimizes the route based on the user's emotional state to ensure a comfortable journey for the user.

[1592] System Configuration

[1593] A system for implementing the present invention includes the following major hardware and software components:

[1594] Location information acquisition method: Use a global positioning system sensor or similar to acquire latitude and longitude information of the current location.

[1595] Input method: The user inputs destination information using a touchscreen or voice input device.

[1596] Traffic information acquisition method: Use a web API (e.g., Google Maps API) to acquire real-time traffic information.

[1597] Route calculation method: An algorithm is used to calculate the optimal route based on the acquired traffic information.

[1598] Route provision means: Calculated route information is provided to the user through a display or voice guidance system.

[1599] Emotion recognition: The user's emotional state is analyzed using a camera and microphone. The analysis is performed using an emotion recognition engine such as Microsoft Azure Cognitive Services.

[1600] Route optimization means: The emotion data obtained by emotion recognition is taken into account to adjust the optimal route.

[1601] Information presentation means: Presents the route information provided by the route provision means through a display device and a voice guidance system in the vehicle.

[1602] Program processing and explanation

[1603] The whole system works as follows:

[1604] 1. Acquisition and analysis of emotion data

[1605] Cameras and microphones installed in the vehicle capture the user's facial images and voice. This data is passed to an emotion recognition engine for analysis. The analysis results identify the user's emotional state, such as whether they are in a hurry or relaxed.

[1606] 2. Data collection for route calculation

[1607] The server obtains the current location information and the destination information set by the user, and generates an API request that includes the emotional state along with the obtained current location and destination information.

[1608] 3. Obtaining traffic information and calculating the optimal route

[1609] The server retrieves real-time traffic information from a traffic information database via API requests. Traffic information includes operation status and delay information. The optimal route is calculated based on the retrieved traffic information and the user's emotional state.

[1610] 4. Providing route information

[1611] The calculated optimal route is sent from the server to the terminal inside the vehicle. The terminal analyzes the route information and displays it on the vehicle's display device. The user is then guided through the voice guidance system.

[1612] Specific examples

[1613] Below is a specific example where a user wants to go from Shibuya Station to Shinjuku Station. At this time, the user is a little impatient.

[1614] Acquiring and analyzing emotion data

[1615] A camera inside the vehicle captures the user's face, and an emotion recognition engine determines that the user is anxious.

[1616] Data collection for route calculation

[1617] The server obtains the location information of the current location (Shibuya Station) and the destination information (Shinjuku Station) entered by the user.

[1618] Obtaining traffic information and calculating the optimal route

[1619] Traffic information is obtained through a web API and the shortest route using the Keio Line is calculated.

[1620] Providing route information

[1621] The calculated route to Shinjuku Station using the Keio Line is sent to the terminal, which then displays this information on its display and announces via a voice guidance system that "If you take the Keio Line to Shinjuku Station, the journey will take approximately eight minutes."

[1622] Prompt Sentence Examples

[1623] An example of a prompt sentence is, "I'm going to Shinjuku Station. I'm currently at Shibuya Station and I'm a little anxious. Please tell me the best route."

[1624] In this way, a system can be realized that suggests optimal routes based on real-time traffic conditions while taking into account the user's emotional state.

[1625] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1626] Step 1:

[1627] The device uses cameras and microphones installed in the vehicle to capture the user's facial images and voice, which then generates data reflecting the user's emotional state. The acquired data is then sent to emotion recognition means.

[1628] Input: face image, voice data

[1629] Output: Raw data passed to the emotion recognition engine

[1630] Step 2:

[1631] The server uses an emotion recognition engine (e.g., Microsoft Azure Cognitive Services) to analyze the acquired facial images and voice data. As a result of the analysis, the user's emotional state (e.g., anxious, relaxed, etc.) is identified.

[1632] Input: face image, voice data

[1633] Output: User's emotional state (e.g., anxious, relaxed)

[1634] Step 3:

[1635] The user inputs the destination into the device. In this process, the destination information, such as "Shinjuku Station," is entered using a touch screen or voice input device. The device then sends the entered destination information along with the current location information to the server.

[1636] Input: Destination information (e.g. Shinjuku Station)

[1637] Output: Destination information is sent to the server

[1638] Step 4:

[1639] The device uses the GPS sensor to obtain the user's current location. The latitude and longitude information of the current location is acquired and sent to the server.

[1640] Input: GPS data

[1641] Output: Current location information (latitude and longitude) is sent to the server

[1642] Step 5:

[1643] The server generates API request data including the current location information, destination information, and emotional state, and sends the request to the web API for traffic information acquisition. The request includes a query to find out the current traffic situation.

[1644] Input: current location information, destination information, emotional state

[1645] Output: API request data sent

[1646] Step 6:

[1647] The server receives real-time traffic information from a web API for traffic information acquisition. The received data includes real-time traffic information such as operation status and delay information.

[1648] Input: API request data

[1649] Output: Real-time traffic information

[1650] Step 7:

[1651] The server calculates the optimal route based on traffic information and the user's emotional state. If the user is in a hurry, the shortest route will be selected, and if the user is relaxed, the route with the best scenery will be selected. The calculated route information is generated in JSON format.

[1652] Input: Traffic information, emotional state

[1653] Output: Optimal route information (JSON format)

[1654] Step 8:

[1655] The server sends the calculated optimal route information to the terminal.

[1656] Input: Optimal route information (JSON format)

[1657] Output: Route information is sent to the device

[1658] Step 9:

[1659] The device analyzes the received route information and displays it in a user-friendly format (e.g., displaying the route on the vehicle's display). It also provides route instructions to the user through a voice guidance system.

[1660] Input: Optimal route information (JSON format)

[1661] Output: The route is displayed on the display and voice guidance is provided.

[1662] In this way, an optimal route is provided based on real-time traffic information while taking into account the user's emotional state.

[1663] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1664] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1665] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1666] [Fourth embodiment]

[1667] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1668] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1669] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1670] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1671] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1672] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1673] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1674] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1675] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1676] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1677] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1678] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1679] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1680] The present invention relates to a system that suggests public transportation options in real time that will allow a user to arrive at their destination from their current location in the shortest time. This system includes a location information acquisition means, an input means, a traffic information acquisition means, a route calculation means, and a route provision means. Details of each means and the operation of the system are explained below.

[1681] 1. Obtaining location information

[1682] 1.1 Getting your current location

[1683] Device: Uses the GPS sensor to obtain current location. The device's GPS chip receives signals from satellites and obtains latitude and longitude information. This information is passed to the application.

[1684] 1.2 Entering your destination

[1685] User: Enters a destination through the application interface, for example, "Shinjuku Station."

[1686] Terminal: Passes the destination entered by the user to the application.

[1687] 2. Traffic information inquiry

[1688] 2.1 Making an API request

[1689] Device: Creates data for requesting the real-time traffic information API based on the current location and destination information, including latitude and longitude information and text information for the destination.

[1690] 2.2 Submitting a Request

[1691] Terminal: Sends the created request to the traffic information API. Sends data securely using the HTTPS protocol.

[1692] 3. Calculating the optimal route

[1693] 3.1 Receiving a Request

[1694] Server: Receives requests from the terminal via the traffic information API.

[1695] 3.2 Querying the Database

[1696] Server: Based on the received request, queries the transport company's database, which contains information on timetables, operation status, delays, etc.

[1697] 3.3 Obtaining real-time information

[1698] Server: Obtain real-time information on transportation status, such as delays on the Yamanote Line.

[1699] 3.4 Optimal Route Calculation

[1700] Server: Based on the acquired information, the server calculates the optimal route from the current location to the destination. The calculation takes into account the travel time, number of transfers, and whether or not there are any delays. Specifically, route search algorithms such as the Dijkstra algorithm and the A algorithm are used.

[1701] 4. Route provision

[1702] 4.1 Generating calculation results

[1703] Server: Generates the results of the calculation in a data format such as JSON, which contains the following information:

[1704] Departure and arrival points

[1705] Route name to be used

[1706] How to transfer

[1707] Estimated arrival time

[1708] 4.2 Sending a Response

[1709] Server: Sends the calculation results to the terminal.

[1710] 4.3 Receiving and Displaying Data

[1711] Terminal: Parses the calculation results received from the server and displays them in a user-friendly format. Route guidance, timetables, transfer instructions, etc. are displayed on the screen.

[1712] 4.4 User Visibility

[1713] User: Looks at the device screen to see the optimal route displayed, and receives instructions on which train or bus to take next, where to transfer, etc.

[1714] Example: From Shibuya Station to Shinjuku Station

[1715] 1. Obtaining location information

[1716] Device: Obtain current location (Shibuya Station) using GPS.

[1717] User: Enter "Shinjuku Station" in the app's search box.

[1718] 2. Traffic information inquiry

[1719] Terminal: Generate API request data including latitude and longitude information (Shibuya Station) and destination "Shinjuku Station".

[1720] Terminal: Sends the generated request to the API.

[1721] 3. Calculating the optimal route

[1722] Server: Receives API requests and sends queries to the database.

[1723] Server: Obtain information about delays on the Yamanote Line.

[1724] Server: Calculates a route that uses the Keio Line instead of the Yamanote Line.

[1725] 4. Route provision

[1726] Server: Generate calculation results (Shibuya Station to Shinjuku Station, using the Keio Line) in JSON format.

[1727] Server: Sends the calculation results to the terminal.

[1728] Terminal: Parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take approximately 8 minutes."

[1729] User: Follow the instructions displayed and board the Keio Line.

[1730] In this way, the user can receive optimal route instructions utilizing real-time information. The specific processing steps will be explained separately.

[1731] The processing flow will be explained below.

[1732] Step 1:

[1733] The device uses the GPS sensor to obtain the latitude and longitude of the current location, and passes this information to the application.

[1734] Step 2:

[1735] The user uses the application interface to enter a destination, for example, "Shinjuku Station."

[1736] Step 3:

[1737] The device creates request data based on the current location information acquired and the destination entered by the user. This data includes the latitude and longitude of the current location and the name of the destination.

[1738] Step 4:

[1739] The request data created by the device is sent to the traffic information acquisition API. The data is sent securely using protocols such as HTTPS.

[1740] Step 5:

[1741] The server receives the request sent from the terminal.

[1742] Step 6:

[1743] Based on the request received by the server, the server queries the transport company database to obtain information such as transport schedules, operation status, and delay information.

[1744] Step 7:

[1745] The server obtains real-time information about the operation status of public transport, such as delays on the Yamanote Line.

[1746] Step 8:

[1747] Based on the information acquired by the server, the optimal route from the current location to the destination is calculated, taking into account the required time, number of transfers, and information on train delays.

[1748] Step 9:

[1749] The server generates the calculation results in data format such as JSON, which includes the departure point, arrival point, route name, transfer method, estimated arrival time, etc.

[1750] Step 10:

[1751] The server sends the generated calculation results to the terminal.

[1752] Step 11:

[1753] The terminal receives the calculation results sent from the server, parses the received data, and displays it in a format that is easy for the user to view.

[1754] Step 12:

[1755] The user looks at the device screen to see the optimal route displayed, and then receives instructions on which public transport to use next, where to transfer, etc.

[1756] Example 1

[1757] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1758] Currently, when users reach their destination using public transportation, it is difficult to determine the optimal route that takes into account real-time traffic information and service delays. Furthermore, when transferring between multiple modes of transportation, there are few systems that provide integrated transfer information, which often makes users' travel complicated. Therefore, there is a need for a system that instantly provides the optimal route based on real-time information.

[1759] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1760] In this invention, the server includes a location information acquisition means for acquiring the current location, an input means for receiving input information about the destination, and a means for acquiring traffic information in real time, thereby making it possible to provide an optimal route taking into account real-time traffic conditions.

[1761] "Location information acquisition means" refers to devices or functions for acquiring the user's current location.

[1762] "Input means" refers to an interface or device for receiving destination-related information from a user.

[1763] "Traffic information acquisition means" refers to functions and devices for acquiring traffic information in real time.

[1764] "Optimal route calculation means" refers to a function or device for calculating the optimal travel route based on acquired traffic information and transfer information.

[1765] "Route providing means" refers to a means for providing information on the calculated optimal route to a user.

[1766] "Satellite Positioning System Sensor" means a positioning system that uses signals from satellites to determine location on Earth.

[1767] "Web Application Programming Interface" refers to a means for communicating with external software or applications through web services.

[1768] "Transportation information" refers to information such as public transportation schedules, operation status, and delay information.

[1769] The present invention relates to a system that suggests public transportation options in real time that will allow a user to arrive at their destination from their current location in the shortest time. This system is composed of the following means, including location information acquisition means, input means, traffic information acquisition means, optimal route calculation means, and route provision means.

[1770] Obtaining location information

[1771] Get current location

[1772] Device: The GPS sensor is used to obtain the user's current location. The GPS chip in the device receives signals from satellites, extracts latitude and longitude information, and passes it to the application.

[1773] Enter your destination

[1774] User: Enters a destination through the application interface. For example, the user enters "Shinjuku Station."

[1775] Terminal: Passes the destination information entered by the user to the application.

[1776] Traffic information inquiry

[1777] Making API Requests

[1778] Device: Creates request data for the real-time traffic information API based on the current location and destination information. This data includes the acquired latitude and longitude information and the destination information entered by the user.

[1779] Submitting a Request

[1780] Terminal: Sends the created request data to the API using the HTTPS protocol.

[1781] Calculating the best route

[1782] Receiving a request

[1783] Server: Queries the transit agency database based on requests received via the API.

[1784] Server: The database contains information on transportation schedules, operation status, delays, etc.

[1785] Obtaining real-time information

[1786] Server: Obtains real-time information about the operation of public transport. For example, obtains information such as "The Yamanote Line is currently delayed by 5 minutes."

[1787] Calculating the best route

[1788] Server: Based on the acquired data, the server calculates the optimal route from the user's current location to the destination. The calculation takes into account the travel time, number of transfers, and real-time delay information. The specific algorithms used for route search are the Dijkstra algorithm and the A algorithm.

[1789] Route provision

[1790] Generate calculation results

[1791] Server: Generates the optimal route calculation results in data format such as JSON. This data includes information such as the departure point, arrival point, route name, transfer method, and estimated arrival time.

[1792] Sending a response

[1793] Server: Sends the generated calculation results to the terminal.

[1794] Receiving and displaying data

[1795] Terminal: Parses the calculation results received from the server and displays them in a user-friendly format. For example, route guidance, timetables, transfer procedures, etc. are displayed on the screen.

[1796] User Visibility

[1797] User: Check the information displayed on the device and follow the instructions to begin traveling. For example, follow the instructions such as "Take the Keio Line to Shinjuku Station, it will take about 8 minutes."

[1798] Example: From Shibuya Station to Shinjuku Station

[1799] Get current location:

[1800] Device: Obtain current location (Shibuya Station) using GPS.

[1801] Enter your destination:

[1802] User: Enters "Shinjuku Station" into the app's search box.

[1803] Terminal: Obtain the entered destination information.

[1804] Make an API request:

[1805] Terminal: Generate API request data including latitude and longitude information (Shibuya Station) and destination "Shinjuku Station".

[1806] Sending an API request:

[1807] Terminal: Sends the generated request data to the API.

[1808] Query the database:

[1809] Server: Receives API requests and sends queries to the transit agency database.

[1810] Get real-time information:

[1811] Server: Obtain delay information for the Yamanote Line.

[1812] Calculating the best route:

[1813] Server: Calculate a route that avoids delays on the Yamanote Line and uses the Keio Line.

[1814] Generate calculation results:

[1815] Server: Generate the calculation result (Shibuya Station to Shinjuku Station, using the Keio Line) in JSON format.

[1816] Sending a response:

[1817] Server: Sends the calculation results to the terminal.

[1818] Receiving and displaying data:

[1819] Terminal: Parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take approximately 8 minutes."

[1820] Check the results and follow the instructions:

[1821] User: Follow the displayed instructions and use the Keio Line.

[1822] Example prompt sentence:

[1823] "I'm currently at Shibuya Station and my destination is Shinjuku Station. Please tell me the best route taking real-time traffic information into account."

[1824] This system allows users to receive optimal route instructions based on real-time information, enabling them to use public transportation efficiently.

[1825] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1826] Step 1: Obtaining location information

[1827] Device: The device uses a GPS sensor to obtain the user's current location. Specifically, the device's GPS chip receives signals from satellites and generates latitude and longitude information. This location information is passed to the application. The input is the signal from the GPS, and the output is the user's latitude and longitude information.

[1828] Step 2: Enter your destination

[1829] User: The user inputs a destination through the application interface. For example, they input "Shinjuku Station." The input is text input by the user, and the output is the destination information "Shinjuku Station."

[1830] Terminal: The terminal passes the destination information entered by the user to the application. The input is text data from the user, and the output is the destination information passed to the application.

[1831] Step 3: Making an API Request

[1832] Terminal: The terminal creates request data for the real-time traffic information API based on the current location (latitude and longitude information) and destination information. Specifically, this information is combined and placed in a request format. The input is latitude and longitude information and destination information, and the output is the API request data.

[1833] Step 4: Sending an API Request

[1834] Terminal: The terminal uses the HTTPS protocol to send the created request data to the API. The input is the API request data, and the output is communication to the API server.

[1835] Step 5: Receiving the request

[1836] Server: The API server receives the request data sent from the terminal. The input is the request data, and the output is a confirmation of the received request data.

[1837] Step 6: Query the database

[1838] Server: The server queries the traffic information database based on the received request. Specifically, it searches for schedule information, operation status, and delay information. The input is the request data, and the output is the traffic information retrieved from the database.

[1839] Step 7: Get real-time information

[1840] Server: The server obtains real-time operational status of transportation facilities. For example, it obtains information such as "The Yamanote Line is currently delayed by 5 minutes." The input is real-time data of transportation facilities, and the output is the latest operational status.

[1841] Step 8: Calculate the optimal route

[1842] Server: The server calculates the optimal route from the current location to the destination based on the acquired data. It uses the Dijkstra algorithm and the A algorithm to take into account the travel time, number of transfers, and real-time delay information. The input is traffic information and real-time operation status, and the output is the optimal route information.

[1843] Step 9: Generate calculation results

[1844] Server: The server generates the optimal route calculation results in data format such as JSON. This data includes the departure point, arrival point, route name to be used, transfer method, and estimated arrival time. The input is the optimal route information, and the output is data in JSON format.

[1845] Step 10: Sending the Response

[1846] Server: The server sends the generated calculation results to the terminal. The input is JSON format data, and the output is the response to the terminal.

[1847] Step 11: Receiving and displaying data

[1848] Terminal: The terminal parses the calculation results received from the server and displays them in a user-friendly format. Route guidance, timetables, and transfer procedures are displayed on the screen. The input is the response data from the server, and the output is the information displayed on the user interface.

[1849] Step 12: Review the results and follow the instructions

[1850] User: The user checks the information displayed on the device and starts moving according to the instructions. For example, they follow instructions such as "Take the Keio Line to Shinjuku Station, it will take about 8 minutes." The input is the information displayed on the device, and the output is the user's actions.

[1851] (Application example 1)

[1852] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1853] While many conventional public transportation guidance systems provide the optimal route from a user's current location to their destination, they are inadequate when it comes to providing destination guidance within shopping malls and large facilities. Providing adequate guidance is particularly difficult when a user has multiple destinations or when congestion needs to be taken into consideration. There is also a growing need for intuitive operation using voice input in addition to visual guidance information. The present invention aims to solve these problems by providing a system that provides real-time guidance on the shortest route to a destination within a shopping mall.

[1854] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1855] In this invention, the server includes location information acquisition means for acquiring location information, input means for receiving input information related to a destination, information acquisition means for acquiring traffic information in real time, route calculation means for calculating an optimal route, route provision means for providing the optimal route to a user, display means for visually guiding the optimal route based on real-time congestion status, and voice input means for inputting a destination using voice recognition, thereby making it possible to provide the shortest route to a destination within a shopping mall in real time by visual and voice input.

[1856] "Location information acquisition means" refers to a device or system for acquiring the user's current location, and utilizes a satellite system or GPS sensor.

[1857] "Input means" refers to a device or system for receiving information about a destination from a user, and includes a touch panel, a keyboard, a voice recognition system, and the like.

[1858] The "information acquisition means" is a device or system for acquiring data such as traffic information and congestion status in real time, and acquires data via a network.

[1859] The "route calculation means" is a device or system for calculating the optimum route based on the acquired information, and uses a route search algorithm that takes into account the required travel time and traffic congestion.

[1860] "Route providing means" refers to a device or system for providing the user with the calculated optimum route, and conveys information through a screen display, voice guidance, etc.

[1861] "Display means" refers to a device or system that visually guides passengers along optimal routes based on real-time congestion information, and uses smart glasses or a head-mounted display.

[1862] "Voice input means" refers to a device or system that utilizes voice recognition technology to receive destination information from a user.

[1863] The embodiment of the present invention is directed to constructing a system that guides users so that they can reach their destination from their current location in the shortest time possible. This system is particularly effective for providing guidance within large facilities such as shopping malls. Detailed embodiments of this system are described below.

[1864] 1. System Configuration

[1865] The system consists of the following major components:

[1866] Location information acquisition means

[1867] Input Method

[1868] Information acquisition means

[1869] Route calculation method

[1870] Route provision method

[1871] Display means

[1872] Voice input means

[1873] Location information acquisition means

[1874] Location information acquisition means has the function of acquiring the user's current location. It mainly uses satellite systems and GPS sensors to acquire the current latitude and longitude information. For example, a GPS chip installed in smart glasses measures the user's location in real time.

[1875] Input Method

[1876] The input means has the function of receiving information about the destination from the user. This can be done using a touch panel, keyboard, or voice recognition system. Specifically, it is possible to use the voice recognition system built into the smart glasses to input destinations such as "supermarket" or "toilet" by voice.

[1877] Information acquisition means

[1878] The information acquisition means has the function of acquiring data such as traffic information and congestion status in real time. This data is acquired via the network, for example, using the Google Maps API or a shopping mall's dedicated API.

[1879] Route calculation method

[1880] The route calculation means has the function of calculating the optimal route based on the acquired information. For this, a route search algorithm (such as the Dijkstra algorithm or the A algorithm) is used that takes into account the required time and congestion status.

[1881] Route provision method

[1882] The route providing means has the function of providing the user with the calculated optimal route, and displays visual guidance information on the smart glasses display or the smartphone screen.

[1883] Display means

[1884] The display function visually guides users to the optimal route based on real-time congestion information, primarily using smart glasses or head-mounted displays to visually show the user the next direction to go and the route to their destination.

[1885] Voice input means

[1886] The voice input means has the function of receiving destination information from the user using voice recognition technology, using a microphone and voice recognition software built into the smart glasses.

[1887] Program processing explanation

[1888] This system works in conjunction with each mode of transportation to provide users with optimal route guidance. The specific processing flow is explained below.

[1889] 1. Location information acquisition means

[1890] It uses a GPS sensor to obtain the user's current location and transmits that information to a server. For example, the GPS chip in smart glasses measures the user's location in real time and passes the latitude and longitude data to the application.

[1891] 2. Input Method

[1892] The user inputs their destination through a voice input system, for example by saying "I'm going to the supermarket," and the voice recognition software recognizes the destination and sends that information to the server.

[1893] 3. Information acquisition means

[1894] The server makes a request to a real-time congestion information API based on the acquired current location and destination information. Specifically, it accesses the Google Maps API or a congestion information API dedicated to shopping malls.

[1895] 4. Route calculation method

[1896] The server calculates the shortest route using the Dijkstra algorithm or A algorithm based on the acquired congestion information. For example, it calculates a route that avoids congested areas.

[1897] 5. Route provision method

[1898] The server sends the calculation results in JSON format to the smart glasses, which then display arrows and text instructions to the user based on the route information received.

[1899] Examples of concrete examples and prompts

[1900] Hardware / Software used

[1901] Reference Hardware: Smart glasses, e.g. Vuzix Blade or North Focals

[1902] Real-time location information acquisition API: Google Maps API

[1903] Server: AWS EC2

[1904] Real-time congestion information API: SmartMall API (tentative)

[1905] Prompt Sentence Examples

[1906] Generative AI model used: GPT-4

[1907] Example prompt: "Code the process flow for a smart glasses app that provides visual guidance on the shortest route from a user's current location to a destination. The destination is specified via voice input, and the route display takes real-time information into account."

[1908] This makes it possible to provide the shortest route to a destination within a shopping mall in real time through visual and voice input.

[1909] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1910] Step 1:

[1911] Get current location

[1912] Subject: Device

[1913] What it does: Uses the device's GPS sensor to obtain the user's current location.

[1914] Input: The device's GPS chip receives signals from satellites and obtains latitude and longitude information.

[1915] Output: Latitude and longitude information (e.g., current location coordinates). The device passes this information to the next processing step.

[1916] Step 2:

[1917] Enter your destination

[1918] Subject: User

[1919] How it works: The user enters destination information through the voice recognition system.

[1920] Input: The user's spoken command (e.g., "Go to the supermarket").

[1921] Output: Textual destination information (e.g., destination name) is generated and passed to the terminal.

[1922] Step 3:

[1923] Obtaining real-time information

[1924] Subject: Device

[1925] How it works: The device sends a request to a real-time congestion information API based on its current location and destination.

[1926] Input: Latitude and longitude information of current location, text information of destination.

[1927] Output: Real-time congestion data (e.g., congestion and traffic information obtained from an API) is returned to the device.

[1928] Step 4:

[1929] Calculating the best route

[1930] Subject: Server

[1931] How it works: The server calculates the optimal route based on the real-time data it receives.

[1932] Input: current location latitude and longitude, destination text, real-time congestion data.

[1933] Data processing / calculation: Using a path-finding algorithm (e.g., Dijkstra algorithm or A algorithm), calculate the route that can be reached in the shortest time.

[1934] Output: The calculated optimal route (e.g., a list of route information) is generated.

[1935] Step 5:

[1936] Route information format conversion

[1937] Subject: Server

[1938] How it works: The server converts the optimal route information into a data format such as JSON.

[1939] Input: The calculated optimal route information.

[1940] Output: Route information (e.g., navigation data) in JSON format is generated and sent to the device.

[1941] Step 6:

[1942] View route information

[1943] Subject: Device

[1944] How it works: The device parses the route information it receives and displays it visually to the user.

[1945] Input: Route information in JSON format.

[1946] Data processing / computation: The device parses the JSON data and generates a visual navigation display.

[1947] Output: Route guidance is displayed on the smart glasses display (e.g., arrows and text instructions showing the direction of travel).

[1948] Step 7:

[1949] User movement guidance

[1950] Subject: User

[1951] How it works: The user follows the guidance displayed on the smart glasses to reach their destination.

[1952] Input: Visual navigation information (e.g., arrows and text instructions).

[1953] Output: The user arrives at the destination.

[1954] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1955] The present invention combines an emotion engine with a system that suggests in real time the public transportation that will take a user from their current location to their destination in the shortest time. This system includes a location information acquisition means, an input means, a traffic information acquisition means, a route calculation means, a route provision means, and an emotion recognition means. Details of each means and the operation of the system are described below.

[1956] 1. Obtaining location information

[1957] 1.1 Getting your current location

[1958] The device uses the GPS sensor to obtain the latitude and longitude of the current location, and passes this information to the application.

[1959] 1.2 Entering your destination

[1960] The user uses the application interface to enter a destination, for example, "Shinjuku Station."

[1961] The device passes the destination entered by the user to the application.

[1962] 2. Emotional Recognition

[1963] 2.1 Acquiring Emotion Data

[1964] The device uses a microphone and camera to capture the user's voice and facial images.

[1965] The terminal passes the voice and image data to the emotion recognition means.

[1966] 2.2 Emotion Data Analysis

[1967] The server analyzes the received audio or image data to recognize the user's emotions. For example, it can identify whether the user is anxious, impatient, calm, etc. based on changes in voice tone or facial expression.

[1968] 3. Traffic information inquiry

[1969] 3.1 Making an API request

[1970] Request data is created based on the current location information acquired by the device, the destination entered by the user, and the recognized emotion data.

[1971] 3.2 Sending a Request

[1972] The request data created by the device is sent to the traffic information acquisition API. The data is sent securely using protocols such as HTTPS.

[1973] 4. Calculating the optimal route

[1974] 4.1 Receiving a Request

[1975] The server receives the request sent from the terminal.

[1976] 4.2 Querying the Database

[1977] Based on the request received by the server, the server queries the transport company database to obtain information such as transport schedules, operation status, and delay information.

[1978] 4.3 Obtaining real-time information

[1979] The server obtains real-time information about the operation status of public transport, such as delays on the Yamanote Line.

[1980] 4.4 Optimal Route Calculation

[1981] The server calculates the optimal route from the current location to the destination based on the information acquired and the user's emotional state. The calculation takes into account the required travel time, number of transfers, and service delay information, as well as the user's emotional state. Specifically, if the user is in a hurry, the system will choose a route with fewer transfers, and if the user has time, it will choose a route with better scenery.

[1982] 5. Route provision

[1983] 5.1 Generating Calculation Results

[1984] The server generates the calculation results in data format such as JSON, which includes the departure point, arrival point, route name, transfer method, and estimated arrival time.

[1985] 5.2 Sending a Response

[1986] The server sends the generated calculation results to the terminal.

[1987] 5.3 Receiving and Displaying Data

[1988] The terminal receives the calculation results sent from the server, parses the received data, and displays it in a format that is easy for the user to view.

[1989] 5.4 User Visibility

[1990] The user looks at the device screen to see the optimal route displayed, and then receives instructions on which public transport to use next, where to transfer, etc.

[1991] Example: From Shibuya Station to Shinjuku Station

[1992] 1. Obtaining location information

[1993] Device: Obtain current location (Shibuya Station) using GPS.

[1994] User: Enter "Shinjuku Station" in the app's search box.

[1995] 2. Emotional Recognition

[1996] Device: Captures your face and voice with a camera or microphone.

[1997] Server: Recognizes the user's emotions using the emotion engine. For example, it detects that the user is in a hurry.

[1998] 3. Traffic information inquiry

[1999] Device: Generate API request data including latitude and longitude information (Shibuya Station), destination "Shinjuku Station", and emotion data.

[2000] Terminal: Sends the generated request to the API.

[2001] 4. Calculating the optimal route

[2002] Server: Receives API requests and sends queries to the database.

[2003] Server: Obtain information about delays on the Yamanote Line.

[2004] Server: Because the user is in a hurry, calculate a route using the Keio Line with fewer transfers.

[2005] 5. Route provision

[2006] Server: Generate calculation results (Shibuya Station to Shinjuku Station, using the Keio Line) in JSON format.

[2007] Server: Sends the calculation results to the terminal.

[2008] Terminal: Parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take approximately 8 minutes."

[2009] User: Follow the instructions displayed and board the Keio Line.

[2010] In this way, the user can receive optimal route instructions that utilize real-time information and take into account their emotional state. The specific processing steps will be described later.

[2011] The processing flow will be explained below.

[2012] Step 1:

[2013] The device uses the GPS sensor to obtain the latitude and longitude of the current location, and passes this information to the application.

[2014] Step 2:

[2015] The user uses the application interface to enter a destination, for example, "Shinjuku Station."

[2016] Step 3:

[2017] The device uses a microphone or camera to capture the user's voice and facial images, and passes this emotional data to the application.

[2018] Step 4:

[2019] The request data is created based on the current location information acquired by the device, the destination entered by the user, and emotion data. This data includes latitude and longitude information, the name of the destination, and emotion data.

[2020] Step 5:

[2021] The request data created by the device is sent to the traffic information acquisition API. The data is sent securely using the HTTPS protocol.

[2022] Step 6:

[2023] The server receives the request sent from the terminal.

[2024] Step 7:

[2025] Based on the request received by the server, the server queries the transport company database to obtain information such as transport schedules, operation status, and delay information.

[2026] Step 8:

[2027] The server obtains real-time information about the operation status of public transport, such as delays on the Yamanote Line.

[2028] Step 9:

[2029] The server uses an emotion engine to analyze the user's emotional information, such as whether the user is in a hurry or relaxed, based on changes in voice tone and facial expressions.

[2030] Step 10:

[2031] The server calculates the optimal route from the current location to the destination based on traffic information, timetable information, operation status, and user emotion information, taking into account travel time, number of transfers, operation delays, and route adjustments according to the user's emotion.

[2032] Step 11:

[2033] The server generates the calculation results in data format such as JSON, which includes the departure point, arrival point, route name, transfer method, and estimated arrival time.

[2034] Step 12:

[2035] The server sends the generated calculation results to the terminal.

[2036] Step 13:

[2037] The terminal receives the calculation results sent from the server, parses the received data, and displays it in a format that is easy for the user to view.

[2038] Step 14:

[2039] The user looks at the device screen to see the optimal route displayed, and then receives instructions on which public transport to use next, where to transfer, etc.

[2040] Example 2

[2041] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2042] Conventional public transport guidance systems provide the optimal route based on the user's current location and destination, but they do not take into account the user's emotional state, making it difficult to respond flexibly to the user's situation. Furthermore, simply providing the shortest route does not necessarily provide the optimal method for the user, as it cannot respond to sudden delays or congestion.

[2043] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: location information acquisition means for acquiring a current location; input means for receiving input information related to a destination; information acquisition means for acquiring traffic information in real time based on the current location acquired by the location information acquisition means and the destination received by the input means; calculation means for calculating an optimal route based on the traffic information acquired by the information acquisition means; emotion recognition means for acquiring a user's voice or image data and recognizing emotions; calculation means for calculating an optimal route taking into account the user's emotion recognized by the emotion recognition means; and provision means for providing the user with the optimal route calculated by the calculation means. This enables a more flexible and adaptable optimal route to be proposed taking into account the user's emotional state. Furthermore, by acquiring and analyzing traffic information in real time, a route that can accommodate unexpected service delays and congestion can be provided.

[2044] "Location information acquisition means" refers to a device or function for acquiring the user's current location using a satellite positioning system sensor.

[2045] The "input means" is an interface through which the user inputs information about the destination.

[2046] The "information acquisition means" is a device or function for acquiring traffic information in real time based on the current location information acquired by the position information acquisition means and the destination information received by the input means.

[2047] The "calculation means" is a device or function for calculating an optimum route based on the traffic information acquired by the information acquisition means.

[2048] The "emotion recognition means" is a device or function for acquiring the user's voice and image data, analyzing them, and recognizing the user's emotions.

[2049] "Providing means" refers to a device or function for providing the calculated optimum route to the user.

[2050] "User's emotional state" is the user's mental or emotional state recognized by the emotion recognition means.

[2051] A "satellite positioning system sensor" is a sensor that acquires location information using a global positioning system (GPS) or similar.

[2052] A "data communication interface" is a device or function for obtaining external information using a communication protocol.

[2053] "Getting traffic information in real time" means getting current traffic conditions and operation information instantly.

[2054] The present invention is a system that suggests public transportation options in real time that will get a user from their current location to their destination in the shortest time, and also combines an emotion engine. This system includes a location information acquisition means, an input means, an information acquisition means, a calculation means, an emotion recognition means, and a provision means. Details of each means and the operation of the system are explained below.

[2055] Specific hardware and software

[2056] Location information acquisition means: A device that acquires the user's current location using a satellite positioning system (GPS) sensor.

[2057] Input method: The interface of the application through which the user enters their destination.

[2058] Information acquisition means: A device for acquiring traffic information through a data communication interface.

[2059] Calculation means: The calculation module in the server calculates the optimal route based on the data obtained from the information acquisition means.

[2060] Emotion recognition means: An emotion recognition module that acquires and analyzes the user's voice and image data.

[2061] Provision means: A terminal display and application for providing the user with the optimal route calculated by the calculation means.

[2062] System Operation

[2063] First, the device uses the GPS sensor to acquire the user's current location. The user inputs the destination through the application interface and the information is sent to the system. Then, the location information acquisition means identifies the user's current location, and the input means receives the destination information.

[2064] Next, the device captures the user's voice and facial images using a microphone and camera, and passes them to the emotion recognition means. The emotion recognition means runs on the server and analyzes the received voice and image data to recognize the user's emotions. As a result of the analysis, the user's emotional state, such as impatience, anxiety, or calmness, is identified.

[2065] Based on these data, the terminal compiles the current location information, destination information, and the user's emotional state into one request data, and acquires real-time traffic information through the information acquisition means. This data acquisition is performed via the data communication interface.

[2066] The server receives the request data sent from the information acquisition means and queries an internal database to obtain information on the operation status of public transport, delay information, transfer information, etc. It also obtains the latest public transport information using an external Web API. Based on this data, the calculation means calculates the optimal route.

[2067] In the calculation, the server takes into account the user's emotional state. For example, if the user is in a hurry, it will prioritize routes with fewer transfers. On the other hand, if the user has time, it will suggest scenic or comfortable routes.

[2068] The calculation results are sent from the server to the terminal and provided to the user. The application on the terminal displays the calculation results to the user through the means of provision, and gives specific instructions such as "If you take the Keio Line to Shinjuku Station, the journey will take about 8 minutes."

[2069] Specific examples

[2070] For example, consider a case where a user wants to travel from Shibuya Station to Shinjuku Station. First, the device's GPS acquires the current location, Shibuya Station, and the user enters "Shinjuku Station" into the application. Next, the device uses the camera and microphone to capture the user's facial expressions and voice, and the server recognizes the emotion of "I'm in a hurry."

[2071] The device then generates a request containing the current location (Shibuya Station), the destination (Shinjuku Station), and the user's emotion data, and sends it to the traffic information API. The server receives the request and queries the database and external APIs to collect traffic information. If the Yamanote Line is delayed, it is perceived that the user is in a hurry, and it is determined that the Keio Line, which has fewer transfers, is the optimal route.

[2072] Finally, the server generates the calculation result in JSON format and sends it to the terminal. The terminal parses the received result and displays to the user, "If you take the Keio Line to Shinjuku Station, the journey will take about 8 minutes." The user follows this and boards the Keio Line to Shinjuku Station.

[2073] Prompt Sentence Examples

[2074] Examples of prompts for generating emotion recognition and route calculation instructions using generative AI models are as follows:

[2075] "Please explain how a system that suggests the public transport option that will get a user from their current location to their destination in real time works by combining it with an emotion engine. Specifically, please explain how the current location is obtained using GPS, the user inputs their destination, emotion recognition is performed, the procedure for collecting traffic information, how the optimal route is calculated, and the process for providing the route to the user."

[2076] The above is a specific embodiment of the present invention. This system provides users with real-time recommendations for optimal public transport routes, and even tailored to their individual emotional state, greatly improving convenience.

[2077] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2078] Step 1: Get your current location

[2079] The device uses the GPS sensor to obtain the latitude and longitude of the user's current location, which is then sent to the application backend.

[2080] Input: Location data from the GPS sensor

[2081] Data processing: Convert location data into application format

[2082] Output: Latitude and longitude information (e.g., latitude: 35.6581, longitude: 139.7414)

[2083] Step 2: Enter your destination

[2084] The user enters a destination into the application's search box, and the device sends the entered data to the application's backend.

[2085] Input: Destination information entered by the user (text format)

[2086] Data processing: Save the entered text as destination information

[2087] Output: Destination information (e.g., destination: "Shinjuku Station")

[2088] Step 3: Obtaining emotion data

[2089] The device uses a microphone and camera to capture the user's voice and facial images, and the captured data is passed to the emotion recognition means.

[2090] Input: Audio and image data obtained from microphones and cameras

[2091] Data processing: Converting audio and image data into a format that can be analyzed by emotion recognition algorithms

[2092] Output: Audio data and image data (e.g., image_data: [binary data], audio_data: [binary data])

[2093] Step 4: Analyze the sentiment data

[2094] The server analyzes the acquired voice and image data to recognize the user's emotions. This analysis is performed by the emotion recognition module.

[2095] Input: Audio and image data

[2096] Data processing: Analyzing voice tone and facial expression changes

[2097] Output: User's emotional state (e.g. emotion: "anxious")

[2098] Step 5: Check traffic information

[2099] The terminal creates request data based on the current location, destination, and emotion data, and acquires traffic information in real time through the information acquisition means.

[2100] Input: current location information, destination information, emotional state

[2101] Data processing: Assemble these data into a single request object

[2102] Output: Request data (e.g., { currentLocation: {latitude: 35.6581, longitude: 139.7414}, destination: "Shinjuku Station", emotion: "Impatience"})

[2103] Step 6: Obtaining real-time traffic information

[2104] The terminal sends the request data to the traffic information acquisition API to obtain real-time traffic information.

[2105] Input: Request data

[2106] Data processing: Communication with external traffic information databases via API

[2107] Output: Real-time traffic information (e.g. { line: "Yamanote Line", status: "Delay"})

[2108] Step 7: Calculate the optimal route

[2109] The optimal route is calculated based on real-time traffic information obtained by the server and the user's emotional state.

[2110] Input: Real-time traffic information, emotional state, current location information, destination information

[2111] Data processing: This information is input into an algorithm to calculate the optimal route.

[2112] Output: Optimal route data (e.g., Optimal route calculated: Shibuya Station → Keio Line → Shinjuku Station)

[2113] Step 8: Generate route calculation results

[2114] The server generates the optimal route calculation results in JSON format and sends them to the terminal.

[2115] Input: Optimal route data

[2116] Data processing: Convert calculation results into JSON format

[2117] Output: Calculation result in JSON format (e.g., {"origin": "Shibuya Station", "destination": "Shinjuku Station", "route": "Keio Line", "estimated_time": "8 minutes"})

[2118] Step 9: Receive and display the calculation results

[2119] The terminal receives the calculation results sent from the server, analyzes them, and provides them to the user.

[2120] Input: Calculation result in JSON format

[2121] Data processing: Parse JSON data and format it for display

[2122] Output: Display data (e.g. "The journey to Shinjuku Station on the Keio Line takes approximately 8 minutes.")

[2123] Step 10: Deliver to users

[2124] The device displays the formatted calculation results to the user and gives instructions on what to do next.

[2125] Input: Display data

[2126] Data processing: Display on screen

[2127] Output: Displayed instructions (e.g., the user looks at the device and performs the action "take the Keio Line")

[2128] (Application example 2)

[2129] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2130] Conventional navigation systems provide an optimal route based on a user's current location and destination, but do not consider the user's emotional state. As a result, they are unable to provide an appropriate route based on the user's emotional state, such as when the user is in a hurry or relaxed. For example, a user may want to be presented with the shortest route when in a hurry, or with a scenic route when relaxed. The present invention aims to realize a more comfortable and stress-free journey by providing route suggestions based on the user's emotional state.

[2131] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2132] In this invention, the server includes emotion recognition means for analyzing the emotional state of the user, route optimization means for adjusting an optimal route taking into consideration the emotional state of the user acquired by the emotion recognition means, and information presentation means for presenting the route information provided by the route provision means through an in-vehicle display device and voice guidance system, thereby making it possible to propose an optimal route based on the emotional state of the user.

[2133] "Location information acquisition means" is a device for acquiring latitude and longitude information of the current location. It mainly has the function of identifying the current location using a global positioning system sensor, etc.

[2134] An "input means" is a device through which a user inputs destinations and other information. This typically includes a touchscreen or voice input device.

[2135] "Traffic information acquisition means" refers to a device for acquiring traffic conditions and operation information in real time. This includes methods of acquiring information via web APIs.

[2136] The "route calculation means" is a device that calculates the optimal route based on the current location, destination, and traffic information. It has the function of using an algorithm to calculate the optimal route based on conditions such as the shortest time and the fewest transfers.

[2137] "Route providing means" refers to a device that provides optimal route information to users. It has the function of displaying or notifying route information through a display, voice guidance system, etc.

[2138] The "emotion recognition means" is a device for analyzing the user's emotional state. It has the function of analyzing the user's emotions based on data acquired using a camera or microphone.

[2139] The "route optimization means" is a device that adjusts the optimal route taking into consideration the emotional state of the user acquired by the emotion recognition means. For example, it has the function of suggesting the shortest route if the user is in a hurry, or the scenic route if the user is relaxed, depending on the user's emotions.

[2140] The "information presentation means" is a device for presenting the route information provided by the route provision means through a display device and a voice guidance system in the vehicle, and has the function of conveying the route information to the user visually and audibly.

[2141] The present invention is a navigation system for autonomous vehicles that proposes the optimal route from a user's current location to their destination, and is characterized by its consideration of the user's emotional state. This system optimizes the route based on the user's emotional state to ensure a comfortable journey for the user.

[2142] System Configuration

[2143] A system for implementing the present invention includes the following major hardware and software components:

[2144] Location information acquisition method: Use a global positioning system sensor or similar to acquire latitude and longitude information of the current location.

[2145] Input method: The user inputs destination information using a touchscreen or voice input device.

[2146] Traffic information acquisition method: Use a web API (e.g., Google Maps API) to acquire real-time traffic information.

[2147] Route calculation method: An algorithm is used to calculate the optimal route based on the acquired traffic information.

[2148] Route provision means: Calculated route information is provided to the user through a display or voice guidance system.

[2149] Emotion recognition: The user's emotional state is analyzed using a camera and microphone. The analysis is performed using an emotion recognition engine such as Microsoft Azure Cognitive Services.

[2150] Route optimization means: The emotion data obtained by emotion recognition is taken into account to adjust the optimal route.

[2151] Information presentation means: Presents the route information provided by the route provision means through a display device and a voice guidance system in the vehicle.

[2152] Program processing and explanation

[2153] The whole system works as follows:

[2154] 1. Acquisition and analysis of emotion data

[2155] Cameras and microphones installed in the vehicle capture the user's facial images and voice. This data is passed to an emotion recognition engine for analysis. The analysis results identify the user's emotional state, such as whether they are in a hurry or relaxed.

[2156] 2. Data collection for route calculation

[2157] The server obtains the current location information and the destination information set by the user, and generates an API request that includes the emotional state along with the obtained current location and destination information.

[2158] 3. Obtaining traffic information and calculating the optimal route

[2159] The server retrieves real-time traffic information from a traffic information database via API requests. Traffic information includes operation status and delay information. The optimal route is calculated based on the retrieved traffic information and the user's emotional state.

[2160] 4. Providing route information

[2161] The calculated optimal route is sent from the server to the terminal inside the vehicle. The terminal analyzes the route information and displays it on the vehicle's display device. The user is then guided through the voice guidance system.

[2162] Specific examples

[2163] Below is a specific example where a user wants to go from Shibuya Station to Shinjuku Station. At this time, the user is a little impatient.

[2164] Acquiring and analyzing emotion data

[2165] A camera inside the vehicle captures the user's face, and an emotion recognition engine determines that the user is anxious.

[2166] Data collection for route calculation

[2167] The server obtains the location information of the current location (Shibuya Station) and the destination information (Shinjuku Station) entered by the user.

[2168] Obtaining traffic information and calculating the optimal route

[2169] Traffic information is obtained through a web API and the shortest route using the Keio Line is calculated.

[2170] Providing route information

[2171] The calculated route to Shinjuku Station using the Keio Line is sent to the terminal, which then displays this information on its display and announces via a voice guidance system that "If you take the Keio Line to Shinjuku Station, the journey will take approximately eight minutes."

[2172] Prompt Sentence Examples

[2173] An example of a prompt sentence is, "I'm going to Shinjuku Station. I'm currently at Shibuya Station and I'm a little anxious. Please tell me the best route."

[2174] In this way, a system can be realized that suggests optimal routes based on real-time traffic conditions while taking into account the user's emotional state.

[2175] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2176] Step 1:

[2177] The device uses cameras and microphones installed in the vehicle to capture the user's facial images and voice, which then generates data reflecting the user's emotional state. The acquired data is then sent to emotion recognition means.

[2178] Input: face image, voice data

[2179] Output: Raw data passed to the emotion recognition engine

[2180] Step 2:

[2181] The server uses an emotion recognition engine (e.g., Microsoft Azure Cognitive Services) to analyze the acquired facial images and voice data. As a result of the analysis, the user's emotional state (e.g., anxious, relaxed, etc.) is identified.

[2182] Input: face image, voice data

[2183] Output: User's emotional state (e.g., anxious, relaxed)

[2184] Step 3:

[2185] The user inputs the destination into the device. In this process, the destination information, such as "Shinjuku Station," is entered using a touch screen or voice input device. The device then sends the entered destination information along with the current location information to the server.

[2186] Input: Destination information (e.g. Shinjuku Station)

[2187] Output: Destination information is sent to the server

[2188] Step 4:

[2189] The device uses the GPS sensor to obtain the user's current location. The latitude and longitude information of the current location is acquired and sent to the server.

[2190] Input: GPS data

[2191] Output: Current location information (latitude and longitude) is sent to the server

[2192] Step 5:

[2193] The server generates API request data including the current location information, destination information, and emotional state, and sends the request to the web API for traffic information acquisition. The request includes a query to find out the current traffic situation.

[2194] Input: current location information, destination information, emotional state

[2195] Output: API request data sent

[2196] Step 6:

[2197] The server receives real-time traffic information from a web API for traffic information acquisition. The received data includes real-time traffic information such as operation status and delay information.

[2198] Input: API request data

[2199] Output: Real-time traffic information

[2200] Step 7:

[2201] The server calculates the optimal route based on traffic information and the user's emotional state. If the user is in a hurry, the shortest route will be selected, and if the user is relaxed, the route with the best scenery will be selected. The calculated route information is generated in JSON format.

[2202] Input: Traffic information, emotional state

[2203] Output: Optimal route information (JSON format)

[2204] Step 8:

[2205] The server sends the calculated optimal route information to the terminal.

[2206] Input: Optimal route information (JSON format)

[2207] Output: Route information is sent to the device

[2208] Step 9:

[2209] The device analyzes the received route information and displays it in a user-friendly format (e.g., displaying the route on the vehicle's display). It also provides route instructions to the user through a voice guidance system.

[2210] Input: Optimal route information (JSON format)

[2211] Output: The route is displayed on the display and voice guidance is provided.

[2212] In this way, an optimal route is provided based on real-time traffic information while taking into account the user's emotional state.

[2213] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2214] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2215] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2216] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2217] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2218] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2219] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2220] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a s...

Claims

1. location information acquisition means for acquiring a current location; input means for receiving input information regarding a destination; traffic information acquiring means for acquiring traffic information in real time based on the current location acquired by the position information acquiring means and the destination received by the input means; a route calculation means for calculating an optimum route based on the traffic information acquired by the traffic information acquisition means; a route providing means for providing a user with the optimum route calculated by the route calculation means; A system including:

2. 2. The system according to claim 1, wherein the location information acquisition means acquires the current location using a GPS sensor.

3. The system according to claim 1 , wherein the traffic information acquisition means acquires traffic information through a web API.

Images