System

Abstract

A system is provided.SOLUTION: The system includes a means including a generation AI for generating an optimal route without an obstacle or a step, a terminal means for collecting information on a destination or a category of interest based on an input of a user, and a means for displaying the generated optimal route and information on related facilities on a user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, information on optimal routes without obstacles or steps, related facilities (shops, restrooms, elevators, etc.), tourist spots, and events is scattered separately, making it difficult for users to integrate and use this information. In addition, there is a lack of means for users to obtain the latest information in real time while traveling. Therefore, a comprehensive information provision system is needed that allows users to travel stress-free and efficiently use the necessary facilities and tourist spots. [Means for solving the problem]

[0005] The present invention solves this problem with a system that includes means including a generation AI for generating an optimal route free of obstacles and steps, terminal means for collecting information on destinations and categories of interest based on user input, and means for displaying the generated optimal route and information on related facilities on the user terminal.The system also includes server means for acquiring information on related facilities and tourist spots from a database based on the input information, and further includes server means for updating and providing route information and related information in real time while the user is traveling, thereby providing a system that allows users to travel to their destination without stress and obtain the information they need in a timely manner.

[0006] "Generative AI" is an artificial intelligence system that calculates and generates optimal routes without obstacles or steps.

[0007] "Terminal means" refers to an electronic device and its interface means for a user to input information, communicate with a server, and display received information.

[0008] The "server means" is a central processing unit that receives requests from users, collects related information in cooperation with a database, and calculates the optimal route using a generation AI.

[0009] A "database" is an information system that stores information on routes, facilities, tourist attractions, events, etc., and that is accessed as needed.

[0010] "Route information" is information about the route a user must take to reach their destination, taking into consideration the presence or absence of obstacles or steps.

[0011] "Related facilities" refers to information about facilities (e.g., shops, toilets, elevators, etc.) that can be used by the user during their travels.

[0012] "Tourist Attractions" refers to information about interesting places and landmarks that users can visit.

[0013] "Real-time information updates" refers to the process of continuously updating and providing the latest route and related information while the user is traveling.

[0014] A "user terminal" is a device carried by a user, which has the function of receiving and displaying information from a server. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a comprehensive information providing system that provides users with information on routes without obstacles or steps and related facilities. Specific embodiments of the system are described below.

[0037] System Overview

[0038] The system mainly consists of the following components:

[0039] 1. User terminal: A device used by users to input and view information. This includes smartphones and tablets.

[0040] 2. Server: Responsible for major backend processing such as receiving requests, communicating with the database, running generative AI, and providing generated information.

[0041] 3. Database: Stores data such as route information, facility information, tourist spots and event information, and provides information in response to inquiries from the server.

[0042] System Operation

[0043] 1. User Request Processing

[0044] User: Launches the application and inputs the destination and the category of facility they wish to visit (e.g., cafe, restroom, tourist spot, etc.).

[0045] Terminal: Sends the entered destination and category information to the server.

[0046] 2. Request Processing on the Server

[0047] Server: Receives and analyzes user requests. Based on the analyzed information, retrieves related route, facility, tourist attraction, and event information from the database.

[0048] 3. Generating optimal information using generative AI

[0049] Server: Using generation AI, it calculates the optimal route without obstacles or steps. It also takes into account the acquired facility and tourist spot information to generate the optimal information to provide to the user.

[0050] 4. Transmission and display of optimal information

[0051] Server: Sends the generated information to the user's terminal.

[0052] Terminal: Displays the received information so that the user can check it.

[0053] 5. Real-time information updates

[0054] Server: Updates the latest route and facility information in real time while the user is traveling, and re-executes the generation AI as necessary.

[0055] Terminal: Receives updated information in real time and updates the display.

[0056] Specific examples

[0057] User request input

[0058] User: Launches the app and enters, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way."

[0059] Device: Sends this information to the server.

[0060] Server-side processing

[0061] Server: Analyzes the request and retrieves route information from the database from Shinjuku Station to Shibuya Station, as well as information about cafes along the way.

[0062] Server: Using generative AI, it calculates the optimal route without obstacles or steps and suggests cafes based on the user's request.

[0063] Information generation and provision

[0064] Server: Sends the calculation results to the user's terminal.

[0065] Terminal: Displays optimal route and cafe information.

[0066] For example, the generation AI calculates, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. Along the way, we recommend the barrier-free cafe "Cafe A" near Yoyogi Station." and provides this to the user.

[0067] Real-time information updates

[0068] Server: Monitors user movements and periodically checks for changes in route and facility information.

[0069] On your device: If there are any changes, you will be notified and the new information will be displayed.

[0070] For example, if construction information is updated while you are traveling, real-time information such as "Due to construction work underway near Yoyogi Station, you will be changed to Cafe B, a cafe just before Yoyogi Station," will be provided.

[0071] In this way, a system is provided that allows users to reach their destination without stress and efficiently use facilities and tourist spots along the way.

[0072] The processing flow will be explained below.

[0073] Step 1:

[0074] User: Launches the application and enters authentication information on the login screen. The device accepts the user's input and moves to the next screen.

[0075] Step 2:

[0076] User: Enter the destination and the category of facilities you would like to stop at along the way. For example, you might enter, "I want to go from Shinjuku Station to Shibuya Station. I would like to stop at a cafe along the way."

[0077] Step 3:

[0078] Terminal: Generates request data for sending the input information to the server. The request data includes the user's input information (destination, category) and the user's ID.

[0079] Step 4:

[0080] Server: Receives and analyzes the request data sent from the user terminal. Extracts the destination, category information, and user ID.

[0081] Step 5:

[0082] Server: Accesses the database to obtain route information to the destination, as well as information about cafes that can be visited along the way.

[0083] Step 6:

[0084] Server: Using generation AI, the acquired route information is analyzed and an optimal route is generated without obstacles or steps. The optimal location is determined by taking into account information about cafes along the route.

[0085] Step 7:

[0086] Server: Integrates the generated optimal route and related cafe information and sends it to the user's device as a single data set.

[0087] Step 8:

[0088] Terminal: Analyzes the optimal route and cafe information received from the server and displays it to the user. For example, it displays information such as, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a barrier-free cafe near Yoyogi Station."

[0089] Step 9:

[0090] Server: Periodically obtains user location information, updates route information and facility information in real time, and re-executes the generation AI as necessary.

[0091] Step 10:

[0092] Server: If there are any changes to the latest route information or facility information, the server notifies the user device. For example, if construction work occurs on the route, the server regenerates new route and cafe information and sends it to the user.

[0093] Step 11:

[0094] Terminal: Receives updated information from the server and notifies the user while updating the display. For example, it may notify the user that "Due to construction work occurring near Yoyogi Station, the location will be changed to Cafe B, which is located just before Yoyogi Station."

[0095] This series of processing steps allows the user to always obtain the latest information on the optimal route without obstacles or steps, as well as information on related facilities, in real time.

[0096] Example 1

[0097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] Conventional route guidance systems lacked sufficient functionality to avoid obstacles and steps, causing inconvenience to many users. Furthermore, because routes were not updated in real time, they were unable to respond quickly when new obstacles arose during travel. Furthermore, information about facilities and tourist spots that users might want to stop at along the way was often not provided properly.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0100] In this invention, the server includes means for analyzing category information of destinations and desired facilities input from a user terminal, means for retrieving related route information and facility information from a database based on the analyzed information, means for calculating an optimal route free of obstacles and steps using a generative AI model and providing the optimal route to the user, means for transmitting and displaying the generated optimal route and related facility information to the user terminal, means for monitoring the user's location information while traveling and updating the route information and related facility information in real time, and means for retransmitting and displaying this data to the user terminal. This allows the user to reach their destination without stress and efficiently use facilities and tourist spots along the way.

[0101] A "user terminal" is a device that a user uses to input and view information, and includes smartphones and tablets.

[0102] A "server" is a back-end system that receives and analyzes user requests, obtains and generates relevant data, and sends it to the user terminal.

[0103] A "generative AI model" refers to an algorithm or program that uses machine learning or deep learning to generate optimal routes and related facility information in response to a user request.

[0104] The "destination" is the final destination specified by the user, and serves as the basis for the system to calculate the optimal route.

[0105] "Category information" is information indicating the type of intermediate point or facility the user desires to stop at, and examples include cafes and tourist spots.

[0106] "Route information" is data relating to detailed instructions and directions for a user to reach a destination.

[0107] "Facility information" refers to detailed data about restaurants, tourist spots, and other facilities located along or nearby the route.

[0108] "Analysis means" refers to the methods and functions for analyzing request information sent by users and extracting and organizing necessary data.

[0109] A "database" is a data management system that organizes and stores related data such as route information, facility information, and tourist spot information, and provides information in response to queries from the server.

[0110] "Real-time update" is a function that instantly updates route information and facility information in response to changes in the user's movement status and environment, and reflects that information on the user's terminal.

[0111] This invention is a comprehensive information provision system that provides users with information on obstacle-free and step-free routes and related facilities. This system consists of a user terminal, a server, a database, and a generative AI model.

[0112] System configuration

[0113] User device:

[0114] A device that allows users to input and view information. This includes smartphones and tablets. Specifically, Android (registered trademark) and iOS devices are used.

[0115] server:

[0116] This is a backend system that receives user requests, analyzes them, and retrieves and generates relevant data. It uses the Python Flask framework to process data and handle API requests.

[0117] Database:

[0118] It stores data such as route information, facility information, and tourist spot information, and provides information in response to queries from the server. It uses a MySQL (registered trademark) database.

[0119] Generative AI models:

[0120] This refers to algorithms and programs that use machine learning and deep learning to generate optimal routes and related facility information in response to user requests. Specifically, it uses GPT-4 (registered trademark).

[0121] Program processing

[0122] The user uses the user device to launch the application and input their destination and the category of facility they would like to stop at (e.g., cafe, restroom, tourist spot, etc.). The user can input, "I want to go from Shinjuku Station to Shibuya Station. I would like to stop at a cafe on the way."

[0123] The terminal sends the entered information to the server, where the data is encoded in JSON format or similar and sent via the HTTPS protocol.

[0124] The server analyzes the received requests and separates the information by destination and category. It uses the Python Flask framework for analysis. Based on the analysis results, the server retrieves relevant route and facility information from a MySQL database.

[0125] Next, the server uses the acquired information to calculate the optimal route using a generative AI model. At this time, a prompt sentence is input to the AI ​​to generate a route without obstacles or steps. For example, "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps, along with information on barrier-free cafes along the way."

[0126] The generative AI model calculates the optimal route based on the prompt and generates related facility information. The server then sends the generated information to the user's device. The data is again encoded in JSON format and transmitted via HTTPS.

[0127] The user's device displays the received information. Specifically, it displays information such as the optimal route and cafes that can be stopped off along the way on a map and in guide text. For example, it displays information such as, "Take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, we recommend the barrier-free cafe 'Cafe A' near Yoyogi Station."

[0128] Furthermore, the server monitors the user's movement status and updates the latest route information and facility information in real time. For example, the generation AI recalculates information such as "Due to construction work near Yoyogi Station, change to Cafe B, a cafe just before Yoyogi Station," and sends it to the device. The user's device receives the updated information in real time and immediately updates the display.

[0129] As a result, users can reach their destination without stress and efficiently use facilities and tourist spots along the way.

[0130] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0131] Step 1:

[0132] The user launches the application using a user device such as a smartphone or tablet and inputs the destination and the category of facilities they would like to stop at along the way (e.g., cafe, restroom, tourist spot, etc.).

[0133] Input: Destination and facility category information (e.g., "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way.")

[0134] Output: The destination and category information are saved as input data.

[0135] Specifically, the user uses the device's touch screen to enter information into a text field.

[0136] Step 2:

[0137] The terminal encodes the data entered by the user into JSON format and sends it to the server using the HTTPS protocol.

[0138] Input: User-entered destination and category information

[0139] Output: Encoded JSON data sent to the server as an HTTPS request.

[0140] The terminal transmits data using the network interface.

[0141] Step 3:

[0142] The server analyzes the received requests and separates the information by destination and category, using the Python Flask framework for analysis.

[0143] Input: Encoded JSON data received from the terminal

[0144] Output: Parsed destination and category information

[0145] The server parses the request and executes functions to extract the information.

[0146] Step 4:

[0147] Based on the analyzed information, the server retrieves relevant route and facility information from a MySQL database.

[0148] Input: Parsed destination and category information

[0149] Output: Route information and facility information obtained from the database

[0150] The server generates SQL queries and executes them against a MySQL database.

[0151] Step 5:

[0152] The server uses the acquired information to calculate the optimal route using a generative AI model, and inputs prompts to the AI ​​to generate a route without obstacles or steps.

[0153] Input: Route and facility information obtained from the database, prompt text (e.g., "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps, along with information on barrier-free cafes along the way.")

[0154] Output: Optimal route and facility information generated by the generative AI model

[0155] The server passes the acquired data and prompt sentences to the AI ​​model and calculates the optimal route.

[0156] Step 6:

[0157] The server sends the generated information to the user's terminal. The data is again encoded in JSON format and sent via the HTTPS protocol.

[0158] Input: Generated optimal route and facility information

[0159] Output: JSON data encoded as an HTTPS request is sent to the terminal

[0160] The server sends the data to the device in the form of an HTTPS request.

[0161] Step 7:

[0162] The user terminal parses the received information and displays the optimal route and information on facilities to stop at on a map or in guide text.

[0163] Input: Encoded JSON data received from the server

[0164] Output: Optimal route and facility information displayed in a user-viewable format

[0165] The user device parses the JSON data and updates the UI components for map display and text display.

[0166] Step 8:

[0167] The server monitors the user's location information and updates the latest route and facility information in real time. If necessary, it re-executes the generation AI and sends the updated information to the device.

[0168] Input: Real-time location information of the user, information on changes in the environment (e.g., construction information)

[0169] Output: Updated route information and facility information is sent to the user terminal.

[0170] The server uses a location monitoring system, re-runs the AI ​​model as needed, and sends updated data to the device.

[0171] (Application example 1)

[0172] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0173] When an autonomous vehicle heads to its destination, the challenge is to provide the user with an optimal route free of obstacles and steps, and to provide real-time updates to the information, thereby ensuring a safe and comfortable journey. Another challenge is to provide information on facilities and tourist spots that the user may want to stop at along the way, and generate an optimal route that meets the user's needs.

[0174] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0175] In this invention, the server includes a generation AI means for generating an optimal route free of obstacles and steps, a terminal means for collecting information on the destination and categories of interest based on user input, a means for calculating and providing a safe route for the autonomous vehicle, a means for displaying the generated optimal route and information on related facilities on the user terminal, and a means for updating the route information and calculating new information in real time, thereby enabling safe and efficient route guidance for the autonomous vehicle when heading to the destination and enabling route and facility information to be updated in real time while the user is traveling.

[0176] "Generative AI" is an artificial intelligence technology that generates optimal routes free of obstacles and steps based on user input information.

[0177] The "terminal means" is a device operated by a user, and is a means for inputting and viewing information.

[0178] An "autonomous vehicle" is a vehicle that drives autonomously according to a calculated route.

[0179] "Means for calculating and providing a safe route" refers to a system that calculates and provides a safe route that avoids obstacles and steps for an autonomous vehicle to travel.

[0180] "Related facilities" are facilities suggested based on the user's interests and needs, such as cafes and tourist spots that can be visited along the route.

[0181] A "user terminal" is an information display device such as a smartphone carried by a user or an in-vehicle display.

[0182] "Means for updating in real time" refers to means for obtaining the latest data on route information and related facility information while the user is moving and updating it immediately.

[0183] The "server means" is a server system that performs major back-end processing such as receiving requests, accessing databases, and executing generated AI.

[0184] The system for implementing this invention mainly consists of a "generation AI" that generates an optimal route free of obstacles and steps, a "terminal" that collects information on destinations and categories of interest based on user input, a "server" that calculates and provides safe routes for autonomous vehicles, and a "user terminal" that displays the generated optimal route and information on related facilities on the user terminal. It also includes means for updating route information in real time and calculating new information.

[0185] System Operation

[0186] 1. Request input on user terminal:

[0187] The user uses a device such as a smartphone or an in-car display to input the destination and the category of facility they want to stop at (e.g., cafe, tourist spot, etc.). Once the user has entered this information, the device sends it to the server.

[0188] 2. Request processing on the server side:

[0189] The server analyzes the information sent by the user and retrieves relevant data from a database. It also uses generative AI to generate an optimal route free of obstacles and steps, providing a safe route for the autonomous vehicle. At the same time, it also retrieves and suggests information on stopover facilities according to the user's preferences.

[0190] 3. Display of generated information:

[0191] The server transmits the calculated optimal route and related facility information to the user terminal, which displays this information for the user to review.

[0192] 4. Real-time information updates:

[0193] The server constantly monitors the latest route and facility information while the user is traveling, and re-executes the generation AI to update the route as necessary. The updated information is sent to the user's device in real time, and the display is updated.

[0194] Specific examples

[0195] As a concrete example, consider the case where a user sends a request saying, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way." In this case, the device sends the request to the server. The server analyzes the request and retrieves route information from Shinjuku Station to Shibuya Station and information about cafes along the way from a database. Next, the generation AI calculates the optimal route without obstacles or steps, and suggests cafes that meet the user's request. For example, the generation AI might calculate, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. We recommend a barrier-free cafe near Yoyogi Station on the way," and provide this information to the user.

[0196] If a change occurs in the route while the user is traveling, the server recalculates the route based on the latest information and notifies the user's device, presenting the new route. For example, construction information may be updated while the user is traveling, providing real-time information such as "Because construction is underway near Yoyogi Station, change to a cafe just before Yoyogi Station."

[0197] Prompt Sentence Examples

[0198] An example of a prompt sentence is, "I'm going from Shinjuku Station to Shibuya Station, and I'd like to stop at a cafe on the way. Please tell me the safe route." By inputting this prompt sentence into a generative AI model, it is possible to calculate the optimal route and provide real-time navigation to the user.

[0199] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0200] Step 1:

[0201] The user uses the terminal to input their destination and categories of interest (e.g., cafes, tourist spots, etc.). The input information is sent from the terminal to the server. The input of this step is the user's request information, and the output is the request data sent to the server.

[0202] Step 2:

[0203] The server analyzes the request data received from the user. Based on the results of the analysis, a query is made to the database. Route information to the destination and information on related facilities are obtained from the database. The input to this step is the request data, and the output is the obtained route information and facility information.

[0204] Step 3:

[0205] The server uses the analyzed information and the generative AI model to generate an optimal route without obstacles or steps. It also recommends facilities along the route that meet the user's preferences. The input for this step is route information and facility information, and the output is the optimal route information and recommended facility information.

[0206] Step 4:

[0207] The server sends the generated optimal route and related facility information to the user terminal, which displays this information through an appropriate user interface. The input of this step is the optimal route information and recommended facility information, and the output is the information displayed on the user terminal.

[0208] Step 5:

[0209] While the user is traveling, the server monitors in real time whether the route information and related facility information have been updated. If necessary, it re-runs the generation AI to calculate the latest route. The input of this step is the real-time updated information, and the output is the recalculated, latest route information.

[0210] Step 6:

[0211] The server sends updated information to the user terminal in real time, and the user terminal updates the display. The input of this step is the recalculated latest route information, and the output is the updated display information of the user terminal.

[0212] The specific operations performed at each step include data collection, analysis, generation, transmission, and display, and the entire system works together to provide users with optimal route and facility information in real time.

[0213] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0214] This invention combines a comprehensive information provision system that provides users with information on routes without obstacles or steps and related facilities with a function that recognizes the user's emotions and adjusts the information accordingly. A specific embodiment of the system is described below.

[0215] System Overview

[0216] The system mainly consists of the following components:

[0217] 1. User terminal: A device used by users to input and view information. This includes smartphones and tablets.

[0218] 2. Server: Responsible for major backend processing such as receiving requests, communicating with the database, running generative AI, providing generated information, and analyzing user sentiment.

[0219] 3. Database: Stores data such as route information, facility information, tourist spots and event information, and provides information in response to inquiries from the server.

[0220] 4. Emotion engine: An engine that analyzes user emotions and utilizes that information throughout the system.

[0221] System Operation

[0222] 1. User Request Processing

[0223] User: Launches the application and enters authentication information on the login screen, then enters the destination and the category of facility they would like to stop at along the way (e.g., cafe, restroom, tourist spot, etc.).

[0224] Terminal: Generates and sends request data to send this information to the server.

[0225] 2. Request Processing on the Server

[0226] Server: Receives and analyzes user requests. Based on the requests, it retrieves relevant information from the database. The retrieved information includes route information to the destination and information on facilities along the way.

[0227] 3. Generating optimal information using generative AI

[0228] Server: Using generation AI, the acquired route information is analyzed and an optimal route without obstacles or steps is generated. The acquired facility information is also taken into consideration to generate optimal information to provide to the user.

[0229] 4. Emotional Engine Adjustment

[0230] Emotion Engine: Detects and recognizes the user's emotions and adjusts the optimal route and related information accordingly. For example, if the user is feeling stressed, it will prioritize relaxing cafes and wide roads.

[0231] 5. Transmission and display of optimal information

[0232] Server: Transmits the generated optimal route and related facility information to the user terminal.

[0233] Terminal: Displays the received information to the user. For example, it displays information such as, "The best route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a barrier-free cafe near Yoyogi Station."

[0234] 6. Real-time information updates

[0235] Server: Periodically acquires user location information and updates route and facility information in real time. If necessary, re-executes the generation AI and readjusts the emotion engine.

[0236] Terminal: Receives updated information in real time, updates the display and notifies the user.

[0237] Specific examples

[0238] User request input

[0239] User: Launches the app and enters, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop at a cafe on the way."

[0240] Device: Sends this information to the server.

[0241] Server-side processing

[0242] Server: Analyzes the request and retrieves route information from the database from Shinjuku Station to Shibuya Station and information on cafes along the way. Using generative AI, it calculates the optimal route without obstacles or steps, and suggests cafes taking into account the user's emotional state.

[0243] Emotional Engine Adjustment

[0244] Emotion engine: If the system detects that the user is in an emotional state that requires relaxation, it will prioritize providing information about quiet and relaxing cafes.

[0245] Information generation and provision

[0246] Server: Sends the calculation results to the user's terminal.

[0247] Terminal: Display the best route and cafe information. For example, display the following: "The best route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station."

[0248] Real-time information updates

[0249] Server: If construction information or other disruption information is updated during travel, the generation AI and emotion engine are re-run to generate new information.

[0250] Device: Receives new information and updates the display. For example, a notification will be sent saying, "Due to construction work occurring near Yoyogi Station, we have changed the location to Cafe B, a cafe just before Yoyogi Station."

[0251] This series of processes allows users to always obtain the latest information on the optimal route, free of obstacles and steps, as well as information on related facilities in real time.In addition, adjustments are made according to the user's emotional state, making travel more comfortable.

[0252] The processing flow will be explained below.

[0253] Step 1:

[0254] User: Launches the application and enters authentication information on the login screen. Once authentication is successfully completed on the device, the destination entry screen will be displayed.

[0255] Step 2:

[0256] User: Enter the destination and the category of facility you want to stop at (e.g., cafe, restroom, tourist spot, etc.). For example, you might enter, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop at a cafe on the way."

[0257] Step 3:

[0258] Terminal: Generates request data including the input destination and category information, and sends it to the server along with the user ID.

[0259] Step 4:

[0260] Server: Analyzes the received request data and extracts the user's destination and the requested category information. For example, "Shinjuku Station to Shibuya Station" is extracted as the destination information and "cafe" is extracted as the category information.

[0261] Step 5:

[0262] Server: Accesses the database and retrieves route information related to the extracted destination. It also retrieves information about cafes along the way.

[0263] Step 6:

[0264] Server: Launches the generation AI and analyzes the acquired route information. It generates an optimal route without obstacles or steps, taking into account cafe information that meets the user's requirements.

[0265] Step 7:

[0266] Server: Activates the emotion engine and analyzes the user's emotional state. For example, if it detects that the user is feeling stressed, it will prioritize quiet and relaxing cafes.

[0267] Step 8:

[0268] Server: Integrates the optimal route generated by the generation AI and the adjusted facility information generated by the emotion engine, and sends it to the user's device as a single data set.

[0269] Step 9:

[0270] Terminal: Analyzes the optimal route and related facility information received from the server and displays it to the user in a visually easy-to-understand format. For example, it displays, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station."

[0271] Step 10:

[0272] Server: Periodically collects user location information and monitors changes to route and facility information in real time. If new information is confirmed, the generative AI and emotion engine are re-run to update the information.

[0273] Step 11:

[0274] Server: Notifies the user device of the optimal route and facility information updated in real time. For example, if it detects that construction work has started at a station along the way, it calculates new route information and re-adjusts the cafe information.

[0275] Step 12:

[0276] Terminal: Receives updated information from the server and notifies the user. For example, the terminal may notify the user that "Due to construction work occurring near Yoyogi Station, change to Cafe B, a cafe just before Yoyogi Station," and display the new route and cafe information on the screen.

[0277] Through this series of processes, users can consistently obtain the latest information on optimal routes free of obstacles and steps, as well as information on related facilities that takes into account the user's emotional state.

[0278] Example 2

[0279] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0280] The present invention aims to provide a system that not only avoids obstacles and steps when a user moves, but also provides information on the optimal route and facilities to stop at based on the user's emotional state. Another objective is to provide a system that updates route information and related facility information in real time and provides it to the user, thereby realizing a comfortable journey based on the latest information at all times.

[0281] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means including a generation artificial intelligence for generating an optimal route free of obstacles and steps, terminal means for collecting information on destinations and categories of interest based on user input, means for displaying the generated optimal route and information on related facilities on the user terminal, emotion recognition means for recognizing the user's emotions and adjusting the route and related facility information based on the emotions, and means for updating and providing route information and related information in real time. This not only enables the user to use an optimal route free of obstacles and steps, but also provides information tailored to the user's emotional state at the time, enabling more comfortable and efficient travel. Furthermore, providing the latest information in real time allows the user to always obtain optimal information on the route and facilities to be visited.

[0282] An "optimal route without obstacles or steps" is a route that allows a user to reach their destination in the safest and most efficient manner without encountering any physical obstacles or steps when traveling.

[0283] "Generative artificial intelligence" refers to technology or systems that generate optimal routes and related information based on large amounts of data, and specifically refers to software that uses natural language processing and machine learning.

[0284] A "user terminal" is a device that a user uses to input and view information, and includes smartphones, tablets, personal computers, etc.

[0285] "Terminal means" refers to a device or application that has the function of collecting information based on user input and sending it to a server.

[0286] "Emotion recognition means" refers to technology or systems that analyze a user's facial expressions, voice, and other physiological data to determine the user's emotional state at that time.

[0287] "Means for updating and providing in real time" refers to technologies and systems that periodically acquire user location information and instantly generate and provide new route information and related facility information.

[0288] This invention combines a comprehensive information provision system that provides users with information on routes without obstacles or steps and related facilities, with a function that recognizes the user's emotions and adjusts the information based on those emotions. Specific embodiments for carrying out the invention are described below.

[0289] The system mainly consists of the following components:

[0290] 1. User terminal: A device used by a user to input and view information. This includes smartphones and tablets.

[0291] 2. Server: Responsible for major backend processing such as receiving requests, communicating with the database, running generative AI, providing generated information, and analyzing user sentiment.

[0292] 3. Database: Stores data such as route information, facility information, tourist spots, and event information, and provides information in response to inquiries from the server.

[0293] 4. Emotion Engine: An engine that analyzes user emotions and utilizes that information throughout the system. Specifically, it uses facial expression recognition and voice analysis technology.

[0294] System Operation

[0295] User Input

[0296] The user starts the application and enters their authentication information on the login screen. If successful, they enter their destination and the category of facilities they would like to stop at along the way (e.g., cafe, restroom, tourist spot). The device generates request data based on this information and sends it to the server.

[0297] Request processing on the server

[0298] The server analyzes the received request data, queries the database, and obtains route information to the destination and information on related facilities. The information obtained includes route information to the destination and information on facilities that can be stopped off along the way.

[0299] Generating optimal information using generative AI

[0300] The server calls a generation AI (e.g., ChatGPT (registered trademark)) and analyzes the acquired route information and facility information. The prompt text is input as "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps. Also, please include information on cafes that can be stopped off along the way." The generation AI generates the optimal route and information on facilities to stop at based on the prompt text.

[0301] Emotional Engine Adjustment

[0302] The emotion engine (e.g., Emotion API) captures and analyzes the user's emotions via the camera or microphone. For example, if it determines that the user is feeling stressed, it will prioritize relaxing cafes and wide roads. The server will readjust the optimal route and facility information based on the emotion engine's results, ensuring that the most appropriate information is provided to the user.

[0303] Sending and displaying optimal information

[0304] The server sends the generated optimal route and related information to the user's device. The device analyzes the received information and displays it to the user. For example, it may display information such as, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station."

[0305] Real-time information updates

[0306] The server periodically obtains the user's location information and updates route information and related facility information in real time. It re-executes the generation AI and emotion engine to generate new information as needed. The device receives the updated information, keeps the display up to date, and notifies the user. For example, a notification may be sent saying, "Due to construction work occurring near Yoyogi Station, we will change to Cafe B, a cafe just before Yoyogi Station."

[0307] This allows users to always obtain the latest information on the optimal route without obstacles or steps, as well as information on related facilities, in real time.In addition, adjustments are made according to the user's emotional state, making travel more comfortable.

[0308] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0309] Step 1: User Input

[0310] The user starts the application and enters authentication information on the login screen. The authentication information is entered as a "user name" and "password." The device generates request data to send this authentication information to the server and sends it to the server. The input is the user authentication information, and the output is the authentication request data to the server.

[0311] Step 2: Authentication process

[0312] The server compares the received authentication information with the database, and if authentication is successful, the user is successfully logged in. The server sends the comparison result to the terminal. The input is authentication request data, and the output is response data indicating whether the login was successful or failed.

[0313] Step 3: User request input

[0314] The user inputs their destination and the category of facilities they would like to stop at along the way (e.g., cafe, restroom, tourist spot) into the app's interface. The device generates request data based on this input information and sends it to the server. The input is the destination and facility category information, and the output is the request data sent to the server.

[0315] Step 4: Request Parsing and Database Query

[0316] The server analyzes the received request data and queries the database. This query obtains route information to the destination and related facility information. Specifically, it obtains route information from Shinjuku Station to Shibuya Station and information about cafes along the way. The input is the request data, and the output is route information and facility information.

[0317] Step 5: Route generation by generative AI

[0318] The server analyzes the route information and facility information obtained using a generative AI model (e.g., ChatGPT) and generates an optimal route without obstacles or steps. It also generates information about facilities along the route. The prompt is entered as "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps. Please also include information about cafes along the way." The input is route information and the prompt, and the output is the optimal route and facility information.

[0319] Step 6: Emotional Engine Alignment

[0320] The emotion engine (e.g., Emotion API) captures and analyzes the user's facial expressions and voice via a camera and microphone. The emotion engine determines the user's emotional state, such as whether they are feeling stressed, and prioritizes locations such as relaxing cafes and wide roads. The input is facial expressions and voice data, and the output is the user's emotional state and adjusted route information.

[0321] Step 7: Sending the best information

[0322] The server sends the generated optimal route and related facility information to the user's terminal. The terminal analyzes the received information and displays it to the user. For example, the information displayed might be, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station." The input is the adjusted route information, and the output is the route and facility information displayed to the user.

[0323] Step 8: Update real-time information

[0324] The server periodically obtains the user's location information and updates route information and related facility information in real time. If necessary, it re-executes the generation AI and emotion engine to generate new information. The device receives the updated information, keeps the display up to date, and notifies the user. For example, it may notify the user that "due to construction work occurring near Yoyogi Station, change to Cafe B, a cafe just before Yoyogi Station." The input is location information and the latest environmental data, and the output is updated route and facility information.

[0325] (Application example 2)

[0326] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0327] In conventional delivery work, delivery personnel have limited means to quickly determine the optimal route to reach the destination while avoiding obstacles and steps. In addition, appropriate responses and route adjustments that take into account the emotional state of the customer at the delivery destination are rarely made, making improving customer satisfaction a challenge.

[0328] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0329] In this invention, the server includes means including a generation AI for generating an optimal route free of obstacles and steps, terminal means for collecting information on destinations and categories of interest based on user input, means for displaying the generated optimal route and information on related facilities on the user terminal, emotion recognition means for recognizing the user's emotions and adjusting the information provided based on the emotions, and means for providing emotion-based advice during delivery work. This enables delivery personnel to grasp the optimal route in real time and respond appropriately according to the customer's emotions.

[0330] "Generative AI" refers to artificial intelligence technology that generates optimal routes based on user input.

[0331] "Terminal means" refers to a device that allows a user to input and collect information about destinations and categories of interest.

[0332] "Display means" refers to technology for displaying the generated optimum route and information on related facilities on a user terminal.

[0333] "Emotion recognition means" refers to technology that analyzes the user's emotions and adjusts the information provided based on that information.

[0334] "Delivery operations" refers to a series of tasks related to the delivery and transportation of goods.

[0335] "Advice means" refers to technology for providing advice on appropriate responses and route adjustments based on the user's emotions.

[0336] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following describes an embodiment of the present invention.

[0337] The system is designed to help delivery personnel determine the optimal route in real time and respond appropriately based on customer sentiment. The system mainly consists of the following components:

[0338] Terminal means: A device such as a smartphone or tablet used by the delivery person, which allows the delivery person to input and collect information about the destination and categories of interest.

[0339] Server means: The server uses generation AI to generate an optimal route without obstacles or steps, retrieves information on facilities and tourist spots from a database, and uses emotion recognition means to analyze the user's emotional state and adjust the route and recommended information.

[0340] Display means: This is a technology that displays the generated optimal route and related facility information on the delivery person's device, allowing the delivery person to check the necessary information in real time.

[0341] Hardware and software used

[0342] Smartphone or tablet: A device used by delivery personnel to enter information and confirm routes.

[0343] Server: The backend system responsible for processing requests, retrieving information from databases, running generative AI, and sentiment analysis.

[0344] Generative AI model: An artificial intelligence technique for generating optimal routes based on user input.

[0345] Emotion recognition engine: Technology that analyzes the user's emotions and adjusts the information provided based on that information.

[0346] Overview of data processing and calculation

[0347] The server analyzes the user's request information received from the terminal means and calculates route information using the generative AI model. It also retrieves information on related facilities and tourist spots from a database and analyzes the user's emotions using an emotion recognition engine. The server then adjusts the route and recommended information based on the user's emotions and displays them on the terminal means.

[0348] Specific examples

[0349] For example, consider a case where a delivery person inputs, "When delivering a product from Shinjuku Station to Shibuya Station, I would like to stop by a cafe where I can refresh myself on the way." In this case, the server calculates the optimal route from Shinjuku Station to Shibuya Station and obtains information about cafes along the way. The emotion recognition engine then analyzes the customer's emotions, and if the customer is feeling stressed, for example, it will prioritize providing information about cafes where they can relax.

[0350] Prompt Sentence Examples

[0351] Enter the following prompt into the generative AI model:

[0352] "Analyze the user's image 'user_image.png' and suggest the route this user should take while traveling, as well as obstacles, steps, and relaxation facilities that should be considered. Additionally, provide advice on how the delivery person should respond if the user is feeling stressed."

[0353] This system allows delivery personnel to always have access to optimal route information and real-time updated information on related facilities, enabling them to respond appropriately to customer sentiment, thereby achieving efficient and effective delivery operations.

[0354] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0355] Step 1: User request input

[0356] The user uses the device to input the destination and the category of facility they want to stop at. This input information is formatted by the device and sent to the server. Specifically, the user inputs, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop at a cafe on the way."

[0357] Input: Destination and facility category (e.g. "From Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way.")

[0358] Output: The request data sent to the server

[0359] Step 2: Parsing the request on the server

[0360] The server analyzes the received request data and retrieves related information from the database, specifically route information to the destination and information on facilities along the way.

[0361] Input: Request data

[0362] Output: Route information and facility information

[0363] Step 3: Generative AI generates optimal route

[0364] The server uses generation AI to analyze the acquired route information and generate the optimal route without obstacles or steps. It also takes into account the acquired facility information to generate the optimal information to provide to the user.

[0365] Input: Route information and facility information

[0366] Output: Optimal path without obstacles or steps

[0367] Step 4: Adjustment with emotion recognition engine

[0368] The server uses an emotion recognition engine to detect and recognize the user's emotions. It analyzes emotions from images and other input data provided by the user and adjusts the optimal route and related information based on those emotions. For example, if the user is feeling stressed, it will prioritize information about relaxing cafes.

[0369] Input: User emotion data (e.g., face image)

[0370] Output: Adjusted route and facility information

[0371] Step 5: Send and display optimal information

[0372] The server sends the generated optimal route and related facility information to the terminal. The terminal displays the received information to the user. For example, it displays information such as, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a relaxing cafe near Yoyogi Station."

[0373] Input: Adjusted route and facility information

[0374] Output: Optimal route and facility information displayed on the terminal

[0375] Step 6: Real-time updates

[0376] The server periodically obtains the user's location information and updates route and facility information in real time. If necessary, it re-executes the generation AI and emotion recognition engine to generate new information. The device receives the new information, updates the display, and notifies the user.

[0377] Input: User location information, latest facility information

[0378] Output: Updated route and facility information

[0379] In this way, through a series of processing steps, the system enables delivery personnel to obtain the optimal route and related information in real time, providing service that is tailored to the customer's emotions.

[0380] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0381] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0382] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0383] [Second embodiment]

[0384] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0385] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0386] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0387] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0388] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0389] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0390] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0391] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0392] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0393] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0394] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0395] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0396] The present invention is a comprehensive information providing system that provides users with information on routes without obstacles or steps and related facilities. Specific embodiments of the system are described below.

[0397] System Overview

[0398] The system mainly consists of the following components:

[0399] 1. User terminal: A device used by users to input and view information. This includes smartphones and tablets.

[0400] 2. Server: Responsible for major backend processing such as receiving requests, communicating with the database, running generative AI, and providing generated information.

[0401] 3. Database: Stores data such as route information, facility information, tourist spots and event information, and provides information in response to inquiries from the server.

[0402] System Operation

[0403] 1. User Request Processing

[0404] User: Launches the application and inputs the destination and the category of facility they wish to visit (e.g., cafe, restroom, tourist spot, etc.).

[0405] Terminal: Sends the entered destination and category information to the server.

[0406] 2. Request Processing on the Server

[0407] Server: Receives and analyzes user requests. Based on the analyzed information, retrieves related route, facility, tourist attraction, and event information from the database.

[0408] 3. Generating optimal information using generative AI

[0409] Server: Using generation AI, it calculates the optimal route without obstacles or steps. It also takes into account the acquired facility and tourist spot information to generate the optimal information to provide to the user.

[0410] 4. Transmission and display of optimal information

[0411] Server: Sends the generated information to the user's terminal.

[0412] Terminal: Displays the received information so that the user can check it.

[0413] 5. Real-time information updates

[0414] Server: Updates the latest route and facility information in real time while the user is traveling, and re-executes the generation AI as necessary.

[0415] Terminal: Receives updated information in real time and updates the display.

[0416] Specific examples

[0417] User request input

[0418] User: Launches the app and enters, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way."

[0419] Device: Sends this information to the server.

[0420] Server-side processing

[0421] Server: Analyzes the request and retrieves route information from the database from Shinjuku Station to Shibuya Station, as well as information about cafes along the way.

[0422] Server: Using generative AI, it calculates the optimal route without obstacles or steps and suggests cafes based on the user's request.

[0423] Information generation and provision

[0424] Server: Sends the calculation results to the user's terminal.

[0425] Terminal: Displays optimal route and cafe information.

[0426] For example, the generation AI calculates, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. Along the way, we recommend the barrier-free cafe "Cafe A" near Yoyogi Station." and provides this to the user.

[0427] Real-time information updates

[0428] Server: Monitors user movements and periodically checks for changes in route and facility information.

[0429] On your device: If there are any changes, you will be notified and the new information will be displayed.

[0430] For example, if construction information is updated while you are traveling, real-time information such as "Due to construction work underway near Yoyogi Station, you will be changed to Cafe B, a cafe just before Yoyogi Station," will be provided.

[0431] In this way, a system is provided that allows users to reach their destination without stress and efficiently use facilities and tourist spots along the way.

[0432] The processing flow will be explained below.

[0433] Step 1:

[0434] User: Launches the application and enters authentication information on the login screen. The device accepts the user's input and moves to the next screen.

[0435] Step 2:

[0436] User: Enter the destination and the category of facilities you would like to stop at along the way. For example, you might enter, "I want to go from Shinjuku Station to Shibuya Station. I would like to stop at a cafe along the way."

[0437] Step 3:

[0438] Terminal: Generates request data for sending the input information to the server. The request data includes the user's input information (destination, category) and the user's ID.

[0439] Step 4:

[0440] Server: Receives and analyzes the request data sent from the user terminal. Extracts the destination, category information, and user ID.

[0441] Step 5:

[0442] Server: Accesses the database to obtain route information to the destination, as well as information about cafes that can be visited along the way.

[0443] Step 6:

[0444] Server: Using generation AI, the acquired route information is analyzed and an optimal route is generated without obstacles or steps. The optimal location is determined by taking into account information about cafes along the route.

[0445] Step 7:

[0446] Server: Integrates the generated optimal route and related cafe information and sends it to the user's device as a single data set.

[0447] Step 8:

[0448] Terminal: Analyzes the optimal route and cafe information received from the server and displays it to the user. For example, it displays information such as, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a barrier-free cafe near Yoyogi Station."

[0449] Step 9:

[0450] Server: Periodically obtains user location information, updates route information and facility information in real time, and re-executes the generation AI as necessary.

[0451] Step 10:

[0452] Server: If there are any changes to the latest route information or facility information, the server notifies the user device. For example, if construction work occurs on the route, the server regenerates new route and cafe information and sends it to the user.

[0453] Step 11:

[0454] Terminal: Receives updated information from the server and notifies the user while updating the display. For example, it may notify the user that "Due to construction work occurring near Yoyogi Station, the location will be changed to Cafe B, which is located just before Yoyogi Station."

[0455] This series of processing steps allows the user to always obtain the latest information on the optimal route without obstacles or steps, as well as information on related facilities, in real time.

[0456] Example 1

[0457] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0458] Conventional route guidance systems lacked sufficient functionality to avoid obstacles and steps, causing inconvenience to many users. Furthermore, because routes were not updated in real time, they were unable to respond quickly when new obstacles arose during travel. Furthermore, information about facilities and tourist spots that users might want to stop at along the way was often not provided properly.

[0459] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0460] In this invention, the server includes means for analyzing category information of destinations and desired facilities input from a user terminal, means for retrieving related route information and facility information from a database based on the analyzed information, means for calculating an optimal route free of obstacles and steps using a generative AI model and providing the optimal route to the user, means for transmitting and displaying the generated optimal route and related facility information to the user terminal, means for monitoring the user's location information while traveling and updating the route information and related facility information in real time, and means for retransmitting and displaying this data to the user terminal. This allows the user to reach their destination without stress and efficiently use facilities and tourist spots along the way.

[0461] A "user terminal" is a device that a user uses to input and view information, and includes smartphones and tablets.

[0462] A "server" is a back-end system that receives and analyzes user requests, obtains and generates relevant data, and sends it to the user terminal.

[0463] A "generative AI model" refers to an algorithm or program that uses machine learning or deep learning to generate optimal routes and related facility information in response to a user request.

[0464] The "destination" is the final destination specified by the user, and serves as the basis for the system to calculate the optimal route.

[0465] "Category information" is information indicating the type of intermediate point or facility the user desires to stop at, and examples include cafes and tourist spots.

[0466] "Route information" is data relating to detailed instructions and directions for a user to reach a destination.

[0467] "Facility information" refers to detailed data about restaurants, tourist spots, and other facilities located along or nearby the route.

[0468] "Analysis means" refers to the methods and functions for analyzing request information sent by users and extracting and organizing necessary data.

[0469] A "database" is a data management system that organizes and stores related data such as route information, facility information, and tourist spot information, and provides information in response to queries from the server.

[0470] "Real-time update" is a function that instantly updates route information and facility information in response to changes in the user's movement status and environment, and reflects that information on the user's terminal.

[0471] This invention is a comprehensive information provision system that provides users with information on obstacle-free and step-free routes and related facilities. This system consists of a user terminal, a server, a database, and a generative AI model.

[0472] System configuration

[0473] User device:

[0474] A device that allows users to input and view information. This includes smartphones and tablets. Specifically, Android and iOS devices are used.

[0475] server:

[0476] This is a backend system that receives user requests, analyzes them, and retrieves and generates relevant data. It uses the Python Flask framework to process data and handle API requests.

[0477] Database:

[0478] It stores data such as route information, facility information, and tourist spot information, and provides information in response to queries from the server. It uses a MySQL database.

[0479] Generative AI models:

[0480] This refers to algorithms and programs that use machine learning and deep learning to generate optimal routes and related facility information in response to user requests. Specifically, it uses GPT-4.

[0481] Program processing

[0482] The user uses the user device to launch the application and input their destination and the category of facility they would like to stop at (e.g., cafe, restroom, tourist spot, etc.). The user can input, "I want to go from Shinjuku Station to Shibuya Station. I would like to stop at a cafe on the way."

[0483] The terminal sends the entered information to the server, where the data is encoded in JSON format or similar and sent via the HTTPS protocol.

[0484] The server analyzes the received requests and separates the information by destination and category. It uses the Python Flask framework for analysis. Based on the analysis results, the server retrieves relevant route and facility information from a MySQL database.

[0485] Next, the server uses the acquired information to calculate the optimal route using a generative AI model. At this time, a prompt sentence is input to the AI ​​to generate a route without obstacles or steps. For example, "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps, along with information on barrier-free cafes along the way."

[0486] The generative AI model calculates the optimal route based on the prompt and generates related facility information. The server then sends the generated information to the user's device. The data is again encoded in JSON format and transmitted via HTTPS.

[0487] The user's device displays the received information. Specifically, it displays information such as the optimal route and cafes that can be stopped off along the way on a map and in guide text. For example, it displays information such as, "Take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, we recommend the barrier-free cafe 'Cafe A' near Yoyogi Station."

[0488] Furthermore, the server monitors the user's movement status and updates the latest route information and facility information in real time. For example, the generation AI recalculates information such as "Due to construction work near Yoyogi Station, change to Cafe B, a cafe just before Yoyogi Station," and sends it to the device. The user's device receives the updated information in real time and immediately updates the display.

[0489] As a result, users can reach their destination without stress and efficiently use facilities and tourist spots along the way.

[0490] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0491] Step 1:

[0492] The user launches the application using a user device such as a smartphone or tablet and inputs the destination and the category of facilities they would like to stop at along the way (e.g., cafe, restroom, tourist spot, etc.).

[0493] Input: Destination and facility category information (e.g., "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way.")

[0494] Output: The destination and category information are saved as input data.

[0495] Specifically, the user uses the device's touch screen to enter information into a text field.

[0496] Step 2:

[0497] The terminal encodes the data entered by the user into JSON format and sends it to the server using the HTTPS protocol.

[0498] Input: User-entered destination and category information

[0499] Output: Encoded JSON data sent to the server as an HTTPS request.

[0500] The terminal transmits data using the network interface.

[0501] Step 3:

[0502] The server analyzes the received requests and separates the information by destination and category, using the Python Flask framework for analysis.

[0503] Input: Encoded JSON data received from the terminal

[0504] Output: Parsed destination and category information

[0505] The server parses the request and executes functions to extract the information.

[0506] Step 4:

[0507] Based on the analyzed information, the server retrieves relevant route and facility information from a MySQL database.

[0508] Input: Parsed destination and category information

[0509] Output: Route information and facility information obtained from the database

[0510] The server generates SQL queries and executes them against a MySQL database.

[0511] Step 5:

[0512] The server uses the acquired information to calculate the optimal route using a generative AI model, and inputs prompts to the AI ​​to generate a route without obstacles or steps.

[0513] Input: Route and facility information obtained from the database, prompt text (e.g., "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps, along with information on barrier-free cafes along the way.")

[0514] Output: Optimal route and facility information generated by the generative AI model

[0515] The server passes the acquired data and prompt sentences to the AI ​​model and calculates the optimal route.

[0516] Step 6:

[0517] The server sends the generated information to the user's terminal. The data is again encoded in JSON format and sent via the HTTPS protocol.

[0518] Input: Generated optimal route and facility information

[0519] Output: JSON data encoded as an HTTPS request is sent to the terminal

[0520] The server sends the data to the device in the form of an HTTPS request.

[0521] Step 7:

[0522] The user terminal parses the received information and displays the optimal route and information on facilities to stop at on a map or in guide text.

[0523] Input: Encoded JSON data received from the server

[0524] Output: Optimal route and facility information displayed in a user-viewable format

[0525] The user device parses the JSON data and updates the UI components for map display and text display.

[0526] Step 8:

[0527] The server monitors the user's location information and updates the latest route and facility information in real time. If necessary, it re-executes the generation AI and sends the updated information to the device.

[0528] Input: Real-time location information of the user, information on changes in the environment (e.g., construction information)

[0529] Output: Updated route information and facility information is sent to the user terminal.

[0530] The server uses a location monitoring system, re-runs the AI ​​model as needed, and sends updated data to the device.

[0531] (Application example 1)

[0532] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0533] When an autonomous vehicle heads to its destination, the challenge is to provide the user with an optimal route free of obstacles and steps, and to provide real-time updates to the information, thereby ensuring a safe and comfortable journey. Another challenge is to provide information on facilities and tourist spots that the user may want to stop at along the way, and generate an optimal route that meets the user's needs.

[0534] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0535] In this invention, the server includes a generation AI means for generating an optimal route free of obstacles and steps, a terminal means for collecting information on the destination and categories of interest based on user input, a means for calculating and providing a safe route for the autonomous vehicle, a means for displaying the generated optimal route and information on related facilities on the user terminal, and a means for updating the route information and calculating new information in real time, thereby enabling safe and efficient route guidance for the autonomous vehicle when heading to the destination and enabling route and facility information to be updated in real time while the user is traveling.

[0536] "Generative AI" is an artificial intelligence technology that generates optimal routes free of obstacles and steps based on user input information.

[0537] The "terminal means" is a device operated by a user, and is a means for inputting and viewing information.

[0538] An "autonomous vehicle" is a vehicle that drives autonomously according to a calculated route.

[0539] "Means for calculating and providing a safe route" refers to a system that calculates and provides a safe route that avoids obstacles and steps for an autonomous vehicle to travel.

[0540] "Related facilities" are facilities suggested based on the user's interests and needs, such as cafes and tourist spots that can be visited along the route.

[0541] A "user terminal" is an information display device such as a smartphone carried by a user or an in-vehicle display.

[0542] "Means for updating in real time" refers to means for obtaining the latest data on route information and related facility information while the user is moving and updating it immediately.

[0543] The "server means" is a server system that performs major back-end processing such as receiving requests, accessing databases, and executing generated AI.

[0544] The system for implementing this invention mainly consists of a "generation AI" that generates an optimal route free of obstacles and steps, a "terminal" that collects information on destinations and categories of interest based on user input, a "server" that calculates and provides safe routes for autonomous vehicles, and a "user terminal" that displays the generated optimal route and information on related facilities on the user terminal. It also includes means for updating route information in real time and calculating new information.

[0545] System Operation

[0546] 1. Request input on user terminal:

[0547] The user uses a device such as a smartphone or an in-car display to input the destination and the category of facility they want to stop at (e.g., cafe, tourist spot, etc.). Once the user has entered this information, the device sends it to the server.

[0548] 2. Request processing on the server side:

[0549] The server analyzes the information sent by the user and retrieves relevant data from a database. It also uses generative AI to generate an optimal route free of obstacles and steps, providing a safe route for the autonomous vehicle. At the same time, it also retrieves and suggests information on stopover facilities according to the user's preferences.

[0550] 3. Display of generated information:

[0551] The server transmits the calculated optimal route and related facility information to the user terminal, which displays this information for the user to review.

[0552] 4. Real-time information updates:

[0553] The server constantly monitors the latest route and facility information while the user is traveling, and re-executes the generation AI to update the route as necessary. The updated information is sent to the user's device in real time, and the display is updated.

[0554] Specific examples

[0555] As a concrete example, consider the case where a user sends a request saying, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way." In this case, the device sends the request to the server. The server analyzes the request and retrieves route information from Shinjuku Station to Shibuya Station and information about cafes along the way from a database. Next, the generation AI calculates the optimal route without obstacles or steps, and suggests cafes that meet the user's request. For example, the generation AI might calculate, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. We recommend a barrier-free cafe near Yoyogi Station on the way," and provide this information to the user.

[0556] If a change occurs in the route while the user is traveling, the server recalculates the route based on the latest information and notifies the user's device, presenting the new route. For example, construction information may be updated while the user is traveling, providing real-time information such as "Because construction is underway near Yoyogi Station, change to a cafe just before Yoyogi Station."

[0557] Prompt Sentence Examples

[0558] An example of a prompt sentence is, "I'm going from Shinjuku Station to Shibuya Station, and I'd like to stop at a cafe on the way. Please tell me the safe route." By inputting this prompt sentence into a generative AI model, it is possible to calculate the optimal route and provide real-time navigation to the user.

[0559] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0560] Step 1:

[0561] The user uses the terminal to input their destination and categories of interest (e.g., cafes, tourist spots, etc.). The input information is sent from the terminal to the server. The input of this step is the user's request information, and the output is the request data sent to the server.

[0562] Step 2:

[0563] The server analyzes the request data received from the user. Based on the results of the analysis, a query is made to the database. Route information to the destination and information on related facilities are obtained from the database. The input to this step is the request data, and the output is the obtained route information and facility information.

[0564] Step 3:

[0565] The server uses the analyzed information and the generative AI model to generate an optimal route without obstacles or steps. It also recommends facilities along the route that meet the user's preferences. The input for this step is route information and facility information, and the output is the optimal route information and recommended facility information.

[0566] Step 4:

[0567] The server sends the generated optimal route and related facility information to the user terminal, which displays this information through an appropriate user interface. The input of this step is the optimal route information and recommended facility information, and the output is the information displayed on the user terminal.

[0568] Step 5:

[0569] While the user is traveling, the server monitors in real time whether the route information and related facility information have been updated. If necessary, it re-runs the generation AI to calculate the latest route. The input of this step is the real-time updated information, and the output is the recalculated, latest route information.

[0570] Step 6:

[0571] The server sends updated information to the user terminal in real time, and the user terminal updates the display. The input of this step is the recalculated latest route information, and the output is the updated display information of the user terminal.

[0572] The specific operations performed at each step include data collection, analysis, generation, transmission, and display, and the entire system works together to provide users with optimal route and facility information in real time.

[0573] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0574] This invention combines a comprehensive information provision system that provides users with information on routes without obstacles or steps and related facilities with a function that recognizes the user's emotions and adjusts the information accordingly. A specific embodiment of the system is described below.

[0575] System Overview

[0576] The system mainly consists of the following components:

[0577] 1. User terminal: A device used by users to input and view information. This includes smartphones and tablets.

[0578] 2. Server: Responsible for major backend processing such as receiving requests, communicating with the database, running generative AI, providing generated information, and analyzing user sentiment.

[0579] 3. Database: Stores data such as route information, facility information, tourist spots and event information, and provides information in response to inquiries from the server.

[0580] 4. Emotion engine: An engine that analyzes user emotions and utilizes that information throughout the system.

[0581] System Operation

[0582] 1. User Request Processing

[0583] User: Launches the application and enters authentication information on the login screen, then enters the destination and the category of facility they would like to stop at along the way (e.g., cafe, restroom, tourist spot, etc.).

[0584] Terminal: Generates and sends request data to send this information to the server.

[0585] 2. Request Processing on the Server

[0586] Server: Receives and analyzes user requests. Based on the requests, it retrieves relevant information from the database. The retrieved information includes route information to the destination and information on facilities along the way.

[0587] 3. Generating optimal information using generative AI

[0588] Server: Using generation AI, the acquired route information is analyzed and an optimal route without obstacles or steps is generated. The acquired facility information is also taken into consideration to generate optimal information to provide to the user.

[0589] 4. Emotional Engine Adjustment

[0590] Emotion Engine: Detects and recognizes the user's emotions and adjusts the optimal route and related information accordingly. For example, if the user is feeling stressed, it will prioritize relaxing cafes and wide roads.

[0591] 5. Transmission and display of optimal information

[0592] Server: Transmits the generated optimal route and related facility information to the user terminal.

[0593] Terminal: Displays the received information to the user. For example, it displays information such as, "The best route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a barrier-free cafe near Yoyogi Station."

[0594] 6. Real-time information updates

[0595] Server: Periodically acquires user location information and updates route and facility information in real time. If necessary, re-executes the generation AI and readjusts the emotion engine.

[0596] Terminal: Receives updated information in real time, updates the display and notifies the user.

[0597] Specific examples

[0598] User request input

[0599] User: Launches the app and enters, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop at a cafe on the way."

[0600] Device: Sends this information to the server.

[0601] Server-side processing

[0602] Server: Analyzes the request and retrieves route information from the database from Shinjuku Station to Shibuya Station and information on cafes along the way. Using generative AI, it calculates the optimal route without obstacles or steps, and suggests cafes taking into account the user's emotional state.

[0603] Emotional Engine Adjustment

[0604] Emotion engine: If the system detects that the user is in an emotional state that requires relaxation, it will prioritize providing information about quiet and relaxing cafes.

[0605] Information generation and provision

[0606] Server: Sends the calculation results to the user's terminal.

[0607] Terminal: Display the best route and cafe information. For example, display the following: "The best route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station."

[0608] Real-time information updates

[0609] Server: If construction information or other disruption information is updated during travel, the generation AI and emotion engine are re-run to generate new information.

[0610] Device: Receives new information and updates the display. For example, a notification will be sent saying, "Due to construction work occurring near Yoyogi Station, we have changed the location to Cafe B, a cafe just before Yoyogi Station."

[0611] This series of processes allows users to always obtain the latest information on the optimal route, free of obstacles and steps, as well as information on related facilities in real time.In addition, adjustments are made according to the user's emotional state, making travel more comfortable.

[0612] The processing flow will be explained below.

[0613] Step 1:

[0614] User: Launches the application and enters authentication information on the login screen. Once authentication is successfully completed on the device, the destination entry screen will be displayed.

[0615] Step 2:

[0616] User: Enter the destination and the category of facility you want to stop at (e.g., cafe, restroom, tourist spot, etc.). For example, you might enter, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop at a cafe on the way."

[0617] Step 3:

[0618] Terminal: Generates request data including the input destination and category information, and sends it to the server along with the user ID.

[0619] Step 4:

[0620] Server: Analyzes the received request data and extracts the user's destination and the requested category information. For example, "Shinjuku Station to Shibuya Station" is extracted as the destination information and "cafe" is extracted as the category information.

[0621] Step 5:

[0622] Server: Accesses the database and retrieves route information related to the extracted destination. It also retrieves information about cafes along the way.

[0623] Step 6:

[0624] Server: Launches the generation AI and analyzes the acquired route information. It generates an optimal route without obstacles or steps, taking into account cafe information that meets the user's requirements.

[0625] Step 7:

[0626] Server: Activates the emotion engine and analyzes the user's emotional state. For example, if it detects that the user is feeling stressed, it will prioritize quiet and relaxing cafes.

[0627] Step 8:

[0628] Server: Integrates the optimal route generated by the generation AI and the adjusted facility information generated by the emotion engine, and sends it to the user's device as a single data set.

[0629] Step 9:

[0630] Terminal: Analyzes the optimal route and related facility information received from the server and displays it to the user in a visually easy-to-understand format. For example, it displays, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station."

[0631] Step 10:

[0632] Server: Periodically collects user location information and monitors changes to route and facility information in real time. If new information is confirmed, the generative AI and emotion engine are re-run to update the information.

[0633] Step 11:

[0634] Server: Notifies the user device of the optimal route and facility information updated in real time. For example, if it detects that construction work has started at a station along the way, it calculates new route information and re-adjusts the cafe information.

[0635] Step 12:

[0636] Terminal: Receives updated information from the server and notifies the user. For example, the terminal may notify the user that "Due to construction work occurring near Yoyogi Station, change to Cafe B, a cafe just before Yoyogi Station," and display the new route and cafe information on the screen.

[0637] Through this series of processes, users can consistently obtain the latest information on optimal routes free of obstacles and steps, as well as information on related facilities that takes into account the user's emotional state.

[0638] Example 2

[0639] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0640] The present invention aims to provide a system that not only avoids obstacles and steps when a user moves, but also provides information on the optimal route and facilities to stop at based on the user's emotional state. Another objective is to provide a system that updates route information and related facility information in real time and provides it to the user, thereby realizing a comfortable journey based on the latest information at all times.

[0641] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means including a generation artificial intelligence for generating an optimal route free of obstacles and steps, terminal means for collecting information on destinations and categories of interest based on user input, means for displaying the generated optimal route and information on related facilities on the user terminal, emotion recognition means for recognizing the user's emotions and adjusting the route and related facility information based on the emotions, and means for updating and providing route information and related information in real time. This not only enables the user to use an optimal route free of obstacles and steps, but also provides information tailored to the user's emotional state at the time, enabling more comfortable and efficient travel. Furthermore, providing the latest information in real time allows the user to always obtain optimal information on the route and facilities to be visited.

[0642] An "optimal route without obstacles or steps" is a route that allows a user to reach their destination in the safest and most efficient manner without encountering any physical obstacles or steps when traveling.

[0643] "Generative artificial intelligence" refers to technology or systems that generate optimal routes and related information based on large amounts of data, and specifically refers to software that uses natural language processing and machine learning.

[0644] A "user terminal" is a device that a user uses to input and view information, and includes smartphones, tablets, personal computers, etc.

[0645] "Terminal means" refers to a device or application that has the function of collecting information based on user input and sending it to a server.

[0646] "Emotion recognition means" refers to technology or systems that analyze a user's facial expressions, voice, and other physiological data to determine the user's emotional state at that time.

[0647] "Means for updating and providing in real time" refers to technologies and systems that periodically acquire user location information and instantly generate and provide new route information and related facility information.

[0648] This invention combines a comprehensive information provision system that provides users with information on routes without obstacles or steps and related facilities, with a function that recognizes the user's emotions and adjusts the information based on those emotions. Specific embodiments for carrying out the invention are described below.

[0649] The system mainly consists of the following components:

[0650] 1. User terminal: A device used by a user to input and view information. This includes smartphones and tablets.

[0651] 2. Server: Responsible for major backend processing such as receiving requests, communicating with the database, running generative AI, providing generated information, and analyzing user sentiment.

[0652] 3. Database: Stores data such as route information, facility information, tourist spots, and event information, and provides information in response to inquiries from the server.

[0653] 4. Emotion Engine: An engine that analyzes user emotions and utilizes that information throughout the system. Specifically, it uses facial expression recognition and voice analysis technology.

[0654] System Operation

[0655] User Input

[0656] The user starts the application and enters their authentication information on the login screen. If successful, they enter their destination and the category of facilities they would like to stop at along the way (e.g., cafe, restroom, tourist spot). The device generates request data based on this information and sends it to the server.

[0657] Request processing on the server

[0658] The server analyzes the received request data, queries the database, and obtains route information to the destination and information on related facilities. The information obtained includes route information to the destination and information on facilities that can be stopped off along the way.

[0659] Generating optimal information using generative AI

[0660] The server calls a generation AI (e.g., ChatGPT) and analyzes the acquired route and facility information. The prompt text is input as follows: "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps. Also, please include information about cafes that can be stopped off along the way." The generation AI generates the optimal route and information about facilities to stop at based on the prompt text.

[0661] Emotional Engine Adjustment

[0662] The emotion engine (e.g., Emotion API) captures and analyzes the user's emotions via the camera or microphone. For example, if it determines that the user is feeling stressed, it will prioritize relaxing cafes and wide roads. The server will readjust the optimal route and facility information based on the emotion engine's results, ensuring that the most appropriate information is provided to the user.

[0663] Sending and displaying optimal information

[0664] The server sends the generated optimal route and related information to the user's device. The device analyzes the received information and displays it to the user. For example, it may display information such as, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station."

[0665] Real-time information updates

[0666] The server periodically obtains the user's location information and updates route information and related facility information in real time. It re-executes the generation AI and emotion engine to generate new information as needed. The device receives the updated information, keeps the display up to date, and notifies the user. For example, a notification may be sent saying, "Due to construction work occurring near Yoyogi Station, we will change to Cafe B, a cafe just before Yoyogi Station."

[0667] This allows users to always obtain the latest information on the optimal route without obstacles or steps, as well as information on related facilities, in real time.In addition, adjustments are made according to the user's emotional state, making travel more comfortable.

[0668] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0669] Step 1: User Input

[0670] The user starts the application and enters authentication information on the login screen. The authentication information is entered as a "user name" and "password." The device generates request data to send this authentication information to the server and sends it to the server. The input is the user authentication information, and the output is the authentication request data to the server.

[0671] Step 2: Authentication process

[0672] The server compares the received authentication information with the database, and if authentication is successful, the user is successfully logged in. The server sends the comparison result to the terminal. The input is authentication request data, and the output is response data indicating whether the login was successful or failed.

[0673] Step 3: User request input

[0674] The user inputs their destination and the category of facilities they would like to stop at along the way (e.g., cafe, restroom, tourist spot) into the app's interface. The device generates request data based on this input information and sends it to the server. The input is the destination and facility category information, and the output is the request data sent to the server.

[0675] Step 4: Request Parsing and Database Query

[0676] The server analyzes the received request data and queries the database. This query obtains route information to the destination and related facility information. Specifically, it obtains route information from Shinjuku Station to Shibuya Station and information about cafes along the way. The input is the request data, and the output is route information and facility information.

[0677] Step 5: Route generation by generative AI

[0678] The server analyzes the route information and facility information obtained using a generative AI model (e.g., ChatGPT) and generates an optimal route without obstacles or steps. It also generates information about facilities along the route. The prompt is entered as "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps. Please also include information about cafes along the way." The input is route information and the prompt, and the output is the optimal route and facility information.

[0679] Step 6: Emotional Engine Alignment

[0680] The emotion engine (e.g., Emotion API) captures and analyzes the user's facial expressions and voice via a camera and microphone. The emotion engine determines the user's emotional state, such as whether they are feeling stressed, and prioritizes locations such as relaxing cafes and wide roads. The input is facial expressions and voice data, and the output is the user's emotional state and adjusted route information.

[0681] Step 7: Sending the best information

[0682] The server sends the generated optimal route and related facility information to the user's terminal. The terminal analyzes the received information and displays it to the user. For example, the information displayed might be, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station." The input is the adjusted route information, and the output is the route and facility information displayed to the user.

[0683] Step 8: Update real-time information

[0684] The server periodically obtains the user's location information and updates route information and related facility information in real time. If necessary, it re-executes the generation AI and emotion engine to generate new information. The device receives the updated information, keeps the display up to date, and notifies the user. For example, it may notify the user that "due to construction work occurring near Yoyogi Station, change to Cafe B, a cafe just before Yoyogi Station." The input is location information and the latest environmental data, and the output is updated route and facility information.

[0685] (Application example 2)

[0686] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0687] In conventional delivery work, delivery personnel have limited means to quickly determine the optimal route to reach the destination while avoiding obstacles and steps. In addition, appropriate responses and route adjustments that take into account the emotional state of the customer at the delivery destination are rarely made, making improving customer satisfaction a challenge.

[0688] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0689] In this invention, the server includes means including a generation AI for generating an optimal route free of obstacles and steps, terminal means for collecting information on destinations and categories of interest based on user input, means for displaying the generated optimal route and information on related facilities on the user terminal, emotion recognition means for recognizing the user's emotions and adjusting the information provided based on the emotions, and means for providing emotion-based advice during delivery work. This enables delivery personnel to grasp the optimal route in real time and respond appropriately according to the customer's emotions.

[0690] "Generative AI" refers to artificial intelligence technology that generates optimal routes based on user input.

[0691] "Terminal means" refers to a device that allows a user to input and collect information about destinations and categories of interest.

[0692] "Display means" refers to technology for displaying the generated optimum route and information on related facilities on a user terminal.

[0693] "Emotion recognition means" refers to technology that analyzes the user's emotions and adjusts the information provided based on that information.

[0694] "Delivery operations" refers to a series of tasks related to the delivery and transportation of goods.

[0695] "Advice means" refers to technology for providing advice on appropriate responses and route adjustments based on the user's emotions.

[0696] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following describes an embodiment of the present invention.

[0697] The system is designed to help delivery personnel determine the optimal route in real time and respond appropriately based on customer sentiment. The system mainly consists of the following components:

[0698] Terminal means: A device such as a smartphone or tablet used by the delivery person, which allows the delivery person to input and collect information about the destination and categories of interest.

[0699] Server means: The server uses generation AI to generate an optimal route without obstacles or steps, retrieves information on facilities and tourist spots from a database, and uses emotion recognition means to analyze the user's emotional state and adjust the route and recommended information.

[0700] Display means: This is a technology that displays the generated optimal route and related facility information on the delivery person's device, allowing the delivery person to check the necessary information in real time.

[0701] Hardware and software used

[0702] Smartphone or tablet: A device used by delivery personnel to enter information and confirm routes.

[0703] Server: The backend system responsible for processing requests, retrieving information from databases, running generative AI, and sentiment analysis.

[0704] Generative AI model: An artificial intelligence technique for generating optimal routes based on user input.

[0705] Emotion recognition engine: Technology that analyzes the user's emotions and adjusts the information provided based on that information.

[0706] Overview of data processing and calculation

[0707] The server analyzes the user's request information received from the terminal means and calculates route information using the generative AI model. It also retrieves information on related facilities and tourist spots from a database and analyzes the user's emotions using an emotion recognition engine. The server then adjusts the route and recommended information based on the user's emotions and displays them on the terminal means.

[0708] Specific examples

[0709] For example, consider a case where a delivery person inputs, "When delivering a product from Shinjuku Station to Shibuya Station, I would like to stop by a cafe where I can refresh myself on the way." In this case, the server calculates the optimal route from Shinjuku Station to Shibuya Station and obtains information about cafes along the way. The emotion recognition engine then analyzes the customer's emotions, and if the customer is feeling stressed, for example, it will prioritize providing information about cafes where they can relax.

[0710] Prompt Sentence Examples

[0711] Enter the following prompt into the generative AI model:

[0712] "Analyze the user's image 'user_image.png' and suggest the route this user should take while traveling, as well as obstacles, steps, and relaxation facilities that should be considered. Additionally, provide advice on how the delivery person should respond if the user is feeling stressed."

[0713] This system allows delivery personnel to always have access to optimal route information and real-time updated information on related facilities, enabling them to respond appropriately to customer sentiment, thereby achieving efficient and effective delivery operations.

[0714] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0715] Step 1: User request input

[0716] The user uses the device to input the destination and the category of facility they want to stop at. This input information is formatted by the device and sent to the server. Specifically, the user inputs, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop at a cafe on the way."

[0717] Input: Destination and facility category (e.g. "From Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way.")

[0718] Output: The request data sent to the server

[0719] Step 2: Parsing the request on the server

[0720] The server analyzes the received request data and retrieves related information from the database, specifically route information to the destination and information on facilities along the way.

[0721] Input: Request data

[0722] Output: Route information and facility information

[0723] Step 3: Generative AI generates optimal route

[0724] The server uses generation AI to analyze the acquired route information and generate the optimal route without obstacles or steps. It also takes into account the acquired facility information to generate the optimal information to provide to the user.

[0725] Input: Route information and facility information

[0726] Output: Optimal path without obstacles or steps

[0727] Step 4: Adjustment with emotion recognition engine

[0728] The server uses an emotion recognition engine to detect and recognize the user's emotions. It analyzes emotions from images and other input data provided by the user and adjusts the optimal route and related information based on those emotions. For example, if the user is feeling stressed, it will prioritize information about relaxing cafes.

[0729] Input: User emotion data (e.g., face image)

[0730] Output: Adjusted route and facility information

[0731] Step 5: Send and display optimal information

[0732] The server sends the generated optimal route and related facility information to the terminal. The terminal displays the received information to the user. For example, it displays information such as, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a relaxing cafe near Yoyogi Station."

[0733] Input: Adjusted route and facility information

[0734] Output: Optimal route and facility information displayed on the terminal

[0735] Step 6: Real-time updates

[0736] The server periodically obtains the user's location information and updates route and facility information in real time. If necessary, it re-executes the generation AI and emotion recognition engine to generate new information. The device receives the new information, updates the display, and notifies the user.

[0737] Input: User location information, latest facility information

[0738] Output: Updated route and facility information

[0739] In this way, through a series of processing steps, the system enables delivery personnel to obtain the optimal route and related information in real time, providing service that is tailored to the customer's emotions.

[0740] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0741] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0742] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0743] [Third embodiment]

[0744] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0745] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0746] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0747] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0748] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0749] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0750] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0751] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0752] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0753] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0754] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0755] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0756] The present invention is a comprehensive information providing system that provides users with information on routes without obstacles or steps and related facilities. Specific embodiments of the system are described below.

[0757] System Overview

[0758] The system mainly consists of the following components:

[0759] 1. User terminal: A device used by users to input and view information. This includes smartphones and tablets.

[0760] 2. Server: Responsible for major backend processing such as receiving requests, communicating with the database, running generative AI, and providing generated information.

[0761] 3. Database: Stores data such as route information, facility information, tourist spots and event information, and provides information in response to inquiries from the server.

[0762] System Operation

[0763] 1. User Request Processing

[0764] User: Launches the application and inputs the destination and the category of facility they wish to visit (e.g., cafe, restroom, tourist spot, etc.).

[0765] Terminal: Sends the entered destination and category information to the server.

[0766] 2. Request Processing on the Server

[0767] Server: Receives and analyzes user requests. Based on the analyzed information, retrieves related route, facility, tourist attraction, and event information from the database.

[0768] 3. Generating optimal information using generative AI

[0769] Server: Using generation AI, it calculates the optimal route without obstacles or steps. It also takes into account the acquired facility and tourist spot information to generate the optimal information to provide to the user.

[0770] 4. Transmission and display of optimal information

[0771] Server: Sends the generated information to the user's terminal.

[0772] Terminal: Displays the received information so that the user can check it.

[0773] 5. Real-time information updates

[0774] Server: Updates the latest route and facility information in real time while the user is traveling, and re-executes the generation AI as necessary.

[0775] Terminal: Receives updated information in real time and updates the display.

[0776] Specific examples

[0777] User request input

[0778] User: Launches the app and enters, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way."

[0779] Device: Sends this information to the server.

[0780] Server-side processing

[0781] Server: Analyzes the request and retrieves route information from the database from Shinjuku Station to Shibuya Station, as well as information about cafes along the way.

[0782] Server: Using generative AI, it calculates the optimal route without obstacles or steps and suggests cafes based on the user's request.

[0783] Information generation and provision

[0784] Server: Sends the calculation results to the user's terminal.

[0785] Terminal: Displays optimal route and cafe information.

[0786] For example, the generation AI calculates, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. Along the way, we recommend the barrier-free cafe "Cafe A" near Yoyogi Station." and provides this to the user.

[0787] Real-time information updates

[0788] Server: Monitors user movements and periodically checks for changes in route and facility information.

[0789] On your device: If there are any changes, you will be notified and the new information will be displayed.

[0790] For example, if construction information is updated while you are traveling, real-time information such as "Due to construction work underway near Yoyogi Station, you will be changed to Cafe B, a cafe just before Yoyogi Station," will be provided.

[0791] In this way, a system is provided that allows users to reach their destination without stress and efficiently use facilities and tourist spots along the way.

[0792] The processing flow will be explained below.

[0793] Step 1:

[0794] User: Launches the application and enters authentication information on the login screen. The device accepts the user's input and moves to the next screen.

[0795] Step 2:

[0796] User: Enter the destination and the category of facilities you would like to stop at along the way. For example, you might enter, "I want to go from Shinjuku Station to Shibuya Station. I would like to stop at a cafe along the way."

[0797] Step 3:

[0798] Terminal: Generates request data for sending the input information to the server. The request data includes the user's input information (destination, category) and the user's ID.

[0799] Step 4:

[0800] Server: Receives and analyzes the request data sent from the user terminal. Extracts the destination, category information, and user ID.

[0801] Step 5:

[0802] Server: Accesses the database to obtain route information to the destination, as well as information about cafes that can be visited along the way.

[0803] Step 6:

[0804] Server: Using generation AI, the acquired route information is analyzed and an optimal route is generated without obstacles or steps. The optimal location is determined by taking into account information about cafes along the route.

[0805] Step 7:

[0806] Server: Integrates the generated optimal route and related cafe information and sends it to the user's device as a single data set.

[0807] Step 8:

[0808] Terminal: Analyzes the optimal route and cafe information received from the server and displays it to the user. For example, it displays information such as, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a barrier-free cafe near Yoyogi Station."

[0809] Step 9:

[0810] Server: Periodically obtains user location information, updates route information and facility information in real time, and re-executes the generation AI as necessary.

[0811] Step 10:

[0812] Server: If there are any changes to the latest route information or facility information, the server notifies the user device. For example, if construction work occurs on the route, the server regenerates new route and cafe information and sends it to the user.

[0813] Step 11:

[0814] Terminal: Receives updated information from the server and notifies the user while updating the display. For example, it may notify the user that "Due to construction work occurring near Yoyogi Station, the location will be changed to Cafe B, which is located just before Yoyogi Station."

[0815] This series of processing steps allows the user to always obtain the latest information on the optimal route without obstacles or steps, as well as information on related facilities, in real time.

[0816] Example 1

[0817] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0818] Conventional route guidance systems lacked sufficient functionality to avoid obstacles and steps, causing inconvenience to many users. Furthermore, because routes were not updated in real time, they were unable to respond quickly when new obstacles arose during travel. Furthermore, information about facilities and tourist spots that users might want to stop at along the way was often not provided properly.

[0819] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0820] In this invention, the server includes means for analyzing category information of destinations and desired facilities input from a user terminal, means for retrieving related route information and facility information from a database based on the analyzed information, means for calculating an optimal route free of obstacles and steps using a generative AI model and providing the optimal route to the user, means for transmitting and displaying the generated optimal route and related facility information to the user terminal, means for monitoring the user's location information while traveling and updating the route information and related facility information in real time, and means for retransmitting and displaying this data to the user terminal. This allows the user to reach their destination without stress and efficiently use facilities and tourist spots along the way.

[0821] A "user terminal" is a device that a user uses to input and view information, and includes smartphones and tablets.

[0822] A "server" is a back-end system that receives and analyzes user requests, obtains and generates relevant data, and sends it to the user terminal.

[0823] A "generative AI model" refers to an algorithm or program that uses machine learning or deep learning to generate optimal routes and related facility information in response to a user request.

[0824] The "destination" is the final destination specified by the user, and serves as the basis for the system to calculate the optimal route.

[0825] "Category information" is information indicating the type of intermediate point or facility the user desires to stop at, and examples include cafes and tourist spots.

[0826] "Route information" is data relating to detailed instructions and directions for a user to reach a destination.

[0827] "Facility information" refers to detailed data about restaurants, tourist spots, and other facilities located along or nearby the route.

[0828] "Analysis means" refers to the methods and functions for analyzing request information sent by users and extracting and organizing necessary data.

[0829] A "database" is a data management system that organizes and stores related data such as route information, facility information, and tourist spot information, and provides information in response to queries from the server.

[0830] "Real-time update" is a function that instantly updates route information and facility information in response to changes in the user's movement status and environment, and reflects that information on the user's terminal.

[0831] This invention is a comprehensive information provision system that provides users with information on obstacle-free and step-free routes and related facilities. This system consists of a user terminal, a server, a database, and a generative AI model.

[0832] System configuration

[0833] User device:

[0834] A device that allows users to input and view information. This includes smartphones and tablets. Specifically, Android and iOS devices are used.

[0835] server:

[0836] This is a backend system that receives user requests, analyzes them, and retrieves and generates relevant data. It uses the Python Flask framework to process data and handle API requests.

[0837] Database:

[0838] It stores data such as route information, facility information, and tourist spot information, and provides information in response to queries from the server. It uses a MySQL database.

[0839] Generative AI models:

[0840] This refers to algorithms and programs that use machine learning and deep learning to generate optimal routes and related facility information in response to user requests. Specifically, it uses GPT-4.

[0841] Program processing

[0842] The user uses the user device to launch the application and input their destination and the category of facility they would like to stop at (e.g., cafe, restroom, tourist spot, etc.). The user can input, "I want to go from Shinjuku Station to Shibuya Station. I would like to stop at a cafe on the way."

[0843] The terminal sends the entered information to the server, where the data is encoded in JSON format or similar and sent via the HTTPS protocol.

[0844] The server analyzes the received requests and separates the information by destination and category. It uses the Python Flask framework for analysis. Based on the analysis results, the server retrieves relevant route and facility information from a MySQL database.

[0845] Next, the server uses the acquired information to calculate the optimal route using a generative AI model. At this time, a prompt sentence is input to the AI ​​to generate a route without obstacles or steps. For example, "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps, along with information on barrier-free cafes along the way."

[0846] The generative AI model calculates the optimal route based on the prompt and generates related facility information. The server then sends the generated information to the user's device. The data is again encoded in JSON format and transmitted via HTTPS.

[0847] The user's device displays the received information. Specifically, it displays information such as the optimal route and cafes that can be stopped off along the way on a map and in guide text. For example, it displays information such as, "Take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, we recommend the barrier-free cafe 'Cafe A' near Yoyogi Station."

[0848] Furthermore, the server monitors the user's movement status and updates the latest route information and facility information in real time. For example, the generation AI recalculates information such as "Due to construction work near Yoyogi Station, change to Cafe B, a cafe just before Yoyogi Station," and sends it to the device. The user's device receives the updated information in real time and immediately updates the display.

[0849] As a result, users can reach their destination without stress and efficiently use facilities and tourist spots along the way.

[0850] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0851] Step 1:

[0852] The user launches the application using a user device such as a smartphone or tablet and inputs the destination and the category of facilities they would like to stop at along the way (e.g., cafe, restroom, tourist spot, etc.).

[0853] Input: Destination and facility category information (e.g., "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way.")

[0854] Output: The destination and category information are saved as input data.

[0855] Specifically, the user uses the device's touch screen to enter information into a text field.

[0856] Step 2:

[0857] The terminal encodes the data entered by the user into JSON format and sends it to the server using the HTTPS protocol.

[0858] Input: User-entered destination and category information

[0859] Output: Encoded JSON data sent to the server as an HTTPS request.

[0860] The terminal transmits data using the network interface.

[0861] Step 3:

[0862] The server analyzes the received requests and separates the information by destination and category, using the Python Flask framework for analysis.

[0863] Input: Encoded JSON data received from the terminal

[0864] Output: Parsed destination and category information

[0865] The server parses the request and executes functions to extract the information.

[0866] Step 4:

[0867] Based on the analyzed information, the server retrieves relevant route and facility information from a MySQL database.

[0868] Input: Parsed destination and category information

[0869] Output: Route information and facility information obtained from the database

[0870] The server generates SQL queries and executes them against a MySQL database.

[0871] Step 5:

[0872] The server uses the acquired information to calculate the optimal route using a generative AI model, and inputs prompts to the AI ​​to generate a route without obstacles or steps.

[0873] Input: Route and facility information obtained from the database, prompt text (e.g., "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps, along with information on barrier-free cafes along the way.")

[0874] Output: Optimal route and facility information generated by the generative AI model

[0875] The server passes the acquired data and prompt sentences to the AI ​​model and calculates the optimal route.

[0876] Step 6:

[0877] The server sends the generated information to the user's terminal. The data is again encoded in JSON format and sent via the HTTPS protocol.

[0878] Input: Generated optimal route and facility information

[0879] Output: JSON data encoded as an HTTPS request is sent to the terminal

[0880] The server sends the data to the device in the form of an HTTPS request.

[0881] Step 7:

[0882] The user terminal parses the received information and displays the optimal route and information on facilities to stop at on a map or in guide text.

[0883] Input: Encoded JSON data received from the server

[0884] Output: Optimal route and facility information displayed in a user-viewable format

[0885] The user device parses the JSON data and updates the UI components for map display and text display.

[0886] Step 8:

[0887] The server monitors the user's location information and updates the latest route and facility information in real time. If necessary, it re-executes the generation AI and sends the updated information to the device.

[0888] Input: Real-time location information of the user, information on changes in the environment (e.g., construction information)

[0889] Output: Updated route information and facility information is sent to the user terminal.

[0890] The server uses a location monitoring system, re-runs the AI ​​model as needed, and sends updated data to the device.

[0891] (Application example 1)

[0892] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0893] When an autonomous vehicle heads to its destination, the challenge is to provide the user with an optimal route free of obstacles and steps, and to provide real-time updates to the information, thereby ensuring a safe and comfortable journey. Another challenge is to provide information on facilities and tourist spots that the user may want to stop at along the way, and generate an optimal route that meets the user's needs.

[0894] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0895] In this invention, the server includes a generation AI means for generating an optimal route free of obstacles and steps, a terminal means for collecting information on the destination and categories of interest based on user input, a means for calculating and providing a safe route for the autonomous vehicle, a means for displaying the generated optimal route and information on related facilities on the user terminal, and a means for updating the route information and calculating new information in real time, thereby enabling safe and efficient route guidance for the autonomous vehicle when heading to the destination and enabling route and facility information to be updated in real time while the user is traveling.

[0896] "Generative AI" is an artificial intelligence technology that generates optimal routes free of obstacles and steps based on user input information.

[0897] The "terminal means" is a device operated by a user, and is a means for inputting and viewing information.

[0898] An "autonomous vehicle" is a vehicle that drives autonomously according to a calculated route.

[0899] "Means for calculating and providing a safe route" refers to a system that calculates and provides a safe route that avoids obstacles and steps for an autonomous vehicle to travel.

[0900] "Related facilities" are facilities suggested based on the user's interests and needs, such as cafes and tourist spots that can be visited along the route.

[0901] A "user terminal" is an information display device such as a smartphone carried by a user or an in-vehicle display.

[0902] "Means for updating in real time" refers to means for obtaining the latest data on route information and related facility information while the user is moving and updating it immediately.

[0903] The "server means" is a server system that performs major back-end processing such as receiving requests, accessing databases, and executing generated AI.

[0904] The system for implementing this invention mainly consists of a "generation AI" that generates an optimal route free of obstacles and steps, a "terminal" that collects information on destinations and categories of interest based on user input, a "server" that calculates and provides safe routes for autonomous vehicles, and a "user terminal" that displays the generated optimal route and information on related facilities on the user terminal. It also includes means for updating route information in real time and calculating new information.

[0905] System Operation

[0906] 1. Request input on user terminal:

[0907] The user uses a device such as a smartphone or an in-car display to input the destination and the category of facility they want to stop at (e.g., cafe, tourist spot, etc.). Once the user has entered this information, the device sends it to the server.

[0908] 2. Request processing on the server side:

[0909] The server analyzes the information sent by the user and retrieves relevant data from a database. It also uses generative AI to generate an optimal route free of obstacles and steps, providing a safe route for the autonomous vehicle. At the same time, it also retrieves and suggests information on stopover facilities according to the user's preferences.

[0910] 3. Display of generated information:

[0911] The server transmits the calculated optimal route and related facility information to the user terminal, which displays this information for the user to review.

[0912] 4. Real-time information updates:

[0913] The server constantly monitors the latest route and facility information while the user is traveling, and re-executes the generation AI to update the route as necessary. The updated information is sent to the user's device in real time, and the display is updated.

[0914] Specific examples

[0915] As a concrete example, consider the case where a user sends a request saying, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way." In this case, the device sends the request to the server. The server analyzes the request and retrieves route information from Shinjuku Station to Shibuya Station and information about cafes along the way from a database. Next, the generation AI calculates the optimal route without obstacles or steps, and suggests cafes that meet the user's request. For example, the generation AI might calculate, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. We recommend a barrier-free cafe near Yoyogi Station on the way," and provide this information to the user.

[0916] If a change occurs in the route while the user is traveling, the server recalculates the route based on the latest information and notifies the user's device, presenting the new route. For example, construction information may be updated while the user is traveling, providing real-time information such as "Because construction is underway near Yoyogi Station, change to a cafe just before Yoyogi Station."

[0917] Prompt Sentence Examples

[0918] An example of a prompt sentence is, "I'm going from Shinjuku Station to Shibuya Station, and I'd like to stop at a cafe on the way. Please tell me the safe route." By inputting this prompt sentence into a generative AI model, it is possible to calculate the optimal route and provide real-time navigation to the user.

[0919] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0920] Step 1:

[0921] The user uses the terminal to input their destination and categories of interest (e.g., cafes, tourist spots, etc.). The input information is sent from the terminal to the server. The input of this step is the user's request information, and the output is the request data sent to the server.

[0922] Step 2:

[0923] The server analyzes the request data received from the user. Based on the results of the analysis, a query is made to the database. Route information to the destination and information on related facilities are obtained from the database. The input to this step is the request data, and the output is the obtained route information and facility information.

[0924] Step 3:

[0925] The server uses the analyzed information and the generative AI model to generate an optimal route without obstacles or steps. It also recommends facilities along the route that meet the user's preferences. The input for this step is route information and facility information, and the output is the optimal route information and recommended facility information.

[0926] Step 4:

[0927] The server sends the generated optimal route and related facility information to the user terminal, which displays this information through an appropriate user interface. The input of this step is the optimal route information and recommended facility information, and the output is the information displayed on the user terminal.

[0928] Step 5:

[0929] While the user is traveling, the server monitors in real time whether the route information and related facility information have been updated. If necessary, it re-runs the generation AI to calculate the latest route. The input of this step is the real-time updated information, and the output is the recalculated, latest route information.

[0930] Step 6:

[0931] The server sends updated information to the user terminal in real time, and the user terminal updates the display. The input of this step is the recalculated latest route information, and the output is the updated display information of the user terminal.

[0932] The specific operations performed at each step include data collection, analysis, generation, transmission, and display, and the entire system works together to provide users with optimal route and facility information in real time.

[0933] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0934] This invention combines a comprehensive information provision system that provides users with information on routes without obstacles or steps and related facilities with a function that recognizes the user's emotions and adjusts the information accordingly. A specific embodiment of the system is described below.

[0935] System Overview

[0936] The system mainly consists of the following components:

[0937] 1. User terminal: A device used by users to input and view information. This includes smartphones and tablets.

[0938] 2. Server: Responsible for major backend processing such as receiving requests, communicating with the database, running generative AI, providing generated information, and analyzing user sentiment.

[0939] 3. Database: Stores data such as route information, facility information, tourist spots and event information, and provides information in response to inquiries from the server.

[0940] 4. Emotion engine: An engine that analyzes user emotions and utilizes that information throughout the system.

[0941] System Operation

[0942] 1. User Request Processing

[0943] User: Launches the application and enters authentication information on the login screen, then enters the destination and the category of facility they would like to stop at along the way (e.g., cafe, restroom, tourist spot, etc.).

[0944] Terminal: Generates and sends request data to send this information to the server.

[0945] 2. Request Processing on the Server

[0946] Server: Receives and analyzes user requests. Based on the requests, it retrieves relevant information from the database. The retrieved information includes route information to the destination and information on facilities along the way.

[0947] 3. Generating optimal information using generative AI

[0948] Server: Using generation AI, the acquired route information is analyzed and an optimal route without obstacles or steps is generated. The acquired facility information is also taken into consideration to generate optimal information to provide to the user.

[0949] 4. Emotional Engine Adjustment

[0950] Emotion Engine: Detects and recognizes the user's emotions and adjusts the optimal route and related information accordingly. For example, if the user is feeling stressed, it will prioritize relaxing cafes and wide roads.

[0951] 5. Transmission and display of optimal information

[0952] Server: Transmits the generated optimal route and related facility information to the user terminal.

[0953] Terminal: Displays the received information to the user. For example, it displays information such as, "The best route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a barrier-free cafe near Yoyogi Station."

[0954] 6. Real-time information updates

[0955] Server: Periodically acquires user location information and updates route and facility information in real time. If necessary, re-executes the generation AI and readjusts the emotion engine.

[0956] Terminal: Receives updated information in real time, updates the display and notifies the user.

[0957] Specific examples

[0958] User request input

[0959] User: Launches the app and enters, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop at a cafe on the way."

[0960] Device: Sends this information to the server.

[0961] Server-side processing

[0962] Server: Analyzes the request and retrieves route information from the database from Shinjuku Station to Shibuya Station and information on cafes along the way. Using generative AI, it calculates the optimal route without obstacles or steps, and suggests cafes taking into account the user's emotional state.

[0963] Emotional Engine Adjustment

[0964] Emotion engine: If the system detects that the user is in an emotional state that requires relaxation, it will prioritize providing information about quiet and relaxing cafes.

[0965] Information generation and provision

[0966] Server: Sends the calculation results to the user's terminal.

[0967] Terminal: Display the best route and cafe information. For example, display the following: "The best route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station."

[0968] Real-time information updates

[0969] Server: If construction information or other disruption information is updated during travel, the generation AI and emotion engine are re-run to generate new information.

[0970] Device: Receives new information and updates the display. For example, a notification will be sent saying, "Due to construction work occurring near Yoyogi Station, we have changed the location to Cafe B, a cafe just before Yoyogi Station."

[0971] This series of processes allows users to always obtain the latest information on the optimal route, free of obstacles and steps, as well as information on related facilities in real time.In addition, adjustments are made according to the user's emotional state, making travel more comfortable.

[0972] The processing flow will be explained below.

[0973] Step 1:

[0974] User: Launches the application and enters authentication information on the login screen. Once authentication is successfully completed on the device, the destination entry screen will be displayed.

[0975] Step 2:

[0976] User: Enter the destination and the category of facility you want to stop at (e.g., cafe, restroom, tourist spot, etc.). For example, you might enter, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop at a cafe on the way."

[0977] Step 3:

[0978] Terminal: Generates request data including the input destination and category information, and sends it to the server along with the user ID.

[0979] Step 4:

[0980] Server: Analyzes the received request data and extracts the user's destination and the requested category information. For example, "Shinjuku Station to Shibuya Station" is extracted as the destination information and "cafe" is extracted as the category information.

[0981] Step 5:

[0982] Server: Accesses the database and retrieves route information related to the extracted destination. It also retrieves information about cafes along the way.

[0983] Step 6:

[0984] Server: Launches the generation AI and analyzes the acquired route information. It generates an optimal route without obstacles or steps, taking into account cafe information that meets the user's requirements.

[0985] Step 7:

[0986] Server: Activates the emotion engine and analyzes the user's emotional state. For example, if it detects that the user is feeling stressed, it will prioritize quiet and relaxing cafes.

[0987] Step 8:

[0988] Server: Integrates the optimal route generated by the generation AI and the adjusted facility information generated by the emotion engine, and sends it to the user's device as a single data set.

[0989] Step 9:

[0990] Terminal: Analyzes the optimal route and related facility information received from the server and displays it to the user in a visually easy-to-understand format. For example, it displays, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station."

[0991] Step 10:

[0992] Server: Periodically collects user location information and monitors changes to route and facility information in real time. If new information is confirmed, the generative AI and emotion engine are re-run to update the information.

[0993] Step 11:

[0994] Server: Notifies the user device of the optimal route and facility information updated in real time. For example, if it detects that construction work has started at a station along the way, it calculates new route information and re-adjusts the cafe information.

[0995] Step 12:

[0996] Terminal: Receives updated information from the server and notifies the user. For example, the terminal may notify the user that "Due to construction work occurring near Yoyogi Station, change to Cafe B, a cafe just before Yoyogi Station," and display the new route and cafe information on the screen.

[0997] Through this series of processes, users can consistently obtain the latest information on optimal routes free of obstacles and steps, as well as information on related facilities that takes into account the user's emotional state.

[0998] Example 2

[0999] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1000] The present invention aims to provide a system that not only avoids obstacles and steps when a user moves, but also provides information on the optimal route and facilities to stop at based on the user's emotional state. Another objective is to provide a system that updates route information and related facility information in real time and provides it to the user, thereby realizing a comfortable journey based on the latest information at all times.

[1001] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means including a generation artificial intelligence for generating an optimal route free of obstacles and steps, terminal means for collecting information on destinations and categories of interest based on user input, means for displaying the generated optimal route and information on related facilities on the user terminal, emotion recognition means for recognizing the user's emotions and adjusting the route and related facility information based on the emotions, and means for updating and providing route information and related information in real time. This not only enables the user to use an optimal route free of obstacles and steps, but also provides information tailored to the user's emotional state at the time, enabling more comfortable and efficient travel. Furthermore, providing the latest information in real time allows the user to always obtain optimal information on the route and facilities to be visited.

[1002] An "optimal route without obstacles or steps" is a route that allows a user to reach their destination in the safest and most efficient manner without encountering any physical obstacles or steps when traveling.

[1003] "Generative artificial intelligence" refers to technology or systems that generate optimal routes and related information based on large amounts of data, and specifically refers to software that uses natural language processing and machine learning.

[1004] A "user terminal" is a device that a user uses to input and view information, and includes smartphones, tablets, personal computers, etc.

[1005] "Terminal means" refers to a device or application that has the function of collecting information based on user input and sending it to a server.

[1006] "Emotion recognition means" refers to technology or systems that analyze a user's facial expressions, voice, and other physiological data to determine the user's emotional state at that time.

[1007] "Means for updating and providing in real time" refers to technologies and systems that periodically acquire user location information and instantly generate and provide new route information and related facility information.

[1008] This invention combines a comprehensive information provision system that provides users with information on routes without obstacles or steps and related facilities, with a function that recognizes the user's emotions and adjusts the information based on those emotions. Specific embodiments for carrying out the invention are described below.

[1009] The system mainly consists of the following components:

[1010] 1. User terminal: A device used by a user to input and view information. This includes smartphones and tablets.

[1011] 2. Server: Responsible for major backend processing such as receiving requests, communicating with the database, running generative AI, providing generated information, and analyzing user sentiment.

[1012] 3. Database: Stores data such as route information, facility information, tourist spots, and event information, and provides information in response to inquiries from the server.

[1013] 4. Emotion Engine: An engine that analyzes user emotions and utilizes that information throughout the system. Specifically, it uses facial expression recognition and voice analysis technology.

[1014] System Operation

[1015] User Input

[1016] The user starts the application and enters their authentication information on the login screen. If successful, they enter their destination and the category of facilities they would like to stop at along the way (e.g., cafe, restroom, tourist spot). The device generates request data based on this information and sends it to the server.

[1017] Request processing on the server

[1018] The server analyzes the received request data, queries the database, and obtains route information to the destination and information on related facilities. The information obtained includes route information to the destination and information on facilities that can be stopped off along the way.

[1019] Generating optimal information using generative AI

[1020] The server calls a generation AI (e.g., ChatGPT) and analyzes the acquired route and facility information. The prompt text is input as follows: "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps. Also, please include information about cafes that can be stopped off along the way." The generation AI generates the optimal route and information about facilities to stop at based on the prompt text.

[1021] Emotional Engine Adjustment

[1022] The emotion engine (e.g., Emotion API) captures and analyzes the user's emotions via the camera or microphone. For example, if it determines that the user is feeling stressed, it will prioritize relaxing cafes and wide roads. The server will readjust the optimal route and facility information based on the emotion engine's results, ensuring that the most appropriate information is provided to the user.

[1023] Sending and displaying optimal information

[1024] The server sends the generated optimal route and related information to the user's device. The device analyzes the received information and displays it to the user. For example, it may display information such as, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station."

[1025] Real-time information updates

[1026] The server periodically obtains the user's location information and updates route information and related facility information in real time. It re-executes the generation AI and emotion engine to generate new information as needed. The device receives the updated information, keeps the display up to date, and notifies the user. For example, a notification may be sent saying, "Due to construction work occurring near Yoyogi Station, we will change to Cafe B, a cafe just before Yoyogi Station."

[1027] This allows users to always obtain the latest information on the optimal route without obstacles or steps, as well as information on related facilities, in real time.In addition, adjustments are made according to the user's emotional state, making travel more comfortable.

[1028] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1029] Step 1: User Input

[1030] The user starts the application and enters authentication information on the login screen. The authentication information is entered as a "user name" and "password." The device generates request data to send this authentication information to the server and sends it to the server. The input is the user authentication information, and the output is the authentication request data to the server.

[1031] Step 2: Authentication process

[1032] The server compares the received authentication information with the database, and if authentication is successful, the user is successfully logged in. The server sends the comparison result to the terminal. The input is authentication request data, and the output is response data indicating whether the login was successful or failed.

[1033] Step 3: User request input

[1034] The user inputs their destination and the category of facilities they would like to stop at along the way (e.g., cafe, restroom, tourist spot) into the app's interface. The device generates request data based on this input information and sends it to the server. The input is the destination and facility category information, and the output is the request data sent to the server.

[1035] Step 4: Request Parsing and Database Query

[1036] The server analyzes the received request data and queries the database. This query obtains route information to the destination and related facility information. Specifically, it obtains route information from Shinjuku Station to Shibuya Station and information about cafes along the way. The input is the request data, and the output is route information and facility information.

[1037] Step 5: Route generation by generative AI

[1038] The server analyzes the route information and facility information obtained using a generative AI model (e.g., ChatGPT) and generates an optimal route without obstacles or steps. It also generates information about facilities along the route. The prompt is entered as "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps. Please also include information about cafes along the way." The input is route information and the prompt, and the output is the optimal route and facility information.

[1039] Step 6: Emotional Engine Alignment

[1040] The emotion engine (e.g., Emotion API) captures and analyzes the user's facial expressions and voice via a camera and microphone. The emotion engine determines the user's emotional state, such as whether they are feeling stressed, and prioritizes locations such as relaxing cafes and wide roads. The input is facial expressions and voice data, and the output is the user's emotional state and adjusted route information.

[1041] Step 7: Sending the best information

[1042] The server sends the generated optimal route and related facility information to the user's terminal. The terminal analyzes the received information and displays it to the user. For example, the information displayed might be, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station." The input is the adjusted route information, and the output is the route and facility information displayed to the user.

[1043] Step 8: Update real-time information

[1044] The server periodically obtains the user's location information and updates route information and related facility information in real time. If necessary, it re-executes the generation AI and emotion engine to generate new information. The device receives the updated information, keeps the display up to date, and notifies the user. For example, it may notify the user that "due to construction work occurring near Yoyogi Station, change to Cafe B, a cafe just before Yoyogi Station." The input is location information and the latest environmental data, and the output is updated route and facility information.

[1045] (Application example 2)

[1046] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1047] In conventional delivery work, delivery personnel have limited means to quickly determine the optimal route to reach the destination while avoiding obstacles and steps. In addition, appropriate responses and route adjustments that take into account the emotional state of the customer at the delivery destination are rarely made, making improving customer satisfaction a challenge.

[1048] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1049] In this invention, the server includes means including a generation AI for generating an optimal route free of obstacles and steps, terminal means for collecting information on destinations and categories of interest based on user input, means for displaying the generated optimal route and information on related facilities on the user terminal, emotion recognition means for recognizing the user's emotions and adjusting the information provided based on the emotions, and means for providing emotion-based advice during delivery work. This enables delivery personnel to grasp the optimal route in real time and respond appropriately according to the customer's emotions.

[1050] "Generative AI" refers to artificial intelligence technology that generates optimal routes based on user input.

[1051] "Terminal means" refers to a device that allows a user to input and collect information about destinations and categories of interest.

[1052] "Display means" refers to technology for displaying the generated optimum route and information on related facilities on a user terminal.

[1053] "Emotion recognition means" refers to technology that analyzes the user's emotions and adjusts the information provided based on that information.

[1054] "Delivery operations" refers to a series of tasks related to the delivery and transportation of goods.

[1055] "Advice means" refers to technology for providing advice on appropriate responses and route adjustments based on the user's emotions.

[1056] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following describes an embodiment of the present invention.

[1057] The system is designed to help delivery personnel determine the optimal route in real time and respond appropriately based on customer sentiment. The system mainly consists of the following components:

[1058] Terminal means: A device such as a smartphone or tablet used by the delivery person, which allows the delivery person to input and collect information about the destination and categories of interest.

[1059] Server means: The server uses generation AI to generate an optimal route without obstacles or steps, retrieves information on facilities and tourist spots from a database, and uses emotion recognition means to analyze the user's emotional state and adjust the route and recommended information.

[1060] Display means: This is a technology that displays the generated optimal route and related facility information on the delivery person's device, allowing the delivery person to check the necessary information in real time.

[1061] Hardware and software used

[1062] Smartphone or tablet: A device used by delivery personnel to enter information and confirm routes.

[1063] Server: The backend system responsible for processing requests, retrieving information from databases, running generative AI, and sentiment analysis.

[1064] Generative AI model: An artificial intelligence technique for generating optimal routes based on user input.

[1065] Emotion recognition engine: Technology that analyzes the user's emotions and adjusts the information provided based on that information.

[1066] Overview of data processing and calculation

[1067] The server analyzes the user's request information received from the terminal means and calculates route information using the generative AI model. It also retrieves information on related facilities and tourist spots from a database and analyzes the user's emotions using an emotion recognition engine. The server then adjusts the route and recommended information based on the user's emotions and displays them on the terminal means.

[1068] Specific examples

[1069] For example, consider a case where a delivery person inputs, "When delivering a product from Shinjuku Station to Shibuya Station, I would like to stop by a cafe where I can refresh myself on the way." In this case, the server calculates the optimal route from Shinjuku Station to Shibuya Station and obtains information about cafes along the way. The emotion recognition engine then analyzes the customer's emotions, and if the customer is feeling stressed, for example, it will prioritize providing information about cafes where they can relax.

[1070] Prompt Sentence Examples

[1071] Enter the following prompt into the generative AI model:

[1072] "Analyze the user's image 'user_image.png' and suggest the route this user should take while traveling, as well as obstacles, steps, and relaxation facilities that should be considered. Additionally, provide advice on how the delivery person should respond if the user is feeling stressed."

[1073] This system allows delivery personnel to always have access to optimal route information and real-time updated information on related facilities, enabling them to respond appropriately to customer sentiment, thereby achieving efficient and effective delivery operations.

[1074] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1075] Step 1: User request input

[1076] The user uses the device to input the destination and the category of facility they want to stop at. This input information is formatted by the device and sent to the server. Specifically, the user inputs, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop at a cafe on the way."

[1077] Input: Destination and facility category (e.g. "From Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way.")

[1078] Output: The request data sent to the server

[1079] Step 2: Parsing the request on the server

[1080] The server analyzes the received request data and retrieves related information from the database, specifically route information to the destination and information on facilities along the way.

[1081] Input: Request data

[1082] Output: Route information and facility information

[1083] Step 3: Generative AI generates optimal route

[1084] The server uses generation AI to analyze the acquired route information and generate the optimal route without obstacles or steps. It also takes into account the acquired facility information to generate the optimal information to provide to the user.

[1085] Input: Route information and facility information

[1086] Output: Optimal path without obstacles or steps

[1087] Step 4: Adjustment with emotion recognition engine

[1088] The server uses an emotion recognition engine to detect and recognize the user's emotions. It analyzes emotions from images and other input data provided by the user and adjusts the optimal route and related information based on those emotions. For example, if the user is feeling stressed, it will prioritize information about relaxing cafes.

[1089] Input: User emotion data (e.g., face image)

[1090] Output: Adjusted route and facility information

[1091] Step 5: Send and display optimal information

[1092] The server sends the generated optimal route and related facility information to the terminal. The terminal displays the received information to the user. For example, it displays information such as, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a relaxing cafe near Yoyogi Station."

[1093] Input: Adjusted route and facility information

[1094] Output: Optimal route and facility information displayed on the terminal

[1095] Step 6: Real-time updates

[1096] The server periodically obtains the user's location information and updates route and facility information in real time. If necessary, it re-executes the generation AI and emotion recognition engine to generate new information. The device receives the new information, updates the display, and notifies the user.

[1097] Input: User location information, latest facility information

[1098] Output: Updated route and facility information

[1099] In this way, through a series of processing steps, the system enables delivery personnel to obtain the optimal route and related information in real time, providing service that is tailored to the customer's emotions.

[1100] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1101] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1102] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1103] [Fourth embodiment]

[1104] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1105] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1106] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1107] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1108] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1109] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1110] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1111] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1112] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1113] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1114] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1115] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1116] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1117] The present invention is a comprehensive information providing system that provides users with information on routes without obstacles or steps and related facilities. Specific embodiments of the system are described below.

[1118] System Overview

[1119] The system mainly consists of the following components:

[1120] 1. User terminal: A device used by users to input and view information. This includes smartphones and tablets.

[1121] 2. Server: Responsible for major backend processing such as receiving requests, communicating with the database, running generative AI, and providing generated information.

[1122] 3. Database: Stores data such as route information, facility information, tourist spots and event information, and provides information in response to inquiries from the server.

[1123] System Operation

[1124] 1. User Request Processing

[1125] User: Launches the application and inputs the destination and the category of facility they wish to visit (e.g., cafe, restroom, tourist spot, etc.).

[1126] Terminal: Sends the entered destination and category information to the server.

[1127] 2. Request Processing on the Server

[1128] Server: Receives and analyzes user requests. Based on the analyzed information, retrieves related route, facility, tourist attraction, and event information from the database.

[1129] 3. Generating optimal information using generative AI

[1130] Server: Using generation AI, it calculates the optimal route without obstacles or steps. It also takes into account the acquired facility and tourist spot information to generate the optimal information to provide to the user.

[1131] 4. Transmission and display of optimal information

[1132] Server: Sends the generated information to the user's terminal.

[1133] Terminal: Displays the received information so that the user can check it.

[1134] 5. Real-time information updates

[1135] Server: Updates the latest route and facility information in real time while the user is traveling, and re-executes the generation AI as necessary.

[1136] Terminal: Receives updated information in real time and updates the display.

[1137] Specific examples

[1138] User request input

[1139] User: Launches the app and enters, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way."

[1140] Device: Sends this information to the server.

[1141] Server-side processing

[1142] Server: Analyzes the request and retrieves route information from the database from Shinjuku Station to Shibuya Station, as well as information about cafes along the way.

[1143] Server: Using generative AI, it calculates the optimal route without obstacles or steps and suggests cafes based on the user's request.

[1144] Information generation and provision

[1145] Server: Sends the calculation results to the user's terminal.

[1146] Terminal: Displays optimal route and cafe information.

[1147] For example, the generation AI calculates, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. Along the way, we recommend the barrier-free cafe "Cafe A" near Yoyogi Station." and provides this to the user.

[1148] Real-time information updates

[1149] Server: Monitors user movements and periodically checks for changes in route and facility information.

[1150] On your device: If there are any changes, you will be notified and the new information will be displayed.

[1151] For example, if construction information is updated while you are traveling, real-time information such as "Due to construction work underway near Yoyogi Station, you will be changed to Cafe B, a cafe just before Yoyogi Station," will be provided.

[1152] In this way, a system is provided that allows users to reach their destination without stress and efficiently use facilities and tourist spots along the way.

[1153] The processing flow will be explained below.

[1154] Step 1:

[1155] User: Launches the application and enters authentication information on the login screen. The device accepts the user's input and moves to the next screen.

[1156] Step 2:

[1157] User: Enter the destination and the category of facilities you would like to stop at along the way. For example, you might enter, "I want to go from Shinjuku Station to Shibuya Station. I would like to stop at a cafe along the way."

[1158] Step 3:

[1159] Terminal: Generates request data for sending the input information to the server. The request data includes the user's input information (destination, category) and the user's ID.

[1160] Step 4:

[1161] Server: Receives and analyzes the request data sent from the user terminal. Extracts the destination, category information, and user ID.

[1162] Step 5:

[1163] Server: Accesses the database to obtain route information to the destination, as well as information about cafes that can be visited along the way.

[1164] Step 6:

[1165] Server: Using generation AI, the acquired route information is analyzed and an optimal route is generated without obstacles or steps. The optimal location is determined by taking into account information about cafes along the route.

[1166] Step 7:

[1167] Server: Integrates the generated optimal route and related cafe information and sends it to the user's device as a single data set.

[1168] Step 8:

[1169] Terminal: Analyzes the optimal route and cafe information received from the server and displays it to the user. For example, it displays information such as, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a barrier-free cafe near Yoyogi Station."

[1170] Step 9:

[1171] Server: Periodically obtains user location information, updates route information and facility information in real time, and re-executes the generation AI as necessary.

[1172] Step 10:

[1173] Server: If there are any changes to the latest route information or facility information, the server notifies the user device. For example, if construction work occurs on the route, the server regenerates new route and cafe information and sends it to the user.

[1174] Step 11:

[1175] Terminal: Receives updated information from the server and notifies the user while updating the display. For example, it may notify the user that "Due to construction work occurring near Yoyogi Station, the location will be changed to Cafe B, which is located just before Yoyogi Station."

[1176] This series of processing steps allows the user to always obtain the latest information on the optimal route without obstacles or steps, as well as information on related facilities, in real time.

[1177] Example 1

[1178] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1179] Conventional route guidance systems lacked sufficient functionality to avoid obstacles and steps, causing inconvenience to many users. Furthermore, because routes were not updated in real time, they were unable to respond quickly when new obstacles arose during travel. Furthermore, information about facilities and tourist spots that users might want to stop at along the way was often not provided properly.

[1180] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1181] In this invention, the server includes means for analyzing category information of destinations and desired facilities input from a user terminal, means for retrieving related route information and facility information from a database based on the analyzed information, means for calculating an optimal route free of obstacles and steps using a generative AI model and providing the optimal route to the user, means for transmitting and displaying the generated optimal route and related facility information to the user terminal, means for monitoring the user's location information while traveling and updating the route information and related facility information in real time, and means for retransmitting and displaying this data to the user terminal. This allows the user to reach their destination without stress and efficiently use facilities and tourist spots along the way.

[1182] A "user terminal" is a device that a user uses to input and view information, and includes smartphones and tablets.

[1183] A "server" is a back-end system that receives and analyzes user requests, obtains and generates relevant data, and sends it to the user terminal.

[1184] A "generative AI model" refers to an algorithm or program that uses machine learning or deep learning to generate optimal routes and related facility information in response to a user request.

[1185] The "destination" is the final destination specified by the user, and serves as the basis for the system to calculate the optimal route.

[1186] "Category information" is information indicating the type of intermediate point or facility the user desires to stop at, and examples include cafes and tourist spots.

[1187] "Route information" is data relating to detailed instructions and directions for a user to reach a destination.

[1188] "Facility information" refers to detailed data about restaurants, tourist spots, and other facilities located along or nearby the route.

[1189] "Analysis means" refers to the methods and functions for analyzing request information sent by users and extracting and organizing necessary data.

[1190] A "database" is a data management system that organizes and stores related data such as route information, facility information, and tourist spot information, and provides information in response to queries from the server.

[1191] "Real-time update" is a function that instantly updates route information and facility information in response to changes in the user's movement status and environment, and reflects that information on the user's terminal.

[1192] This invention is a comprehensive information provision system that provides users with information on obstacle-free and step-free routes and related facilities. This system consists of a user terminal, a server, a database, and a generative AI model.

[1193] System configuration

[1194] User device:

[1195] A device that allows users to input and view information. This includes smartphones and tablets. Specifically, Android and iOS devices are used.

[1196] server:

[1197] This is a backend system that receives user requests, analyzes them, and retrieves and generates relevant data. It uses the Python Flask framework to process data and handle API requests.

[1198] Database:

[1199] It stores data such as route information, facility information, and tourist spot information, and provides information in response to queries from the server. It uses a MySQL database.

[1200] Generative AI models:

[1201] This refers to algorithms and programs that use machine learning and deep learning to generate optimal routes and related facility information in response to user requests. Specifically, it uses GPT-4.

[1202] Program processing

[1203] The user uses the user device to launch the application and input their destination and the category of facility they would like to stop at (e.g., cafe, restroom, tourist spot, etc.). The user can input, "I want to go from Shinjuku Station to Shibuya Station. I would like to stop at a cafe on the way."

[1204] The terminal sends the entered information to the server, where the data is encoded in JSON format or similar and sent via the HTTPS protocol.

[1205] The server analyzes the received requests and separates the information by destination and category. It uses the Python Flask framework for analysis. Based on the analysis results, the server retrieves relevant route and facility information from a MySQL database.

[1206] Next, the server uses the acquired information to calculate the optimal route using a generative AI model. At this time, a prompt sentence is input to the AI ​​to generate a route without obstacles or steps. For example, "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps, along with information on barrier-free cafes along the way."

[1207] The generative AI model calculates the optimal route based on the prompt and generates related facility information. The server then sends the generated information to the user's device. The data is again encoded in JSON format and transmitted via HTTPS.

[1208] The user's device displays the received information. Specifically, it displays information such as the optimal route and cafes that can be stopped off along the way on a map and in guide text. For example, it displays information such as, "Take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, we recommend the barrier-free cafe 'Cafe A' near Yoyogi Station."

[1209] Furthermore, the server monitors the user's movement status and updates the latest route information and facility information in real time. For example, the generation AI recalculates information such as "Due to construction work near Yoyogi Station, change to Cafe B, a cafe just before Yoyogi Station," and sends it to the device. The user's device receives the updated information in real time and immediately updates the display.

[1210] As a result, users can reach their destination without stress and efficiently use facilities and tourist spots along the way.

[1211] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1212] Step 1:

[1213] The user launches the application using a user device such as a smartphone or tablet and inputs the destination and the category of facilities they would like to stop at along the way (e.g., cafe, restroom, tourist spot, etc.).

[1214] Input: Destination and facility category information (e.g., "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way.")

[1215] Output: The destination and category information are saved as input data.

[1216] Specifically, the user uses the device's touch screen to enter information into a text field.

[1217] Step 2:

[1218] The terminal encodes the data entered by the user into JSON format and sends it to the server using the HTTPS protocol.

[1219] Input: User-entered destination and category information

[1220] Output: Encoded JSON data sent to the server as an HTTPS request.

[1221] The terminal transmits data using the network interface.

[1222] Step 3:

[1223] The server analyzes the received requests and separates the information by destination and category, using the Python Flask framework for analysis.

[1224] Input: Encoded JSON data received from the terminal

[1225] Output: Parsed destination and category information

[1226] The server parses the request and executes functions to extract the information.

[1227] Step 4:

[1228] Based on the analyzed information, the server retrieves relevant route and facility information from a MySQL database.

[1229] Input: Parsed destination and category information

[1230] Output: Route information and facility information obtained from the database

[1231] The server generates SQL queries and executes them against a MySQL database.

[1232] Step 5:

[1233] The server uses the acquired information to calculate the optimal route using a generative AI model, and inputs prompts to the AI ​​to generate a route without obstacles or steps.

[1234] Input: Route and facility information obtained from the database, prompt text (e.g., "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps, along with information on barrier-free cafes along the way.")

[1235] Output: Optimal route and facility information generated by the generative AI model

[1236] The server passes the acquired data and prompt sentences to the AI ​​model and calculates the optimal route.

[1237] Step 6:

[1238] The server sends the generated information to the user's terminal. The data is again encoded in JSON format and sent via the HTTPS protocol.

[1239] Input: Generated optimal route and facility information

[1240] Output: JSON data encoded as an HTTPS request is sent to the terminal

[1241] The server sends the data to the device in the form of an HTTPS request.

[1242] Step 7:

[1243] The user terminal parses the received information and displays the optimal route and information on facilities to stop at on a map or in guide text.

[1244] Input: Encoded JSON data received from the server

[1245] Output: Optimal route and facility information displayed in a user-viewable format

[1246] The user device parses the JSON data and updates the UI components for map display and text display.

[1247] Step 8:

[1248] The server monitors the user's location information and updates the latest route and facility information in real time. If necessary, it re-executes the generation AI and sends the updated information to the device.

[1249] Input: Real-time location information of the user, information on changes in the environment (e.g., construction information)

[1250] Output: Updated route information and facility information is sent to the user terminal.

[1251] The server uses a location monitoring system, re-runs the AI ​​model as needed, and sends updated data to the device.

[1252] (Application example 1)

[1253] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1254] When an autonomous vehicle heads to its destination, the challenge is to provide the user with an optimal route free of obstacles and steps, and to provide real-time updates to the information, thereby ensuring a safe and comfortable journey. Another challenge is to provide information on facilities and tourist spots that the user may want to stop at along the way, and generate an optimal route that meets the user's needs.

[1255] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1256] In this invention, the server includes a generation AI means for generating an optimal route free of obstacles and steps, a terminal means for collecting information on the destination and categories of interest based on user input, a means for calculating and providing a safe route for the autonomous vehicle, a means for displaying the generated optimal route and information on related facilities on the user terminal, and a means for updating the route information and calculating new information in real time, thereby enabling safe and efficient route guidance for the autonomous vehicle when heading to the destination and enabling route and facility information to be updated in real time while the user is traveling.

[1257] "Generative AI" is an artificial intelligence technology that generates optimal routes free of obstacles and steps based on user input information.

[1258] The "terminal means" is a device operated by a user, and is a means for inputting and viewing information.

[1259] An "autonomous vehicle" is a vehicle that drives autonomously according to a calculated route.

[1260] "Means for calculating and providing a safe route" refers to a system that calculates and provides a safe route that avoids obstacles and steps for an autonomous vehicle to travel.

[1261] "Related facilities" are facilities suggested based on the user's interests and needs, such as cafes and tourist spots that can be visited along the route.

[1262] A "user terminal" is an information display device such as a smartphone carried by a user or an in-vehicle display.

[1263] "Means for updating in real time" refers to means for obtaining the latest data on route information and related facility information while the user is moving and updating it immediately.

[1264] The "server means" is a server system that performs major back-end processing such as receiving requests, accessing databases, and executing generated AI.

[1265] The system for implementing this invention mainly consists of a "generation AI" that generates an optimal route free of obstacles and steps, a "terminal" that collects information on destinations and categories of interest based on user input, a "server" that calculates and provides safe routes for autonomous vehicles, and a "user terminal" that displays the generated optimal route and information on related facilities on the user terminal. It also includes means for updating route information in real time and calculating new information.

[1266] System Operation

[1267] 1. Request input on user terminal:

[1268] The user uses a device such as a smartphone or an in-car display to input the destination and the category of facility they want to stop at (e.g., cafe, tourist spot, etc.). Once the user has entered this information, the device sends it to the server.

[1269] 2. Request processing on the server side:

[1270] The server analyzes the information sent by the user and retrieves relevant data from a database. It also uses generative AI to generate an optimal route free of obstacles and steps, providing a safe route for the autonomous vehicle. At the same time, it also retrieves and suggests information on stopover facilities according to the user's preferences.

[1271] 3. Display of generated information:

[1272] The server transmits the calculated optimal route and related facility information to the user terminal, which displays this information for the user to review.

[1273] 4. Real-time information updates:

[1274] The server constantly monitors the latest route and facility information while the user is traveling, and re-executes the generation AI to update the route as necessary. The updated information is sent to the user's device in real time, and the display is updated.

[1275] Specific examples

[1276] As a concrete example, consider the case where a user sends a request saying, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way." In this case, the device sends the request to the server. The server analyzes the request and retrieves route information from Shinjuku Station to Shibuya Station and information about cafes along the way from a database. Next, the generation AI calculates the optimal route without obstacles or steps, and suggests cafes that meet the user's request. For example, the generation AI might calculate, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. We recommend a barrier-free cafe near Yoyogi Station on the way," and provide this information to the user.

[1277] If a change occurs in the route while the user is traveling, the server recalculates the route based on the latest information and notifies the user's device, presenting the new route. For example, construction information may be updated while the user is traveling, providing real-time information such as "Because construction is underway near Yoyogi Station, change to a cafe just before Yoyogi Station."

[1278] Prompt Sentence Examples

[1279] An example of a prompt sentence is, "I'm going from Shinjuku Station to Shibuya Station, and I'd like to stop at a cafe on the way. Please tell me the safe route." By inputting this prompt sentence into a generative AI model, it is possible to calculate the optimal route and provide real-time navigation to the user.

[1280] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1281] Step 1:

[1282] The user uses the terminal to input their destination and categories of interest (e.g., cafes, tourist spots, etc.). The input information is sent from the terminal to the server. The input of this step is the user's request information, and the output is the request data sent to the server.

[1283] Step 2:

[1284] The server analyzes the request data received from the user. Based on the results of the analysis, a query is made to the database. Route information to the destination and information on related facilities are obtained from the database. The input to this step is the request data, and the output is the obtained route information and facility information.

[1285] Step 3:

[1286] The server uses the analyzed information and the generative AI model to generate an optimal route without obstacles or steps. It also recommends facilities along the route that meet the user's preferences. The input for this step is route information and facility information, and the output is the optimal route information and recommended facility information.

[1287] Step 4:

[1288] The server sends the generated optimal route and related facility information to the user terminal, which displays this information through an appropriate user interface. The input of this step is the optimal route information and recommended facility information, and the output is the information displayed on the user terminal.

[1289] Step 5:

[1290] While the user is traveling, the server monitors in real time whether the route information and related facility information have been updated. If necessary, it re-runs the generation AI to calculate the latest route. The input of this step is the real-time updated information, and the output is the recalculated, latest route information.

[1291] Step 6:

[1292] The server sends updated information to the user terminal in real time, and the user terminal updates the display. The input of this step is the recalculated latest route information, and the output is the updated display information of the user terminal.

[1293] The specific operations performed at each step include data collection, analysis, generation, transmission, and display, and the entire system works together to provide users with optimal route and facility information in real time.

[1294] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1295] This invention combines a comprehensive information provision system that provides users with information on routes without obstacles or steps and related facilities with a function that recognizes the user's emotions and adjusts the information accordingly. A specific embodiment of the system is described below.

[1296] System Overview

[1297] The system mainly consists of the following components:

[1298] 1. User terminal: A device used by users to input and view information. This includes smartphones and tablets.

[1299] 2. Server: Responsible for major backend processing such as receiving requests, communicating with the database, running generative AI, providing generated information, and analyzing user sentiment.

[1300] 3. Database: Stores data such as route information, facility information, tourist spots and event information, and provides information in response to inquiries from the server.

[1301] 4. Emotion engine: An engine that analyzes user emotions and utilizes that information throughout the system.

[1302] System Operation

[1303] 1. User Request Processing

[1304] User: Launches the application and enters authentication information on the login screen, then enters the destination and the category of facility they would like to stop at along the way (e.g., cafe, restroom, tourist spot, etc.).

[1305] Terminal: Generates and sends request data to send this information to the server.

[1306] 2. Request Processing on the Server

[1307] Server: Receives and analyzes user requests. Based on the requests, it retrieves relevant information from the database. The retrieved information includes route information to the destination and information on facilities along the way.

[1308] 3. Generating optimal information using generative AI

[1309] Server: Using generation AI, the acquired route information is analyzed and an optimal route without obstacles or steps is generated. The acquired facility information is also taken into consideration to generate optimal information to provide to the user.

[1310] 4. Emotional Engine Adjustment

[1311] Emotion Engine: Detects and recognizes the user's emotions and adjusts the optimal route and related information accordingly. For example, if the user is feeling stressed, it will prioritize relaxing cafes and wide roads.

[1312] 5. Transmission and display of optimal information

[1313] Server: Transmits the generated optimal route and related facility information to the user terminal.

[1314] Terminal: Displays the received information to the user. For example, it displays information such as, "The best route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a barrier-free cafe near Yoyogi Station."

[1315] 6. Real-time information updates

[1316] Server: Periodically acquires user location information and updates route and facility information in real time. If necessary, re-executes the generation AI and readjusts the emotion engine.

[1317] Terminal: Receives updated information in real time, updates the display and notifies the user.

[1318] Specific examples

[1319] User request input

[1320] User: Launches the app and enters, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop at a cafe on the way."

[1321] Device: Sends this information to the server.

[1322] Server-side processing

[1323] Server: Analyzes the request and retrieves route information from the database from Shinjuku Station to Shibuya Station and information on cafes along the way. Using generative AI, it calculates the optimal route without obstacles or steps, and suggests cafes taking into account the user's emotional state.

[1324] Emotional Engine Adjustment

[1325] Emotion engine: If the system detects that the user is in an emotional state that requires relaxation, it will prioritize providing information about quiet and relaxing cafes.

[1326] Information generation and provision

[1327] Server: Sends the calculation results to the user's terminal.

[1328] Terminal: Display the best route and cafe information. For example, display the following: "The best route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station."

[1329] Real-time information updates

[1330] Server: If construction information or other disruption information is updated during travel, the generation AI and emotion engine are re-run to generate new information.

[1331] Device: Receives new information and updates the display. For example, a notification will be sent saying, "Due to construction work occurring near Yoyogi Station, we have changed the location to Cafe B, a cafe just before Yoyogi Station."

[1332] This series of processes allows users to always obtain the latest information on the optimal route, free of obstacles and steps, as well as information on related facilities in real time.In addition, adjustments are made according to the user's emotional state, making travel more comfortable.

[1333] The processing flow will be explained below.

[1334] Step 1:

[1335] User: Launches the application and enters authentication information on the login screen. Once authentication is successfully completed on the device, the destination entry screen will be displayed.

[1336] Step 2:

[1337] User: Enter the destination and the category of facility you want to stop at (e.g., cafe, restroom, tourist spot, etc.). For example, you might enter, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop at a cafe on the way."

[1338] Step 3:

[1339] Terminal: Generates request data including the input destination and category information, and sends it to the server along with the user ID.

[1340] Step 4:

[1341] Server: Analyzes the received request data and extracts the user's destination and the requested category information. For example, "Shinjuku Station to Shibuya Station" is extracted as the destination information and "cafe" is extracted as the category information.

[1342] Step 5:

[1343] Server: Accesses the database and retrieves route information related to the extracted destination. It also retrieves information about cafes along the way.

[1344] Step 6:

[1345] Server: Launches the generation AI and analyzes the acquired route information. It generates an optimal route without obstacles or steps, taking into account cafe information that meets the user's requirements.

[1346] Step 7:

[1347] Server: Activates the emotion engine and analyzes the user's emotional state. For example, if it detects that the user is feeling stressed, it will prioritize quiet and relaxing cafes.

[1348] Step 8:

[1349] Server: Integrates the optimal route generated by the generation AI and the adjusted facility information generated by the emotion engine, and sends it to the user's device as a single data set.

[1350] Step 9:

[1351] Terminal: Analyzes the optimal route and related facility information received from the server and displays it to the user in a visually easy-to-understand format. For example, it displays, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station."

[1352] Step 10:

[1353] Server: Periodically collects user location information and monitors changes to route and facility information in real time. If new information is confirmed, the generative AI and emotion engine are re-run to update the information.

[1354] Step 11:

[1355] Server: Notifies the user device of the optimal route and facility information updated in real time. For example, if it detects that construction work has started at a station along the way, it calculates new route information and re-adjusts the cafe information.

[1356] Step 12:

[1357] Terminal: Receives updated information from the server and notifies the user. For example, the terminal may notify the user that "Due to construction work occurring near Yoyogi Station, change to Cafe B, a cafe just before Yoyogi Station," and display the new route and cafe information on the screen.

[1358] Through this series of processes, users can consistently obtain the latest information on optimal routes free of obstacles and steps, as well as information on related facilities that takes into account the user's emotional state.

[1359] Example 2

[1360] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1361] The present invention aims to provide a system that not only avoids obstacles and steps when a user moves, but also provides information on the optimal route and facilities to stop at based on the user's emotional state. Another objective is to provide a system that updates route information and related facility information in real time and provides it to the user, thereby realizing a comfortable journey based on the latest information at all times.

[1362] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means including a generation artificial intelligence for generating an optimal route free of obstacles and steps, terminal means for collecting information on destinations and categories of interest based on user input, means for displaying the generated optimal route and information on related facilities on the user terminal, emotion recognition means for recognizing the user's emotions and adjusting the route and related facility information based on the emotions, and means for updating and providing route information and related information in real time. This not only enables the user to use an optimal route free of obstacles and steps, but also provides information tailored to the user's emotional state at the time, enabling more comfortable and efficient travel. Furthermore, providing the latest information in real time allows the user to always obtain optimal information on the route and facilities to be visited.

[1363] An "optimal route without obstacles or steps" is a route that allows a user to reach their destination in the safest and most efficient manner without encountering any physical obstacles or steps when traveling.

[1364] "Generative artificial intelligence" refers to technology or systems that generate optimal routes and related information based on large amounts of data, and specifically refers to software that uses natural language processing and machine learning.

[1365] A "user terminal" is a device that a user uses to input and view information, and includes smartphones, tablets, personal computers, etc.

[1366] "Terminal means" refers to a device or application that has the function of collecting information based on user input and sending it to a server.

[1367] "Emotion recognition means" refers to technology or systems that analyze a user's facial expressions, voice, and other physiological data to determine the user's emotional state at that time.

[1368] "Means for updating and providing in real time" refers to technologies and systems that periodically acquire user location information and instantly generate and provide new route information and related facility information.

[1369] This invention combines a comprehensive information provision system that provides users with information on routes without obstacles or steps and related facilities, with a function that recognizes the user's emotions and adjusts the information based on those emotions. Specific embodiments for carrying out the invention are described below.

[1370] The system mainly consists of the following components:

[1371] 1. User terminal: A device used by a user to input and view information. This includes smartphones and tablets.

[1372] 2. Server: Responsible for major backend processing such as receiving requests, communicating with the database, running generative AI, providing generated information, and analyzing user sentiment.

[1373] 3. Database: Stores data such as route information, facility information, tourist spots, and event information, and provides information in response to inquiries from the server.

[1374] 4. Emotion Engine: An engine that analyzes user emotions and utilizes that information throughout the system. Specifically, it uses facial expression recognition and voice analysis technology.

[1375] System Operation

[1376] User Input

[1377] The user starts the application and enters their authentication information on the login screen. If successful, they enter their destination and the category of facilities they would like to stop at along the way (e.g., cafe, restroom, tourist spot). The device generates request data based on this information and sends it to the server.

[1378] Request processing on the server

[1379] The server analyzes the received request data, queries the database, and obtains route information to the destination and information on related facilities. The information obtained includes route information to the destination and information on facilities that can be stopped off along the way.

[1380] Generating optimal information using generative AI

[1381] The server calls a generation AI (e.g., ChatGPT) and analyzes the acquired route and facility information. The prompt text is input as follows: "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps. Also, please include information about cafes that can be stopped off along the way." The generation AI generates the optimal route and information about facilities to stop at based on the prompt text.

[1382] Emotional Engine Adjustment

[1383] The emotion engine (e.g., Emotion API) captures and analyzes the user's emotions via the camera or microphone. For example, if it determines that the user is feeling stressed, it will prioritize relaxing cafes and wide roads. The server will readjust the optimal route and facility information based on the emotion engine's results, ensuring that the most appropriate information is provided to the user.

[1384] Sending and displaying optimal information

[1385] The server sends the generated optimal route and related information to the user's device. The device analyzes the received information and displays it to the user. For example, it may display information such as, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station."

[1386] Real-time information updates

[1387] The server periodically obtains the user's location information and updates route information and related facility information in real time. It re-executes the generation AI and emotion engine to generate new information as needed. The device receives the updated information, keeps the display up to date, and notifies the user. For example, a notification may be sent saying, "Due to construction work occurring near Yoyogi Station, we will change to Cafe B, a cafe just before Yoyogi Station."

[1388] This allows users to always obtain the latest information on the optimal route without obstacles or steps, as well as information on related facilities, in real time.In addition, adjustments are made according to the user's emotional state, making travel more comfortable.

[1389] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1390] Step 1: User Input

[1391] The user starts the application and enters authentication information on the login screen. The authentication information is entered as a "user name" and "password." The device generates request data to send this authentication information to the server and sends it to the server. The input is the user authentication information, and the output is the authentication request data to the server.

[1392] Step 2: Authentication process

[1393] The server compares the received authentication information with the database, and if authentication is successful, the user is successfully logged in. The server sends the comparison result to the terminal. The input is authentication request data, and the output is response data indicating whether the login was successful or failed.

[1394] Step 3: User request input

[1395] The user inputs their destination and the category of facilities they would like to stop at along the way (e.g., cafe, restroom, tourist spot) into the app's interface. The device generates request data based on this input information and sends it to the server. The input is the destination and facility category information, and the output is the request data sent to the server.

[1396] Step 4: Request Parsing and Database Query

[1397] The server analyzes the received request data and queries the database. This query obtains route information to the destination and related facility information. Specifically, it obtains route information from Shinjuku Station to Shibuya Station and information about cafes along the way. The input is the request data, and the output is route information and facility information.

[1398] Step 5: Route generation by generative AI

[1399] The server analyzes the route information and facility information obtained using a generative AI model (e.g., ChatGPT) and generates an optimal route without obstacles or steps. It also generates information about facilities along the route. The prompt is entered as "Please provide the optimal route from Shinjuku Station to Shibuya Station without obstacles or steps. Please also include information about cafes along the way." The input is route information and the prompt, and the output is the optimal route and facility information.

[1400] Step 6: Emotional Engine Alignment

[1401] The emotion engine (e.g., Emotion API) captures and analyzes the user's facial expressions and voice via a camera and microphone. The emotion engine determines the user's emotional state, such as whether they are feeling stressed, and prioritizes locations such as relaxing cafes and wide roads. The input is facial expressions and voice data, and the output is the user's emotional state and adjusted route information.

[1402] Step 7: Sending the best information

[1403] The server sends the generated optimal route and related facility information to the user's terminal. The terminal analyzes the received information and displays it to the user. For example, the information displayed might be, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a quiet and relaxing cafe near Yoyogi Station." The input is the adjusted route information, and the output is the route and facility information displayed to the user.

[1404] Step 8: Update real-time information

[1405] The server periodically obtains the user's location information and updates route information and related facility information in real time. If necessary, it re-executes the generation AI and emotion engine to generate new information. The device receives the updated information, keeps the display up to date, and notifies the user. For example, it may notify the user that "due to construction work occurring near Yoyogi Station, change to Cafe B, a cafe just before Yoyogi Station." The input is location information and the latest environmental data, and the output is updated route and facility information.

[1406] (Application example 2)

[1407] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1408] In conventional delivery work, delivery personnel have limited means to quickly determine the optimal route to reach the destination while avoiding obstacles and steps. In addition, appropriate responses and route adjustments that take into account the emotional state of the customer at the delivery destination are rarely made, making improving customer satisfaction a challenge.

[1409] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1410] In this invention, the server includes means including a generation AI for generating an optimal route free of obstacles and steps, terminal means for collecting information on destinations and categories of interest based on user input, means for displaying the generated optimal route and information on related facilities on the user terminal, emotion recognition means for recognizing the user's emotions and adjusting the information provided based on the emotions, and means for providing emotion-based advice during delivery work. This enables delivery personnel to grasp the optimal route in real time and respond appropriately according to the customer's emotions.

[1411] "Generative AI" refers to artificial intelligence technology that generates optimal routes based on user input.

[1412] "Terminal means" refers to a device that allows a user to input and collect information about destinations and categories of interest.

[1413] "Display means" refers to technology for displaying the generated optimum route and information on related facilities on a user terminal.

[1414] "Emotion recognition means" refers to technology that analyzes the user's emotions and adjusts the information provided based on that information.

[1415] "Delivery operations" refers to a series of tasks related to the delivery and transportation of goods.

[1416] "Advice means" refers to technology for providing advice on appropriate responses and route adjustments based on the user's emotions.

[1417] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following describes an embodiment of the present invention.

[1418] The system is designed to help delivery personnel determine the optimal route in real time and respond appropriately based on customer sentiment. The system mainly consists of the following components:

[1419] Terminal means: A device such as a smartphone or tablet used by the delivery person, which allows the delivery person to input and collect information about the destination and categories of interest.

[1420] Server means: The server uses generation AI to generate an optimal route without obstacles or steps, retrieves information on facilities and tourist spots from a database, and uses emotion recognition means to analyze the user's emotional state and adjust the route and recommended information.

[1421] Display means: This is a technology that displays the generated optimal route and related facility information on the delivery person's device, allowing the delivery person to check the necessary information in real time.

[1422] Hardware and software used

[1423] Smartphone or tablet: A device used by delivery personnel to enter information and confirm routes.

[1424] Server: The backend system responsible for processing requests, retrieving information from databases, running generative AI, and sentiment analysis.

[1425] Generative AI model: An artificial intelligence technique for generating optimal routes based on user input.

[1426] Emotion recognition engine: Technology that analyzes the user's emotions and adjusts the information provided based on that information.

[1427] Overview of data processing and calculation

[1428] The server analyzes the user's request information received from the terminal means and calculates route information using the generative AI model. It also retrieves information on related facilities and tourist spots from a database and analyzes the user's emotions using an emotion recognition engine. The server then adjusts the route and recommended information based on the user's emotions and displays them on the terminal means.

[1429] Specific examples

[1430] For example, consider a case where a delivery person inputs, "When delivering a product from Shinjuku Station to Shibuya Station, I would like to stop by a cafe where I can refresh myself on the way." In this case, the server calculates the optimal route from Shinjuku Station to Shibuya Station and obtains information about cafes along the way. The emotion recognition engine then analyzes the customer's emotions, and if the customer is feeling stressed, for example, it will prioritize providing information about cafes where they can relax.

[1431] Prompt Sentence Examples

[1432] Enter the following prompt into the generative AI model:

[1433] "Analyze the user's image 'user_image.png' and suggest the route this user should take while traveling, as well as obstacles, steps, and relaxation facilities that should be considered. Additionally, provide advice on how the delivery person should respond if the user is feeling stressed."

[1434] This system allows delivery personnel to always have access to optimal route information and real-time updated information on related facilities, enabling them to respond appropriately to customer sentiment, thereby achieving efficient and effective delivery operations.

[1435] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1436] Step 1: User request input

[1437] The user uses the device to input the destination and the category of facility they want to stop at. This input information is formatted by the device and sent to the server. Specifically, the user inputs, "I want to go from Shinjuku Station to Shibuya Station. I'd like to stop at a cafe on the way."

[1438] Input: Destination and facility category (e.g. "From Shinjuku Station to Shibuya Station. I'd like to stop by a cafe on the way.")

[1439] Output: The request data sent to the server

[1440] Step 2: Parsing the request on the server

[1441] The server analyzes the received request data and retrieves related information from the database, specifically route information to the destination and information on facilities along the way.

[1442] Input: Request data

[1443] Output: Route information and facility information

[1444] Step 3: Generative AI generates optimal route

[1445] The server uses generation AI to analyze the acquired route information and generate the optimal route without obstacles or steps. It also takes into account the acquired facility information to generate the optimal information to provide to the user.

[1446] Input: Route information and facility information

[1447] Output: Optimal path without obstacles or steps

[1448] Step 4: Adjustment with emotion recognition engine

[1449] The server uses an emotion recognition engine to detect and recognize the user's emotions. It analyzes emotions from images and other input data provided by the user and adjusts the optimal route and related information based on those emotions. For example, if the user is feeling stressed, it will prioritize information about relaxing cafes.

[1450] Input: User emotion data (e.g., face image)

[1451] Output: Adjusted route and facility information

[1452] Step 5: Send and display optimal information

[1453] The server sends the generated optimal route and related facility information to the terminal. The terminal displays the received information to the user. For example, it displays information such as, "The optimal route is to take the Yamanote Line from JR Shinjuku Station and get off at Shibuya Station. On the way, stop by Cafe A, a relaxing cafe near Yoyogi Station."

[1454] Input: Adjusted route and facility information

[1455] Output: Optimal route and facility information displayed on the terminal

[1456] Step 6: Real-time updates

[1457] The server periodically obtains the user's location information and updates route and facility information in real time. If necessary, it re-executes the generation AI and emotion recognition engine to generate new information. The device receives the new information, updates the display, and notifies the user.

[1458] Input: User location information, latest facility information

[1459] Output: Updated route and facility information

[1460] In this way, through a series of processing steps, the system enables delivery personnel to obtain the optimal route and related information in real time, providing service that is tailored to the customer's emotions.

[1461] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1462] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1463] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1464] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1465] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1466] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1467] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1468] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1469] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1470] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1471] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1472] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1473] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1474] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1475] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1476] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1477] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1478] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1479] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1480] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1481] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1482] The following is further disclosed regarding the above embodiment.

[1483] (Claim 1)

[1484] a generating AI for generating an optimal path that is free of obstacles and steps;

[1485] terminal means for collecting destination and interest category information based on user input;

[1486] The system includes means for displaying the generated optimum route and information on related facilities on a user terminal.

[1487] (Claim 2)

[1488] 2. The system according to claim 1, further comprising a server means for acquiring information on related facilities and tourist spots from a database based on input information.

[1489] (Claim 3)

[1490] 2. The system according to claim 1, further comprising server means for updating and providing route information and related information in real time while the user is traveling.

[1491] "Example 1"

[1492] (Claim 1)

[1493] A server means for analyzing category information of a destination and a desired facility input from a user terminal;

[1494] means for retrieving related route information and facility information from a database based on the analyzed information;

[1495] A means for calculating an optimal route free of obstacles and steps using a generative AI model and providing it to the user;

[1496] means for transmitting the generated optimum route and related facility information to a user terminal and displaying the information;

[1497] a means for monitoring the location information of a user while moving and updating route information and related facility information in real time;

[1498] means for retransmitting and displaying these data to a user terminal;

[1499] A system including:

[1500] (Claim 2)

[1501] The system of claim 1 uses a generative AI model to create prompt sentences and generate an optimal route free of obstacles and steps.

[1502] (Claim 3)

[1503] 10. The system of claim 1, including server means for monitoring user movement in real time, re-running the generative AI model as needed, and updating route and facility information.

[1504] "Application Example 1"

[1505] (Claim 1)

[1506] a generating AI for generating an optimal path that is free of obstacles and steps;

[1507] terminal means for collecting destination and interest category information based on user input;

[1508] means for calculating and providing a safe route for the autonomous vehicle;

[1509] means for displaying the generated optimum route and information on related facilities on a user terminal;

[1510] means for updating route information and calculating new information in real time;

[1511] A system including:

[1512] (Claim 2)

[1513] 2. The system according to claim 1, further comprising a server means for acquiring information on related facilities and tourist spots from a database based on input information.

[1514] (Claim 3)

[1515] 2. The system according to claim 1, further comprising server means for updating and providing route information and related information in real time while the user is traveling.

[1516] "Example 2: Combining Emotion Engines"

[1517] (Claim 1)

[1518] means including generative artificial intelligence for generating an optimal path free of obstacles and steps;

[1519] terminal means for collecting destination and interest category information based on user input;

[1520] means for displaying the generated optimum route and information on related facilities on a user terminal;

[1521] emotion recognition means for recognizing the user's emotion and adjusting the route and related facility information accordingly;

[1522] means for updating and providing route and related information in real time;

[1523] A system including:

[1524] (Claim 2)

[1525] 2. The system according to claim 1, further comprising a server means for acquiring information on related facilities and tourist spots from a database based on input information.

[1526] (Claim 3)

[1527] 2. The system according to claim 1, further comprising server means for updating and providing route information and related information in real time while the user is traveling.

[1528] "Application example 2 when combining emotion engines"

[1529] (Claim 1)

[1530] a generating AI for generating an optimal path that is free of obstacles and steps;

[1531] terminal means for collecting destination and interest category information based on user input;

[1532] means for displaying the generated optimum route and information on related facilities on a user terminal;

[1533] emotion recognition means for recognizing an emotion of a user and adjusting information to be provided based on the emotion;

[1534] A means of providing emotion-based advice in delivery operations;

[1535] A system including:

[1536] (Claim 2)

[1537] 2. The system according to claim 1, further comprising a server means for acquiring information on related facilities and tourist spots from a database based on input information.

[1538] (Claim 3)

[1539] 2. The system according to claim 1, further comprising server means for updating and providing route information and related information in real time while the user is traveling. [Explanation of symbols]

[1540] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a generating AI for generating an optimal path that is free of obstacles and steps; terminal means for collecting destination and interest category information based on user input; and means for displaying the generated optimum route and information on related facilities on a user terminal.

2. 2. The system according to claim 1, further comprising server means for acquiring information on related facilities and tourist spots from a database based on input information.

3. 2. The system according to claim 1, further comprising server means for updating and providing route information and related information in real time while the user is traveling.

Images