System

Abstract

To provide a system for providing a user with a highly convenient, efficient and comfortable moving experience.SOLUTION: A system comprising: means for receiving information on a departure place and a destination input by a user; means for calculating an optimal travel route based on past travel data and traffic conditions; means for analyzing traffic cameras and public transportation data in real time; means for generating a travel route by combining optimal travel means; means for acquiring store discount information on the travel route and including the information in a route proposal; means for notifying the user of the proposed travel route and discount information; and means for processing a fare of a travel means by using an integrated settlement system.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional mobility assistance systems have difficulty providing optimal travel routes tailored to user needs, and are also insufficient in integrating real-time traffic conditions and discount information. As a result, users often face delays and traffic congestion while traveling, and even miss discount information along their travel route. The present invention aims to solve these problems and provide a system that provides users with a highly convenient, efficient, and comfortable travel experience. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including means for receiving information on departure and destination points input by a user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for analyzing data from traffic cameras and public transport in real time, means for combining optimal travel modes to generate a travel route, means for obtaining store discount information along the travel route and including it in the route proposal, means for notifying the user of the proposed travel route and discount information, and means for processing travel mode fares using an integrated payment system.

[0006] "User" refers to a person who uses the system.

[0007] "Start point" refers to the location where the user begins their journey.

[0008] "Destination" refers to the location where the user ends their journey.

[0009] "Means for receiving information" refers to the function for taking data entered by the user into the system.

[0010] "Past movement data" refers to historical information about movements made by the user in the past.

[0011] "Traffic conditions" refers to current traffic flow, congestion status, delay information, etc.

[0012] The "optimal travel route" refers to the route that best suits the user's travel requirements.

[0013] A "traffic camera" is a device installed on a road that captures traffic conditions in real time.

[0014] "Public transport" refers to means of transport such as buses, trains and taxis used by the general public.

[0015] "Means for analyzing data in real time" refers to the function of instantly processing acquired data on the spot and extracting the necessary information.

[0016] "Transportation" refers to the means (bus, train, taxi, bike share, etc.) used by a user to travel from their departure point to their destination.

[0017] "Means for generating a travel route" refers to a function for calculating the optimal route based on the user's request.

[0018] "Store discount information" refers to information about discounts and special offers offered by stores along the travel route.

[0019] "Means for including in route suggestions" refers to the ability to incorporate additional information (such as discounts or special offers) into the generated travel route.

[0020] "Means of notification" refers to the methods and functions for conveying information to the user.

[0021] An "integrated payment system" refers to a system that centrally manages multiple payment methods and allows users to make payments easily.

[0022] "Means for processing fees" refers to the functionality for settling the cost of the transportation used by the user and completing the transaction.

[0023] The term "system" refers to a device or program that integrates the above functions and supports the user's mobility as a whole. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0025] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0026] First, the terms used in the following description will be explained.

[0027] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0028] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0029] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0030] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0031] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0032] [First embodiment]

[0033] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0034] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0037] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0040] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0041] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0042] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0043] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0044] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0045] The present invention is a system designed to assist a user in efficient movement, and is implemented as follows.

[0046] 1. Acceptance of customer requests

[0047] Device: The user launches a dedicated mobility assistance application and inputs their starting point and destination. For example, if the user inputs "Station A" as the starting point and "Station B" as the destination, the input data is structured within the app.

[0048] Server: Receives the departure and destination information sent from the device and stores it in a database. This stored data is used in subsequent processes.

[0049] 2. AI-based request analysis

[0050] Server: Retrieves request data from the database and passes it to an AI algorithm, which calculates the optimal route in real time based on past travel data and traffic conditions.

[0051] Server: For example, it learns from past data which means of transportation a user prefers depending on the time of day, and presents routes based on that, taking into account traffic congestion and weather conditions.

[0052] 3. Use of video analysis

[0053] Server: Acquires the latest data from traffic cameras and public transport systems and analyzes it in real time, using image processing technology and machine learning algorithms.

[0054] Server: For example, it analyzes images from traffic cameras installed at major intersections and stations and obtains congestion and delay information in real time, making it possible to take measures such as avoiding routes with heavy traffic.

[0055] 4. Utilizing MaaS

[0056] Server: Integrates AI analysis results with real-time video analytics information to generate optimal travel combinations, including train transfers, bus trips, and taxi reservations.

[0057] Server: Automatically makes the necessary reservations and ticket arrangements. For example, if the system suggests a route that combines buses and trains, it purchases bus tickets on behalf of the user.

[0058] 5. Route suggestions

[0059] Server: Generates the optimal travel route and discount information for stores along that route, and notifies the user.

[0060] On the device: A notification appears in the user's app, showing suggested routes and store discount information. For example, the notification might say, "The best route from Station A to Station B is 30 minutes by train. There's also a 10% discount at the cafe near Station B."

[0061] 6. ONEID Payment Execution

[0062] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay," which enters the user's payment information.

[0063] Server: Sends payment information to the integrated payment system and processes the transportation fare. After receiving a successful response, sends the ticket information for each transportation to the user's terminal.

[0064] Examples:

[0065] Terminal: The user inputs "home" as the departure point and "office" as the destination. The desired departure time is "8:00 AM."

[0066] Server: Passes the request data to an AI algorithm and calculates the optimal route: "bus → train → walk." Video analysis also confirms bus congestion.

[0067] Server: Automatically book a bus ticket for 8am based on this route.

[0068] Device: Notifies the user of the optimal route from home to office, along with discount information for a cafe near the office.

[0069] Terminal: The user confirms the route and the fare is processed in bulk through the ONEID payment system. Ticket information is received, allowing for smooth travel.

[0070] The above is an embodiment of the present invention. This system enables users to travel stress-free and efficiently, and also allows users to take advantage of advantageous information along their travel route without missing it.

[0071] The processing flow will be explained below.

[0072] Step 1:

[0073] Device: The user launches the dedicated app and enters their departure and destination. The departure point is entered as "Station A" and the destination as "Station B." The user then selects "Now" as the departure time and presses the "Send Request" button.

[0074] Step 2:

[0075] Server: Receives the request data sent from the terminal. The received data includes information on the departure point (Station A), destination (Station B), and departure time (Now). The server stores this data in a database.

[0076] Step 3:

[0077] Server: Retrieves request data from the database and passes it to the AI ​​algorithm. The AI ​​calculates the optimal route based on past travel data and traffic conditions. For example, if it determines from past data that the user tends to avoid crowded areas, it will prioritize routes that avoid crowds.

[0078] Step 4:

[0079] Server: Obtains the latest information from traffic cameras and public transportation. Obtains real-time video data from the traffic camera API and analyzes traffic congestion information using video analytics technology. For example, it detects congestion at major intersections from traffic camera footage.

[0080] Step 5:

[0081] Server: Combines the results of AI analysis with the results of real-time video analysis to find the optimal means of transportation. For example, it generates the optimal route combining trains and buses.

[0082] Step 6:

[0083] Server: Makes the necessary reservations and ticket arrangements. For example, automatically reserves train tickets and arranges bus tickets at the same time. In this case, the reservation is made using the public transport company's API.

[0084] Step 7:

[0085] Server: Obtains the optimal travel route and discount information for stores along the route, and generates route suggestions. Suggestions include information such as "Take the train from station A to station B, and get a 10% discount coupon at a cafe along the way."

[0086] Step 8:

[0087] On the device: A notification appears in the user's app. The notification includes a suggested route and store discount information. For example, "The best route is a 30-minute train ride, and there's a 10% off coupon at a cafe along the way."

[0088] Step 9:

[0089] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay."

[0090] Step 10:

[0091] Server: Sends payment information to the integrated payment system. The payment system processes the user's credit card information and other details, and pays the fee in one lump sum. A response indicating successful payment is returned to the server, which then sends the response to the user's device.

[0092] Step 11:

[0093] Terminal: The user's terminal will be notified that the fare for each mode of transportation has been paid. Ticket information for each mode of transportation will also be displayed. The user can use this information to smoothly start their journey.

[0094] The above are the specific processing steps of the program for implementing the present invention. This system enables users to travel efficiently and comfortably, and also allows them to enjoy additional benefits along their travel route.

[0095] Example 1

[0096] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0097] Conventional mobility assistance systems require users to select the optimal route from their departure point to their destination and reserve and arrange multiple modes of transportation, which is cumbersome and hinders efficient travel. Furthermore, the lack of optimal route suggestions that reflect real-time traffic conditions and congestion information makes it difficult for users to travel as planned, and they often miss out on discount information that can be used during travel. To solve these issues, a system is needed that automatically proposes and arranges efficient travel routes, taking into account past travel data and real-time traffic conditions.

[0098] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0099] In this invention, the server includes means for receiving information on departure and destination entered by the user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for acquiring and analyzing data from traffic cameras and public transportation in real time, means for generating a travel route by combining optimal means of transportation and automatically making reservations and ticket arrangements, means for acquiring store discount information along the travel route and including it in the route proposal, means for notifying the user of the proposed travel route and discount information, means for processing travel fares and transmitting payment information using an integrated payment system, and means for distributing ticket information for the user's means of transportation in real time. This allows users to efficiently select and arrange travel routes without stress, and enables optimal travel taking into account real-time congestion information and discount information.

[0100] "User" refers to an individual or organization that uses the services of the mobility assistance system.

[0101] "Start point" refers to the point where the user starts their journey.

[0102] "Destination" refers to the point where the user ends their journey.

[0103] A "travel route" refers to a route from a departure point to a destination, and may include a route that combines multiple modes of transportation.

[0104] "Past journey data" refers to data relating to journeys taken in the past by the user or other users, including information such as preferred modes of transportation and time of day.

[0105] "Traffic conditions" refers to factors that affect travel, such as the current congestion situation on roads and public transportation, delay information, and accident information.

[0106] "Traffic cameras" refer to cameras installed to monitor congestion and congestion on roads and public transport.

[0107] "Public transport" refers to means of transport available to the general public, such as buses, trains and taxis.

[0108] "Real time" refers to ongoing situations and information, and indicates the exact moment an event occurs.

[0109] "Analysis" refers to procedures and methods for processing acquired data to facilitate the extraction of specific information.

[0110] "Reservation" refers to the act of a user promising to use something at a later date, including arranging transportation or services.

[0111] "Ticket" means a permit or ticket required to use a particular means of transportation or service.

[0112] "Store discount information" refers to information about discounts and special offers offered by stores along the route.

[0113] "Notification" refers to providing information to users, and is often done through smartphone applications.

[0114] An "integrated payment system" refers to a system for processing payments for different services and products in one place.

[0115] "Fare" means the amount paid for using a means of transportation or service.

[0116] "Payment information" refers to information related to a user's payment, including credit card information and electronic money details.

[0117] "Transportation ticket information" refers to the tickets and reservation information required to use transportation.

[0118] A "system" refers to a mechanism in which multiple elements work together to achieve a specific purpose.

[0119] The system is designed to efficiently support users' mobility and is implemented using multiple components, including hardware and software such as user devices, a central server, traffic cameras, databases, AI algorithms, and an integrated payment system.

[0120] User terminal

[0121] Users use a smartphone or tablet with a dedicated mobility support application installed. This application provides an interface for users to input their departure and destination locations. Payment information and notifications are also entered on this device.

[0122] Central Server

[0123] The server plays a central role in the system. It receives the origin and destination information sent by the user and stores it in a database. It then retrieves the stored data and passes it to an AI algorithm to calculate the optimal travel route.

[0124] Database

[0125] The database stores information such as user request data, past movement data, and traffic conditions, allowing the server to quickly retrieve and process the necessary data.

[0126] AI algorithms

[0127] The AI ​​algorithm calculates the optimal route based on past travel data and real-time traffic conditions. This algorithm learns the user's travel habits and preferences and can suggest the optimal means of transportation for each time of day. For example, it may suggest a "bus → train → walk" route during the morning commute.

[0128] Traffic Camera

[0129] Traffic cameras are installed at major intersections and stations to monitor traffic conditions in real time. The server acquires video data from these cameras and analyzes it using image processing technology. The analysis results include congestion status and delay information.

[0130] Integrated Payment System

[0131] The integrated payment system is used to process all the fares for the travel route selected by the user. When the user selects "Confirm route and pay," the server sends the payment information to the integrated payment system and processes the fare.

[0132] Route suggestions and notifications

[0133] The server generates an optimal route based on the results of AI algorithms and video analysis, and also integrates discount information for stores along the route. This information is sent to the user's device, where the user can check the proposed route and discount information. For example, a notification may appear saying, "The optimal route from Station A to Station B is 30 minutes by train. There is also a 10% discount at the cafe near Station B."

[0134] Specific examples

[0135] As a concrete example, consider the case where a user inputs "home" as the departure point, "office" as the destination, and the desired departure time as "8:00 AM." The server passes the request data to an AI algorithm, which calculates the optimal route: "bus → train → walk." It also analyzes traffic camera footage to check bus congestion. Based on this route, the system automatically reserves a bus ticket for 8:00 AM. The user's device is notified of the optimal route from "home to office," along with discount information for a cafe near the office. Finally, the user confirms the route and the fare is processed in one lump sum via the ONEID payment system. The ticket information is then received by the device, allowing for a smooth journey.

[0136] The above is a specific embodiment of the present invention. This system allows users to efficiently select and arrange travel routes without stress, and enables optimal travel taking into account real-time congestion information and discount information.

[0137] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0138] Step 1:

[0139] The user starts a dedicated mobility assistance application and inputs their starting point and destination. The input information is converted into JSON format such as {"start":"Station A", "destination":"Station B"}. Input: Starting point and destination from the user. Output: Structured JSON format data.

[0140] Step 2:

[0141] The terminal sends structured JSON data to the server. The HTTPS protocol is used for transmission, and data is encrypted. Input: Structured JSON data. Output: Transmission to server.

[0142] Step 3:

[0143] The server saves the received JSON data in the database. The specific save operation uses an SQL insert statement. Input: JSON data from the terminal. Output: Request information saved in the database.

[0144] Step 4:

[0145] The server retrieves the request data from the database and passes it to the AI ​​algorithm. It retrieves the data using an SQL select statement. Input: Request data from the database. Output: Request data passed to the AI ​​algorithm.

[0146] Step 5:

[0147] The AI ​​algorithm on the server calculates the optimal travel route based on past travel data and traffic conditions. It uses a specific machine learning model (e.g., neural network). Input: Request data, past travel data, and traffic condition data. Output: Optimal travel route information.

[0148] Step 6:

[0149] The server stores the calculation results in temporary storage. For the storage operation, an in-memory database such as Redis or Memcached is used. Input: Optimal travel route information. Output: Travel route information stored in temporary storage.

[0150] Step 7:

[0151] The server retrieves the latest data from traffic cameras and public transport. It uses API calls to retrieve real-time data. Input: API call. Output: Latest data from traffic cameras and public transport.

[0152] Step 8:

[0153] The server analyzes the traffic camera video data using image processing technology. It uses a deep learning model (e.g., YOLO). Input: Traffic camera video data. Output: Analyzed congestion and delay information.

[0154] Step 9:

[0155] The server stores the analysis results in temporary storage. Input: Congestion status and delay information. Output: Analysis results stored in temporary storage.

[0156] Step 10:

[0157] The server combines the results of AI analysis and video analysis to generate the optimal combination of transportation methods. The algorithm uses a linear optimization model. Input: Optimal transportation route information and analysis results. Output: Optimal combination of transportation methods.

[0158] Step 11:

[0159] The server automatically makes the necessary reservations and arranges tickets. It uses API calls to connect with bus and train reservation systems. Input: Information on optimal combinations of transportation methods. Output: Information on completed reservations and ticket arrangements.

[0160] Step 12:

[0161] The server saves the reservation and arrangement results in temporary storage. Input: Completed reservation and ticket arrangement information. Output: Reservation and arrangement information saved in temporary storage.

[0162] Step 13:

[0163] The server generates the optimal travel route and store discount information along that route. The store discount information is obtained from an external promotion API. Input: Optimal travel route information. Output: Combination of travel route and store discount information.

[0164] Step 14:

[0165] The server sends the generated information to the user's device. The HTTPS protocol is used for communication. Input: Travel route and store discount information. Output: Notification to the user's device.

[0166] Step 15:

[0167] The device displays the notification to the user. Specifically, it uses the notification function within the app to display information on the screen. Input: Notification information from the server. Output: Visualized notification to the user.

[0168] Step 16:

[0169] The user selects "Confirm route and pay." The entered payment information is encrypted on the terminal. Input: User's payment information. Output: Encrypted payment information.

[0170] Step 17:

[0171] The terminal sends encrypted payment information to the server. The HTTPS protocol is used for communication. Input: Encrypted payment information. Output: Send to server.

[0172] Step 18:

[0173] The server sends the payment information to the integrated payment system and processes the travel fare. Input: Encrypted payment information. Output: Transmission to the integrated payment system and fare processing.

[0174] Step 19:

[0175] The server receives a success response indicating that the fare processing was successful and sends the ticket information for each means of transportation to the user's terminal. Input: Success response from the integrated payment system. Output: Ticket information sent to the user's terminal.

[0176] Step 20:

[0177] The device displays the ticket information to the user. Specifically, it uses the ticket management function within the app. Input: Ticket information from the server. Output: Ticket information displayed to the user.

[0178] (Application example 1)

[0179] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0180] In recent years, advances in autonomous driving technology have improved the convenience of user travel. However, there are still problems with finding optimal travel routes and updating routes in real time according to traffic congestion and weather conditions. Furthermore, there is a lack of personalized travel routes that meet individual user preferences, so there is room for improvement in order to provide an optimal travel experience. Furthermore, it is necessary to improve user convenience by utilizing discount information and services during travel.

[0181] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0182] In this invention, the server includes means for receiving information on departure and destination points input by a user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for analyzing data from traffic cameras and public transportation in real time, means for combining optimal travel modes to generate a travel route, means for acquiring discount information for stores along the travel route and including it in a route proposal, means for notifying the user of the proposed travel route and discount information, means for processing travel mode fares using an integrated payment system, means for updating the route in real time using an application installed in the autonomous vehicle, means for customizing the route based on user preferences, and means for taking traffic congestion and weather conditions into consideration. This allows users to obtain the optimal travel route based on real-time information, enjoy a personalized travel experience tailored to their individual preferences, and also take advantage of discount information along the travel route.

[0183] The "means for receiving information on the departure point and destination" refers to a device or software that acquires the information on the departure point and destination entered by the user and transmits it to the server.

[0184] A "means for calculating an optimal travel route" refers to an algorithm or program that calculates the most efficient and quickest travel route based on past travel data and traffic conditions.

[0185] "Means for analyzing data from traffic cameras and public transportation in real time" refers to technology and devices that analyze data obtained from traffic cameras and public transportation in real time to determine traffic congestion and delay information.

[0186] "Means for generating a travel route by combining optimal means of transportation" refers to a system or software for combining multiple means of transportation to generate the most suitable travel route for a user.

[0187] "Means for acquiring discount information of stores along a travel route and including it in a route proposal" refers to technology or devices that acquire discount information of stores along a travel route and reflect it in a travel route proposal.

[0188] The "means for notifying the user of the proposed travel route and discount information" refers to a system or application for notifying the user of the optimal travel route and discount information related to that route.

[0189] "Means for processing transportation fares using an integrated payment system" refers to an integrated payment system for processing fares for multiple transportation means in one go.

[0190] "Means for updating routes in real time through an application installed on an autonomous vehicle" refers to technology or devices for updating travel routes in real time through an application installed on an autonomous vehicle.

[0191] "Means for customizing routes based on user preferences" means a system or algorithm for providing personalized travel routes based on a user's past travel history and current preferences.

[0192] "Means that take into account traffic congestion and weather conditions" refers to technologies and devices that acquire and analyze traffic congestion and weather conditions in real time and provide optimal travel routes based on that information.

[0193] The present invention is a system designed to assist users in traveling efficiently, and is implemented as follows using an application installed in an autonomous vehicle.

[0194] First, the user launches a dedicated mobility assistance application and inputs their departure and destination. This information is sent from the user's device to the server and stored in a database. The server then calculates the optimal route based on the received departure and destination information, taking into account past travel data and real-time traffic conditions.

[0195] The calculated route is further optimized by analyzing data from traffic cameras and public transport in real time. The server analyzes the video data acquired in real time and adjusts the route based on congestion and delay information. This is done using Python and OpenCV video analysis.

[0196] Once the optimal combination of transportation modes has been determined, the server automatically makes the necessary reservations and ticket arrangements using MaaS (Mobility as a Service), and processes fares through an integrated payment system. Once all arrangements are complete, the user is notified of the proposed route and discount information for stores along the route.

[0197] One of the features of this system is that the application installed in the autonomous vehicle updates the route in real time. The server takes into account traffic congestion and weather conditions in real time and provides the autonomous vehicle with the optimal route.

[0198] Additionally, it also has the ability to customize routes based on user preferences, which takes into account the user's past travel history and current preferences to provide a more personalized travel experience.

[0199] For example, if a user inputs "home" as the departure point and "sports gym" as the destination, setting the departure time to "5:00 PM," the server will calculate a "bus → train → walk" route based on past travel data and real-time traffic information. The server then analyzes traffic camera footage to check bus congestion and automatically reserves a bus ticket for 5:00 PM. The user receives this route and discount information for cafes near the gym via the app, and the fare is processed in one lump sum via the ONEID payment system.

[0200] An example prompt might be, "Based on the user's input of origin and destination, suggest the best route for travel, taking into account traffic congestion and weather conditions. Also, provide personalized suggestions based on the user's preferences and offer discount information along the route."

[0201] As described above, the present invention is a system that can provide users with efficient, personalized, and optimal travel routes by using autonomous vehicles.

[0202] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0203] Step 1:

[0204] The user starts a dedicated mobility assistance application and inputs their starting point and destination. The starting point (e.g., "home") and destination (e.g., "sports gym") information entered by the user is sent from the terminal to the server.

[0205] Step 2:

[0206] The server receives the departure and destination information and stores it in a database. The server then retrieves past travel data and real-time traffic information from the database and calculates the optimal travel route. The input here is the departure and destination information entered by the user, as well as the past travel data and traffic information in the database, and the output is the optimal travel route.

[0207] Step 3:

[0208] The server acquires real-time data from traffic cameras and public transport and uses video analysis to determine traffic congestion and delay information. The server then uses this data to further optimize the optimal travel route. Specifically, it uses Python and OpenCV to analyze the video data and detect congestion and delays. The input is real-time data, and the output is an updated optimal travel route.

[0209] Step 4:

[0210] The server utilizes MaaS (Mobility as a Service) to combine optimal means of transportation and automatically make the necessary reservations and ticket arrangements. The server processes all transportation fares in one go through an integrated payment system. The input is the optimal travel route and reservation information, and the output is reservation confirmation and payment completion information.

[0211] Step 5:

[0212] The server notifies the user of the optimal travel route and discount information for stores along the route. This includes a function to send notifications to the user's application. The input is the optimal travel route and discount information, and the output is a notification to the user's terminal.

[0213] Step 6:

[0214] The application installed in the autonomous vehicle updates the route of the autonomous vehicle in real time based on the optimal travel route information received from the server. The input is the optimal travel route information, and the output is the updated route setting for the autonomous vehicle.

[0215] Step 7:

[0216] The server personalizes and provides travel routes based on the user's past travel history and current preferences. For example, if a user prefers a particular route based on their past travel history, that route will be preferentially suggested. The input is the user's past travel history and current preferences, and the output is a customized travel route.

[0217] Step 8:

[0218] The server acquires and analyzes traffic congestion and weather conditions in real time and provides the optimal travel route based on that information. For example, if there is a sudden change in weather, the server will suggest a route change based on that information. The input is real-time traffic congestion and weather conditions, and the output is an updated optimal travel route.

[0219] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0220] The present invention provides a personalized travel experience that corresponds to the user's emotional state by combining an emotion engine with a system that supports efficient and comfortable travel for users. Specific embodiments of the present invention are described below.

[0221] 1. Acceptance of customer requests

[0222] Device: The user launches a dedicated mobility assistance application and inputs their starting point and destination. For example, if the user inputs "Station A" as the starting point and "Station B" as the destination, the input data is structured within the app.

[0223] Server: Receives the departure and destination information sent from the device and stores it in a database. This stored data is used in subsequent processes.

[0224] 2. Emotion Recognition by Emotion Engine

[0225] Device: The user provides facial expressions and voice through the device's camera and microphone, which the emotion engine analyzes to determine the user's current emotional state.

[0226] Server: The emotion engine analyzes the user's emotion data and classifies it. For example, if the user is feeling stressed, the emotion data is classified as "stress."

[0227] 3. Request Analysis and Route Optimization

[0228] Server: Retrieves request data from the database and passes it to the AI ​​algorithm. The AI ​​calculates the optimal route based on past travel data and traffic conditions. It also takes into account emotional data and selects a route that reflects the user's emotional state.

[0229] Server: For example, if the user is feeling stressed, the AI ​​will prioritize routes with less populated areas or more scenic views.

[0230] 4. Use of video analysis

[0231] Server: Obtains the latest information from traffic cameras and public transport and analyzes it in real time. Obtains real-time video data from the traffic camera API and uses video analysis technology to analyze traffic congestion information. For example, it detects congestion at major intersections.

[0232] Server: Integrates the analysis results of the emotion engine and the video analysis results, and reevaluates the route that best suits the user's emotions.

[0233] 5. Utilizing MaaS

[0234] Server: Integrates AI analysis results, real-time video analysis information, and emotion data to generate the optimal combination of transportation methods. For example, it generates the optimal travel route combining trains and buses.

[0235] Server: Automatically makes the necessary reservations and ticket arrangements. For example, automatically reserves train tickets and arranges bus tickets at the same time. Reservations are made using the public transport API.

[0236] 6. Route suggestions

[0237] Server: Obtains the optimal travel route and discount information for stores along that route, and generates route suggestions. Suggestions include information such as "Take the train from station A to station B, and get a 10% discount coupon at a cafe along the way."

[0238] On the device: The user is notified with a suggested route and store discount information. For example, the notification might say, "The best route is a 30-minute train ride, and there's a 10% off coupon at a cafe along the way."

[0239] 7. ONEID Payment Execution

[0240] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay."

[0241] Server: Sends payment information to the integrated payment system. The payment system processes the user's credit card information and other details, and pays the fee in one lump sum. A response indicating successful payment is returned to the server, which then sends the response to the user's device.

[0242] Terminal: The user's terminal will be notified that the fare for each mode of transportation has been paid. Ticket information for each mode of transportation will also be displayed. The user can use this information to smoothly start their journey.

[0243] Examples:

[0244] Terminal: The user inputs "home" as the departure point, "office" as the destination, and "immediately" as the departure time. In addition, the user provides emotional data (e.g., stress) via the camera.

[0245] Server: Passes request data and emotion data to an AI algorithm, and calculates the optimal route: "bus → train → walk." Video analysis also confirms bus congestion.

[0246] Server: Automatically book a bus ticket for 8am based on this route.

[0247] Device: Notifies the user of the optimal route from home to office, along with discount information for a cafe near the office.

[0248] Terminal: The user confirms the route, the fare is processed in bulk through the ONEID payment system, and ticket information is received.

[0249] The above is an embodiment of the present invention. This system allows users to travel efficiently and comfortably according to their emotional state, and also allows them to enjoy additional benefits along their travel route.

[0250] The processing flow will be explained below.

[0251] Step 1:

[0252] Device: The user launches the dedicated app and enters their departure and destination. They enter "Home" as the departure point and "Office" as the destination. They then set the departure time to "Now" and press the "Send Request" button.

[0253] Step 2:

[0254] Server: Receives the request data sent from the device and stores it in a database. The stored data includes the departure location "Home", the destination "Office", and the departure time "Now".

[0255] Step 3:

[0256] Device: The user provides their own emotional data using the app's camera or microphone. For example, emotions are recognized by showing facial expressions to the camera or speaking into the microphone.

[0257] Step 4:

[0258] Server: The emotion engine receives facial expressions and voice data sent from the device and performs emotion recognition processing. The emotion engine saves the analysis results in a database and records that the user is feeling "stressed."

[0259] Step 5:

[0260] Server: Retrieves request data and emotion data from the database and passes them to an AI algorithm. The AI ​​calculates the optimal route based on past travel data, real-time traffic conditions, and the user's emotion data. For example, if the user is feeling stressed, it will prioritize routes with less congestion or more beautiful scenery.

[0261] Step 6:

[0262] Server: Obtains real-time video data from the traffic camera API and performs video analysis. Using video analysis technology, congestion status on major roads and in areas with heavy traffic can be grasped in real time.

[0263] Step 7:

[0264] Server: Integrates AI analysis results with real-time video analysis results to generate the optimal combination of transportation methods. For example, if trains are crowded, prioritize bus and walking routes.

[0265] Step 8:

[0266] Server: Makes necessary reservations and arranges tickets. For example, automatically reserves bus tickets using an API and arranges train transfer information at the same time.

[0267] Step 9:

[0268] Server: Generates optimal travel routes and store discount information along the route to notify users. Suggestions include "use the bus as a route from home to the office, with a 10% discount coupon at a cafe along the way."

[0269] Step 10:

[0270] Device: Receives "optimal route notification" and displays the notification content. You can check the suggested route and discount information. For example, it may say, "The optimal route from home to the office is a 30-minute bus ride, and there is a 10% off coupon at the cafe along the way."

[0271] Step 11:

[0272] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay."

[0273] Step 12:

[0274] Server: Sends payment information to the integrated payment system and processes the fee. The payment system processes the credit card information and charges the fee in one lump sum. A success response is returned to the server and sent to the user's device.

[0275] Step 13:

[0276] Terminal: The user's terminal is notified that the fare for each mode of transportation has been paid. In addition, ticket information for each mode of transportation is displayed, allowing the user to smoothly start traveling.

[0277] In this way, the system of the present invention provides an optimal travel route that takes into account the user's emotional state, helping to make the trip more comfortable and efficient, and also improving the overall user experience by utilizing discounts and special offers along the route.

[0278] Example 2

[0279] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0280] While modern mobility assistance systems can provide efficient travel routes, they struggle to provide a personalized travel experience that takes into account the user's emotional state. For example, when a user is feeling stressed, it is desirable to avoid congested routes, but current systems struggle to provide such flexible route planning. Furthermore, they rarely properly integrate real-time traffic information and store discount information to provide users with optimal travel options.

[0281] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving information on a departure point and a destination input by a user, a means for acquiring and analyzing the user's emotional state from a camera or a microphone, a means for calculating an optimal travel route based on past travel data, traffic conditions, and the user's emotional state, a means for analyzing data from traffic cameras and public transportation in real time, a means for generating a travel route by combining optimal means of transportation, a means for acquiring discount information from stores along the travel route and including it in a route proposal, a means for notifying the user of the proposed travel route and discount information, and a means for processing travel fare information using an integrated payment system. This makes it possible to provide a personalized travel experience according to the user's emotional state.

[0282] "User" refers to a person who uses the system to receive travel route suggestions.

[0283] "Start point" refers to the point where the user starts.

[0284] "Destination" refers to the point the user wishes to reach.

[0285] "Emotional state" refers to the user's current mental or emotional state, including stress, joy, and the like.

[0286] "Camera" refers to a device that captures video.

[0287] "Microphone" refers to a device that captures sound.

[0288] "Analysis" refers to the act of analyzing acquired data and extracting useful information.

[0289] "Movement data" refers to data including past movement history and route information.

[0290] "Traffic conditions" refers to information showing the current traffic flow and degree of congestion.

[0291] "Real-time analytics" refers to the process of processing data instantly and obtaining results instantly.

[0292] "Travel route" refers to the route taken from a starting point to a destination.

[0293] "Store discount information" refers to information about discounts and coupons available at stores.

[0294] "Suggestion" refers to the recommended route or information provided to the user by the system.

[0295] "Notification" refers to the act of the system transmitting information to a user terminal.

[0296] "Integrated payment system" refers to a system for processing multiple payment methods at once.

[0297] The present invention relates to a system for assisting a user in efficient and comfortable travel. The system can provide a personalized travel experience by taking into account the emotional state of the user. Specific embodiments for implementing the invention will be described below.

[0298] Accepting user requests

[0299] The user launches the mobility assistance application on their device and enters information about their departure and destination. This information is sent to the server in JSON format. The server stores the received request data in a database.

[0300] Emotion recognition by emotion engine

[0301] Users use their device's camera and microphone to provide facial expressions and voice. The emotion engine analyzes this data to identify the user's emotional state. The analysis results are sent to the server and stored in a database as emotion data.

[0302] Request analysis and route optimization

[0303] The server retrieves the saved request data and emotion data and passes them to an AI algorithm. This AI algorithm calculates the optimal route based on past travel data and real-time traffic conditions. If the user is feeling stressed, it will prioritize routes with less congestion or better scenery.

[0304] Utilizing video analysis

[0305] The server receives real-time video data from traffic cameras and uses video analytics to analyze traffic congestion information, checking congestion at major intersections and public transport, and integrating the results with emotion data to reevaluate the optimal route.

[0306] Utilizing MaaS

[0307] The server combines the results of AI analysis, real-time video analysis information, and emotion data to generate the optimal combination of transportation methods. For example, it can generate a route combining trains and buses, and automatically make reservations for each mode of transportation.

[0308] Route suggestions

[0309] The server obtains the optimal route and discount information for stores along the route, and generates a proposal. The generated route proposal is notified to the device and provided to the user. For example, it may include information such as "Take the train as the route from station A to station B, and a 10% discount coupon is being offered at a cafe along the way."

[0310] ONEID payment execution

[0311] If the user is satisfied with the proposed route, they select "Confirm route and pay." The server then sends payment information to the integrated payment system and processes the credit card information. If the payment is successful, the result is notified to the terminal, and ticket information for each mode of transportation is displayed. The user can then use this information to smoothly begin their journey.

[0312] Specific examples

[0313] For example, a user inputs "home" as the departure point and "office" as the destination, and sets the departure time to "now." In addition, the user provides facial expression data showing stress levels via a camera. The server passes the request data and emotional data to an AI algorithm, which calculates the optimal route of "bus → train → walk." It checks the bus congestion status and automatically reserves a bus ticket for 8:00 AM. The user is notified of the optimal route from "home → office" and information about discounts at cafes. After the route is confirmed, the fare is processed in one lump sum via the ONEID payment system, and the ticket information is displayed.

[0314] In this way, by using this system, users can achieve an efficient and comfortable travel experience that suits their emotional state, and can also enjoy additional benefits along their travel route.

[0315] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0316] Step 1:

[0317] User: The user launches a dedicated mobility assistance application and inputs information about the departure and destination locations.

[0318] Input: Departure and destination information (e.g. "Station A", "Station B")

[0319] Output: Request data in JSON format

[0320] Specific operation: The user enters the departure and destination into the input fields in the application and presses the tap button to generate request data.

[0321] Step 2:

[0322] Terminal: Sends the entered data to the server.

[0323] Input: Request data

[0324] Output: Request data sent to the server

[0325] Specific operation: The terminal sends the generated request data to the "POST / api / requests" endpoint, which the server receives and stores in the database.

[0326] Step 3:

[0327] User: Uses the device's camera and microphone to provide facial expressions and voice.

[0328] Input: Camera video data, microphone audio data

[0329] Output: Input data to the emotion engine

[0330] Specific operation: When the user shows their facial expression to the camera or speaks into the microphone, video and audio are captured in real time.

[0331] Step 4:

[0332] Terminal: The emotion engine analyzes the user's emotional state.

[0333] Input: Camera video data, microphone audio data

[0334] Output: Emotion analysis results

[0335] How it works: The device's emotion engine analyzes video and audio data to identify the user's emotional state. For example, an AI model uses a facial expression recognition algorithm to determine stress levels.

[0336] Step 5:

[0337] Server: Receives the sentiment analysis results and stores them in a database.

[0338] Input: Sentiment analysis results

[0339] Output: Emotion data stored in a database

[0340] Specific operation: The analysis results received from the device are stored in the database as a "user ID-emotion data" pair.

[0341] Step 6:

[0342] Server: Obtains request data and emotion data and passes it to the AI ​​algorithm.

[0343] Input: Origin, Destination, and Emotion Data

[0344] Output: Optimal route

[0345] Specific operation: The server executes the "GET / api / requests / {user ID}" and "GET / api / emotions / {user ID}" APIs and passes the data to the AI ​​algorithm, which then calculates the optimal route using past movement data, traffic conditions, and emotion data.

[0346] Step 7:

[0347] Server: Acquires and analyzes real-time video data from traffic cameras.

[0348] Input: Traffic camera video data

[0349] Output: Traffic congestion information

[0350] Specific operation: The server retrieves video data using "GET / api / traffic-cameras" and inputs it into the video analysis model. The analysis results include congestion levels and traffic conditions.

[0351] Step 8:

[0352] Server: Integrates video analysis results with emotion data and reevaluates the optimal route.

[0353] Input: Traffic congestion information, emotion data

[0354] Output: Updated optimal route

[0355] Specific operation: Traffic congestion information and emotion data are passed to the server's "route reoptimization function" and the route is recalculated.

[0356] Step 9:

[0357] Server: Generates the optimal combination of transportation means and makes the necessary reservations and ticket arrangements.

[0358] Input: Optimal Route

[0359] Output: Booked transportation (e.g. train, bus tickets)

[0360] Specific operation: The server uses "POST / api / tickets" to link with the electronic reservation system and arrange for tickets.

[0361] Step 10:

[0362] Server: Notifies the user of the best route and store discount information.

[0363] Input: Optimal route, store discount information

[0364] Output: Notification message

[0365] Specific operation: The server sends a notification message to "PUSH / api / notifications" and it is displayed on the user's terminal.

[0366] Step 11:

[0367] User: Check the proposed route and make payment.

[0368] Input: Proposed route and discount information

[0369] Output: Payment information

[0370] Specific operation: When the user taps the "Confirm route and pay" button in the application, payment information is sent to the server.

[0371] Step 12:

[0372] Server: Processes payments through the integrated payment system.

[0373] Input: Payment information (credit card information, etc.)

[0374] Output: Payment result

[0375] Specific operation: The server executes "POST / api / payment" and processes the payment through the integrated payment system. If successful, it notifies the user terminal of the result.

[0376] Step 13:

[0377] Terminal: Notifies the user of the payment result and ticket information.

[0378] Input: Payment result, ticket information

[0379] Output: User notification

[0380] Specific operation: The terminal notifies the user that the payment was successful and displays ticket information for each mode of transportation.

[0381] (Application example 2)

[0382] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0383] Conventional mobility assistance systems do not take into account the user's emotional state, which means that comfort and satisfaction during travel are not fully ensured. Furthermore, users may feel stressed when multiple modes of transportation and route selection are required. Furthermore, adjusting the in-car environment during travel requires manual adjustments, which places a burden on the user.

[0384] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0385] In this invention, the server includes means for receiving information on departure and destination input by the user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for analyzing data from traffic observation equipment and public transportation in real time, means for generating a travel route by combining optimal travel modes, means for acquiring discount information for commercial facilities along the travel route and including it in a route proposal, means for notifying the user of the proposed travel route and discount information, means for processing travel mode fares using an integrated payment system, means for recognizing the user's emotional state and selecting an optimal travel route based on the emotion, and means for adjusting the internal environment of the travel mode based on the emotion, thereby enabling automatic adjustment of the optimal travel route and in-vehicle environment according to the user's emotional state.

[0386] A "user" is a person who uses the system and who inputs information about the departure point and destination.

[0387] The "starting point" is the point from which the user starts traveling.

[0388] A "destination" is a point where a user ends their journey.

[0389] "Travel data" refers to information about past travel history and traffic patterns.

[0390] "Traffic conditions" refers to the current road conditions and the operation status of public transportation.

[0391] "Traffic observation equipment" refers to devices such as cameras and sensors that are installed to monitor road and traffic conditions.

[0392] "Public transportation" refers to public means of transportation such as buses, trains, and taxis.

[0393] "Real-time analysis" is a technology that analyzes data instantly and grasps the current situation.

[0394] "Transportation" refers to any means of transportation used by a user to travel.

[0395] "Commercial facilities" refer to stores and service facilities that users can stop by while traveling.

[0396] "Discount information" is information about discounts and special offers that users can use at commercial facilities.

[0397] An "integrated payment system" is a system for processing multiple payment methods together.

[0398] "Fee" means the amount paid by a User for transportation or services provided.

[0399] "Emotional state" refers to the emotions or moods that a user is currently experiencing.

[0400] The "optimal travel route" refers to the route that allows the user to travel most comfortably and efficiently, based on past data and real-time information.

[0401] "In-vehicle environment" refers to environmental factors such as temperature, lighting, and music inside a vehicle.

[0402] "Means for adjusting the internal environment" refers to technologies and devices for automatically adjusting the temperature, music, etc. inside the vehicle.

[0403] The present invention is a system for optimizing a user's travel experience based on their emotional state. This system utilizes past travel data and real-time traffic conditions to provide an optimal travel route. It also recognizes the user's emotional state, selects an appropriate route, and adjusts the internal environment of the vehicle to support a user's comfortable travel.

[0404] Hardware and Software

[0405] Hardware used

[0406] Smartphone or smart glasses (with camera and microphone)

[0407] Self-driving cars (equipped with interior environment sensors and control devices)

[0408] Server (cloud computing environment)

[0409] Software used

[0410] EmotionRecognition: A library for analyzing emotion data in real time.

[0411] RouteOptimizer: An algorithm that selects the optimal route based on the user's emotional state.

[0412] CarControl: A control library for autonomous vehicles to coordinate routing and the in-car environment.

[0413] System processing procedure

[0414] 1. Collecting information from users

[0415] Users input their departure and destination information into the system using a smartphone or smart glasses, and also provide emotion data via a camera and microphone.

[0416] 2. Recognizing emotional states

[0417] The server uses the EmotionRecognition library to analyze facial and voice data obtained from the user in real time to identify the user's emotional state.

[0418] 3. Calculating the optimal route

[0419] The server uses the RouteOptimizer algorithm to calculate the optimal route based on past travel data and real-time traffic data, and selects a route that reflects the user's emotional state, taking into account emotional data.

[0420] 4. Real-time data analysis

[0421] The server collects real-time data from traffic observation equipment and public transport operators and analyzes traffic conditions. The analysis results are integrated with those of the emotion engine to reevaluate the most suitable route for the user.

[0422] 5. Adjusting the in-car environment

[0423] Based on the acquired emotional data, the server uses the CarControl library to adjust the in-car environment, such as temperature, music, and lighting, inside the self-driving car.

[0424] Specific examples

[0425] For example, if a user enters "home" as the departure point and "office" as the destination, they set the departure time as "now" and provide emotional data (e.g., stress) via the camera. The server passes the request data and emotional data to an AI algorithm, which calculates the optimal route: "bus → train → walk." Real-time video analysis also checks the bus congestion status. Based on this route, a bus ticket for 8:00 AM is then automatically reserved. The user is notified of the optimal route from "home to office" along with discount information for a cafe near the office. The user confirms the route, the fare is processed in one lump sum via the integrated payment system, and the ticket information is received.

[0426] Example prompts to input to the generative AI model

[0427] "Generate a program that suggests scenic routes and plays relaxation music to help users who are stressed relax."

[0428] This system allows users to have a comfortable and efficient travel experience while also enjoying additional benefits along the travel route.

[0429] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0430] Step 1:

[0431] The user inputs their departure and destination using a smartphone or smart glasses. Specifically, they launch a dedicated application and set their "home" as the departure point and their "office" as the destination through the UI. This information is sent from the application to the server, where the input data (departure point, destination) is output.

[0432] Step 2:

[0433] The device uses a camera and microphone to collect emotional data from the user. Specifically, the camera analyzes facial expressions and the microphone records voice tone. This data is analyzed in real time using the EmotionRecognition library to recognize the user's emotional state (e.g., stress). The input data (facial expression data, voice data) is output as emotional data (stress).

[0434] Step 3:

[0435] The server collects historical travel data and real-time traffic condition data. Specifically, it collects data from traffic observation equipment and public transportation APIs, and passes it to the RouteOptimizer algorithm. The input data (historical travel data, real-time traffic condition data) is output as the optimal route.

[0436] Step 4:

[0437] The server calculates the optimal travel route based on the emotion data. Specifically, it takes into account the user's emotional state (stress) and uses the RouteOptimizer algorithm to select a route with good scenery that reduces stress. The input data (emotion data, traffic condition data) is output as the emotion-based optimal route.

[0438] Step 5:

[0439] The server analyzes data from traffic observation equipment and public transportation in real time to obtain traffic congestion information. Specifically, it uses real-time video analysis technology to analyze data obtained from traffic cameras and sensors to understand congestion conditions. Input data (real-time video data) is output as traffic congestion information.

[0440] Step 6:

[0441] The server integrates traffic congestion information and emotion data and reevaluates the optimal route. Specifically, when traffic congestion information is acquired, it compares it with emotion data and recalculates the optimal travel route. The input data (emotion data, traffic congestion information) is output as the reevaluated optimal route.

[0442] Step 7:

[0443] The server obtains discount information for commercial facilities along the travel route and includes it in the route proposal. Specifically, it obtains discount information from the commercial facility API and adds it to the user's travel route. The input data (commercial facility data, optimal route) is output as a route proposal.

[0444] Step 8:

[0445] The server notifies the user of the proposed travel route and discount information. Specifically, it sends a push notification to the device, and the optimal route and discount information are displayed on the UI. The input data (route proposal, discount information) is notified to the user's device.

[0446] Step 9:

[0447] The user confirms the route and the fare is processed in bulk in the integrated payment system. Specifically, the user presses the route confirmation button within the application, and the payment system processes the fare for each mode of transportation. The input data (route confirmation data, payment information) is output as payment completion information.

[0448] Step 10:

[0449] The server adjusts the in-car environment based on the emotion data. Specifically, it adjusts the temperature, music, lighting, etc. inside the autonomous vehicle through the CarControl library. The input data (emotion data) is output as the adjusted in-car environment.

[0450] In this way, the system provides an optimal travel route and a comfortable in-car environment according to the user's emotional state.

[0451] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0452] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0453] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0454] [Second embodiment]

[0455] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0456] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0457] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0458] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0459] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0460] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0461] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0462] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0463] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0464] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0465] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0466] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0467] The present invention is a system designed to assist a user in efficient movement, and is implemented as follows.

[0468] 1. Acceptance of customer requests

[0469] Device: The user launches a dedicated mobility assistance application and inputs their starting point and destination. For example, if the user inputs "Station A" as the starting point and "Station B" as the destination, the input data is structured within the app.

[0470] Server: Receives the departure and destination information sent from the device and stores it in a database. This stored data is used in subsequent processes.

[0471] 2. AI-based request analysis

[0472] Server: Retrieves request data from the database and passes it to an AI algorithm, which calculates the optimal route in real time based on past travel data and traffic conditions.

[0473] Server: For example, it learns from past data which means of transportation a user prefers depending on the time of day, and presents routes based on that, taking into account traffic congestion and weather conditions.

[0474] 3. Use of video analysis

[0475] Server: Acquires the latest data from traffic cameras and public transport systems and analyzes it in real time, using image processing technology and machine learning algorithms.

[0476] Server: For example, it analyzes images from traffic cameras installed at major intersections and stations and obtains congestion and delay information in real time, making it possible to take measures such as avoiding routes with heavy traffic.

[0477] 4. Utilizing MaaS

[0478] Server: Integrates AI analysis results with real-time video analytics information to generate optimal travel combinations, including train transfers, bus trips, and taxi reservations.

[0479] Server: Automatically makes the necessary reservations and ticket arrangements. For example, if the system suggests a route that combines buses and trains, it purchases bus tickets on behalf of the user.

[0480] 5. Route suggestions

[0481] Server: Generates the optimal travel route and discount information for stores along that route, and notifies the user.

[0482] On the device: A notification appears in the user's app, showing suggested routes and store discount information. For example, the notification might say, "The best route from Station A to Station B is 30 minutes by train. There's also a 10% discount at the cafe near Station B."

[0483] 6. ONEID Payment Execution

[0484] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay," which enters the user's payment information.

[0485] Server: Sends payment information to the integrated payment system and processes the transportation fare. After receiving a successful response, sends the ticket information for each transportation to the user's terminal.

[0486] Examples:

[0487] Terminal: The user inputs "home" as the departure point and "office" as the destination. The desired departure time is "8:00 AM."

[0488] Server: Passes the request data to an AI algorithm and calculates the optimal route: "bus → train → walk." Video analysis also confirms bus congestion.

[0489] Server: Automatically book a bus ticket for 8am based on this route.

[0490] Device: Notifies the user of the optimal route from home to office, along with discount information for a cafe near the office.

[0491] Terminal: The user confirms the route and the fare is processed in bulk through the ONEID payment system. Ticket information is received, allowing for smooth travel.

[0492] The above is an embodiment of the present invention. This system enables users to travel stress-free and efficiently, and also allows users to take advantage of advantageous information along their travel route without missing it.

[0493] The processing flow will be explained below.

[0494] Step 1:

[0495] Device: The user launches the dedicated app and enters their departure and destination. The departure point is entered as "Station A" and the destination as "Station B." The user then selects "Now" as the departure time and presses the "Send Request" button.

[0496] Step 2:

[0497] Server: Receives the request data sent from the terminal. The received data includes information on the departure point (Station A), destination (Station B), and departure time (Now). The server stores this data in a database.

[0498] Step 3:

[0499] Server: Retrieves request data from the database and passes it to the AI ​​algorithm. The AI ​​calculates the optimal route based on past travel data and traffic conditions. For example, if it determines from past data that the user tends to avoid crowded areas, it will prioritize routes that avoid crowds.

[0500] Step 4:

[0501] Server: Obtains the latest information from traffic cameras and public transportation. Obtains real-time video data from the traffic camera API and analyzes traffic congestion information using video analytics technology. For example, it detects congestion at major intersections from traffic camera footage.

[0502] Step 5:

[0503] Server: Combines the results of AI analysis with the results of real-time video analysis to find the optimal means of transportation. For example, it generates the optimal route combining trains and buses.

[0504] Step 6:

[0505] Server: Makes the necessary reservations and ticket arrangements. For example, automatically reserves train tickets and arranges bus tickets at the same time. In this case, the reservation is made using the public transport company's API.

[0506] Step 7:

[0507] Server: Obtains the optimal travel route and discount information for stores along the route, and generates route suggestions. Suggestions include information such as "Take the train from station A to station B, and get a 10% discount coupon at a cafe along the way."

[0508] Step 8:

[0509] On the device: A notification appears in the user's app. The notification includes a suggested route and store discount information. For example, "The best route is a 30-minute train ride, and there's a 10% off coupon at a cafe along the way."

[0510] Step 9:

[0511] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay."

[0512] Step 10:

[0513] Server: Sends payment information to the integrated payment system. The payment system processes the user's credit card information and other details, and pays the fee in one lump sum. A response indicating successful payment is returned to the server, which then sends the response to the user's device.

[0514] Step 11:

[0515] Terminal: The user's terminal will be notified that the fare for each mode of transportation has been paid. Ticket information for each mode of transportation will also be displayed. The user can use this information to smoothly start their journey.

[0516] The above are the specific processing steps of the program for implementing the present invention. This system enables users to travel efficiently and comfortably, and also allows them to enjoy additional benefits along their travel route.

[0517] Example 1

[0518] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0519] Conventional mobility assistance systems require users to select the optimal route from their departure point to their destination and reserve and arrange multiple modes of transportation, which is cumbersome and hinders efficient travel. Furthermore, the lack of optimal route suggestions that reflect real-time traffic conditions and congestion information makes it difficult for users to travel as planned, and they often miss out on discount information that can be used during travel. To solve these issues, a system is needed that automatically proposes and arranges efficient travel routes, taking into account past travel data and real-time traffic conditions.

[0520] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0521] In this invention, the server includes means for receiving information on departure and destination entered by the user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for acquiring and analyzing data from traffic cameras and public transportation in real time, means for generating a travel route by combining optimal means of transportation and automatically making reservations and ticket arrangements, means for acquiring store discount information along the travel route and including it in the route proposal, means for notifying the user of the proposed travel route and discount information, means for processing travel fares and transmitting payment information using an integrated payment system, and means for distributing ticket information for the user's means of transportation in real time. This allows users to efficiently select and arrange travel routes without stress, and enables optimal travel taking into account real-time congestion information and discount information.

[0522] "User" refers to an individual or organization that uses the services of the mobility assistance system.

[0523] "Start point" refers to the point where the user starts their journey.

[0524] "Destination" refers to the point where the user ends their journey.

[0525] A "travel route" refers to a route from a departure point to a destination, and may include a route that combines multiple modes of transportation.

[0526] "Past journey data" refers to data relating to journeys taken in the past by the user or other users, including information such as preferred modes of transportation and time of day.

[0527] "Traffic conditions" refers to factors that affect travel, such as the current congestion situation on roads and public transportation, delay information, and accident information.

[0528] "Traffic cameras" refer to cameras installed to monitor congestion and congestion on roads and public transport.

[0529] "Public transport" refers to means of transport available to the general public, such as buses, trains and taxis.

[0530] "Real time" refers to ongoing situations and information, and indicates the exact moment an event occurs.

[0531] "Analysis" refers to procedures and methods for processing acquired data to facilitate the extraction of specific information.

[0532] "Reservation" refers to the act of a user promising to use something at a later date, including arranging transportation or services.

[0533] "Ticket" means a permit or ticket required to use a particular means of transportation or service.

[0534] "Store discount information" refers to information about discounts and special offers offered by stores along the route.

[0535] "Notification" refers to providing information to users, and is often done through smartphone applications.

[0536] An "integrated payment system" refers to a system for processing payments for different services and products in one place.

[0537] "Fare" means the amount paid for using a means of transportation or service.

[0538] "Payment information" refers to information related to a user's payment, including credit card information and electronic money details.

[0539] "Transportation ticket information" refers to the tickets and reservation information required to use transportation.

[0540] A "system" refers to a mechanism in which multiple elements work together to achieve a specific purpose.

[0541] The system is designed to efficiently support users' mobility and is implemented using multiple components, including hardware and software such as user devices, a central server, traffic cameras, databases, AI algorithms, and an integrated payment system.

[0542] User terminal

[0543] Users use a smartphone or tablet with a dedicated mobility support application installed. This application provides an interface for users to input their departure and destination locations. Payment information and notifications are also entered on this device.

[0544] Central Server

[0545] The server plays a central role in the system. It receives the origin and destination information sent by the user and stores it in a database. It then retrieves the stored data and passes it to an AI algorithm to calculate the optimal travel route.

[0546] Database

[0547] The database stores information such as user request data, past movement data, and traffic conditions, allowing the server to quickly retrieve and process the necessary data.

[0548] AI algorithms

[0549] The AI ​​algorithm calculates the optimal route based on past travel data and real-time traffic conditions. This algorithm learns the user's travel habits and preferences and can suggest the optimal means of transportation for each time of day. For example, it may suggest a "bus → train → walk" route during the morning commute.

[0550] Traffic Camera

[0551] Traffic cameras are installed at major intersections and stations to monitor traffic conditions in real time. The server acquires video data from these cameras and analyzes it using image processing technology. The analysis results include congestion status and delay information.

[0552] Integrated Payment System

[0553] The integrated payment system is used to process all the fares for the travel route selected by the user. When the user selects "Confirm route and pay," the server sends the payment information to the integrated payment system and processes the fare.

[0554] Route suggestions and notifications

[0555] The server generates an optimal route based on the results of AI algorithms and video analysis, and also integrates discount information for stores along the route. This information is sent to the user's device, where the user can check the proposed route and discount information. For example, a notification may appear saying, "The optimal route from Station A to Station B is 30 minutes by train. There is also a 10% discount at the cafe near Station B."

[0556] Specific examples

[0557] As a concrete example, consider the case where a user inputs "home" as the departure point, "office" as the destination, and the desired departure time as "8:00 AM." The server passes the request data to an AI algorithm, which calculates the optimal route: "bus → train → walk." It also analyzes traffic camera footage to check bus congestion. Based on this route, the system automatically reserves a bus ticket for 8:00 AM. The user's device is notified of the optimal route from "home to office," along with discount information for a cafe near the office. Finally, the user confirms the route and the fare is processed in one lump sum via the ONEID payment system. The ticket information is then received by the device, allowing for a smooth journey.

[0558] The above is a specific embodiment of the present invention. This system allows users to efficiently select and arrange travel routes without stress, and enables optimal travel taking into account real-time congestion information and discount information.

[0559] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0560] Step 1:

[0561] The user starts a dedicated mobility assistance application and inputs their starting point and destination. The input information is converted into JSON format such as {"start":"Station A", "destination":"Station B"}. Input: Starting point and destination from the user. Output: Structured JSON format data.

[0562] Step 2:

[0563] The terminal sends structured JSON data to the server. The HTTPS protocol is used for transmission, and data is encrypted. Input: Structured JSON data. Output: Transmission to server.

[0564] Step 3:

[0565] The server saves the received JSON data in the database. The specific save operation uses an SQL insert statement. Input: JSON data from the terminal. Output: Request information saved in the database.

[0566] Step 4:

[0567] The server retrieves the request data from the database and passes it to the AI ​​algorithm. It retrieves the data using an SQL select statement. Input: Request data from the database. Output: Request data passed to the AI ​​algorithm.

[0568] Step 5:

[0569] The AI ​​algorithm on the server calculates the optimal travel route based on past travel data and traffic conditions. It uses a specific machine learning model (e.g., neural network). Input: Request data, past travel data, and traffic condition data. Output: Optimal travel route information.

[0570] Step 6:

[0571] The server stores the calculation results in temporary storage. For the storage operation, an in-memory database such as Redis or Memcached is used. Input: Optimal travel route information. Output: Travel route information stored in temporary storage.

[0572] Step 7:

[0573] The server retrieves the latest data from traffic cameras and public transport. It uses API calls to retrieve real-time data. Input: API call. Output: Latest data from traffic cameras and public transport.

[0574] Step 8:

[0575] The server analyzes the traffic camera video data using image processing technology. It uses a deep learning model (e.g., YOLO). Input: Traffic camera video data. Output: Analyzed congestion and delay information.

[0576] Step 9:

[0577] The server stores the analysis results in temporary storage. Input: Congestion status and delay information. Output: Analysis results stored in temporary storage.

[0578] Step 10:

[0579] The server combines the results of AI analysis and video analysis to generate the optimal combination of transportation methods. The algorithm uses a linear optimization model. Input: Optimal transportation route information and analysis results. Output: Optimal combination of transportation methods.

[0580] Step 11:

[0581] The server automatically makes the necessary reservations and arranges tickets. It uses API calls to connect with bus and train reservation systems. Input: Information on optimal combinations of transportation methods. Output: Information on completed reservations and ticket arrangements.

[0582] Step 12:

[0583] The server saves the reservation and arrangement results in temporary storage. Input: Completed reservation and ticket arrangement information. Output: Reservation and arrangement information saved in temporary storage.

[0584] Step 13:

[0585] The server generates the optimal travel route and store discount information along that route. The store discount information is obtained from an external promotion API. Input: Optimal travel route information. Output: Combination of travel route and store discount information.

[0586] Step 14:

[0587] The server sends the generated information to the user's device. The HTTPS protocol is used for communication. Input: Travel route and store discount information. Output: Notification to the user's device.

[0588] Step 15:

[0589] The device displays the notification to the user. Specifically, it uses the notification function within the app to display information on the screen. Input: Notification information from the server. Output: Visualized notification to the user.

[0590] Step 16:

[0591] The user selects "Confirm route and pay." The entered payment information is encrypted on the terminal. Input: User's payment information. Output: Encrypted payment information.

[0592] Step 17:

[0593] The terminal sends encrypted payment information to the server. The HTTPS protocol is used for communication. Input: Encrypted payment information. Output: Send to server.

[0594] Step 18:

[0595] The server sends the payment information to the integrated payment system and processes the travel fare. Input: Encrypted payment information. Output: Transmission to the integrated payment system and fare processing.

[0596] Step 19:

[0597] The server receives a success response indicating that the fare processing was successful and sends the ticket information for each means of transportation to the user's terminal. Input: Success response from the integrated payment system. Output: Ticket information sent to the user's terminal.

[0598] Step 20:

[0599] The device displays the ticket information to the user. Specifically, it uses the ticket management function within the app. Input: Ticket information from the server. Output: Ticket information displayed to the user.

[0600] (Application example 1)

[0601] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0602] In recent years, advances in autonomous driving technology have improved the convenience of user travel. However, there are still problems with finding optimal travel routes and updating routes in real time according to traffic congestion and weather conditions. Furthermore, there is a lack of personalized travel routes that meet individual user preferences, so there is room for improvement in order to provide an optimal travel experience. Furthermore, it is necessary to improve user convenience by utilizing discount information and services during travel.

[0603] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0604] In this invention, the server includes means for receiving information on departure and destination points input by a user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for analyzing data from traffic cameras and public transportation in real time, means for combining optimal travel modes to generate a travel route, means for acquiring discount information for stores along the travel route and including it in a route proposal, means for notifying the user of the proposed travel route and discount information, means for processing travel mode fares using an integrated payment system, means for updating the route in real time using an application installed in the autonomous vehicle, means for customizing the route based on user preferences, and means for taking traffic congestion and weather conditions into consideration. This allows users to obtain the optimal travel route based on real-time information, enjoy a personalized travel experience tailored to their individual preferences, and also take advantage of discount information along the travel route.

[0605] The "means for receiving information on the departure point and destination" refers to a device or software that acquires the information on the departure point and destination entered by the user and transmits it to the server.

[0606] A "means for calculating an optimal travel route" refers to an algorithm or program that calculates the most efficient and quickest travel route based on past travel data and traffic conditions.

[0607] "Means for analyzing data from traffic cameras and public transportation in real time" refers to technology and devices that analyze data obtained from traffic cameras and public transportation in real time to determine traffic congestion and delay information.

[0608] "Means for generating a travel route by combining optimal means of transportation" refers to a system or software for combining multiple means of transportation to generate the most suitable travel route for a user.

[0609] "Means for acquiring discount information of stores along a travel route and including it in a route proposal" refers to technology or devices that acquire discount information of stores along a travel route and reflect it in a travel route proposal.

[0610] The "means for notifying the user of the proposed travel route and discount information" refers to a system or application for notifying the user of the optimal travel route and discount information related to that route.

[0611] "Means for processing transportation fares using an integrated payment system" refers to an integrated payment system for processing fares for multiple transportation means in one go.

[0612] "Means for updating routes in real time through an application installed on an autonomous vehicle" refers to technology or devices for updating travel routes in real time through an application installed on an autonomous vehicle.

[0613] "Means for customizing routes based on user preferences" means a system or algorithm for providing personalized travel routes based on a user's past travel history and current preferences.

[0614] "Means that take into account traffic congestion and weather conditions" refers to technologies and devices that acquire and analyze traffic congestion and weather conditions in real time and provide optimal travel routes based on that information.

[0615] The present invention is a system designed to assist users in traveling efficiently, and is implemented as follows using an application installed in an autonomous vehicle.

[0616] First, the user launches a dedicated mobility assistance application and inputs their departure and destination. This information is sent from the user's device to the server and stored in a database. The server then calculates the optimal route based on the received departure and destination information, taking into account past travel data and real-time traffic conditions.

[0617] The calculated route is further optimized by analyzing data from traffic cameras and public transport in real time. The server analyzes the video data acquired in real time and adjusts the route based on congestion and delay information. This is done using Python and OpenCV video analysis.

[0618] Once the optimal combination of transportation modes has been determined, the server automatically makes the necessary reservations and ticket arrangements using MaaS (Mobility as a Service), and processes fares through an integrated payment system. Once all arrangements are complete, the user is notified of the proposed route and discount information for stores along the route.

[0619] One of the features of this system is that the application installed in the autonomous vehicle updates the route in real time. The server takes into account traffic congestion and weather conditions in real time and provides the autonomous vehicle with the optimal route.

[0620] Additionally, it also has the ability to customize routes based on user preferences, which takes into account the user's past travel history and current preferences to provide a more personalized travel experience.

[0621] For example, if a user inputs "home" as the departure point and "sports gym" as the destination, setting the departure time to "5:00 PM," the server will calculate a "bus → train → walk" route based on past travel data and real-time traffic information. The server then analyzes traffic camera footage to check bus congestion and automatically reserves a bus ticket for 5:00 PM. The user receives this route and discount information for cafes near the gym via the app, and the fare is processed in one lump sum via the ONEID payment system.

[0622] An example prompt might be, "Based on the user's input of origin and destination, suggest the best route for travel, taking into account traffic congestion and weather conditions. Also, provide personalized suggestions based on the user's preferences and offer discount information along the route."

[0623] As described above, the present invention is a system that can provide users with efficient, personalized, and optimal travel routes by using autonomous vehicles.

[0624] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0625] Step 1:

[0626] The user starts a dedicated mobility assistance application and inputs their starting point and destination. The starting point (e.g., "home") and destination (e.g., "sports gym") information entered by the user is sent from the terminal to the server.

[0627] Step 2:

[0628] The server receives the departure and destination information and stores it in a database. The server then retrieves past travel data and real-time traffic information from the database and calculates the optimal travel route. The input here is the departure and destination information entered by the user, as well as the past travel data and traffic information in the database, and the output is the optimal travel route.

[0629] Step 3:

[0630] The server acquires real-time data from traffic cameras and public transport and uses video analysis to determine traffic congestion and delay information. The server then uses this data to further optimize the optimal travel route. Specifically, it uses Python and OpenCV to analyze the video data and detect congestion and delays. The input is real-time data, and the output is an updated optimal travel route.

[0631] Step 4:

[0632] The server utilizes MaaS (Mobility as a Service) to combine optimal means of transportation and automatically make the necessary reservations and ticket arrangements. The server processes all transportation fares in one go through an integrated payment system. The input is the optimal travel route and reservation information, and the output is reservation confirmation and payment completion information.

[0633] Step 5:

[0634] The server notifies the user of the optimal travel route and discount information for stores along the route. This includes a function to send notifications to the user's application. The input is the optimal travel route and discount information, and the output is a notification to the user's terminal.

[0635] Step 6:

[0636] The application installed in the autonomous vehicle updates the route of the autonomous vehicle in real time based on the optimal travel route information received from the server. The input is the optimal travel route information, and the output is the updated route setting for the autonomous vehicle.

[0637] Step 7:

[0638] The server personalizes and provides travel routes based on the user's past travel history and current preferences. For example, if a user prefers a particular route based on their past travel history, that route will be preferentially suggested. The input is the user's past travel history and current preferences, and the output is a customized travel route.

[0639] Step 8:

[0640] The server acquires and analyzes traffic congestion and weather conditions in real time and provides the optimal travel route based on that information. For example, if there is a sudden change in weather, the server will suggest a route change based on that information. The input is real-time traffic congestion and weather conditions, and the output is an updated optimal travel route.

[0641] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0642] The present invention provides a personalized travel experience that corresponds to the user's emotional state by combining an emotion engine with a system that supports efficient and comfortable travel for users. Specific embodiments of the present invention are described below.

[0643] 1. Acceptance of customer requests

[0644] Device: The user launches a dedicated mobility assistance application and inputs their starting point and destination. For example, if the user inputs "Station A" as the starting point and "Station B" as the destination, the input data is structured within the app.

[0645] Server: Receives the departure and destination information sent from the device and stores it in a database. This stored data is used in subsequent processes.

[0646] 2. Emotion Recognition by Emotion Engine

[0647] Device: The user provides facial expressions and voice through the device's camera and microphone, which the emotion engine analyzes to determine the user's current emotional state.

[0648] Server: The emotion engine analyzes the user's emotion data and classifies it. For example, if the user is feeling stressed, the emotion data is classified as "stress."

[0649] 3. Request Analysis and Route Optimization

[0650] Server: Retrieves request data from the database and passes it to the AI ​​algorithm. The AI ​​calculates the optimal route based on past travel data and traffic conditions. It also takes into account emotional data and selects a route that reflects the user's emotional state.

[0651] Server: For example, if the user is feeling stressed, the AI ​​will prioritize routes with less populated areas or more scenic views.

[0652] 4. Use of video analysis

[0653] Server: Obtains the latest information from traffic cameras and public transport and analyzes it in real time. Obtains real-time video data from the traffic camera API and uses video analysis technology to analyze traffic congestion information. For example, it detects congestion at major intersections.

[0654] Server: Integrates the analysis results of the emotion engine and the video analysis results, and reevaluates the route that best suits the user's emotions.

[0655] 5. Utilizing MaaS

[0656] Server: Integrates AI analysis results, real-time video analysis information, and emotion data to generate the optimal combination of transportation methods. For example, it generates the optimal travel route combining trains and buses.

[0657] Server: Automatically makes the necessary reservations and ticket arrangements. For example, automatically reserves train tickets and arranges bus tickets at the same time. Reservations are made using the public transport API.

[0658] 6. Route suggestions

[0659] Server: Obtains the optimal travel route and discount information for stores along that route, and generates route suggestions. Suggestions include information such as "Take the train from station A to station B, and get a 10% discount coupon at a cafe along the way."

[0660] On the device: The user is notified with a suggested route and store discount information. For example, the notification might say, "The best route is a 30-minute train ride, and there's a 10% off coupon at a cafe along the way."

[0661] 7. ONEID Payment Execution

[0662] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay."

[0663] Server: Sends payment information to the integrated payment system. The payment system processes the user's credit card information and other details, and pays the fee in one lump sum. A response indicating successful payment is returned to the server, which then sends the response to the user's device.

[0664] Terminal: The user's terminal will be notified that the fare for each mode of transportation has been paid. Ticket information for each mode of transportation will also be displayed. The user can use this information to smoothly start their journey.

[0665] Examples:

[0666] Terminal: The user inputs "home" as the departure point, "office" as the destination, and "immediately" as the departure time. In addition, the user provides emotional data (e.g., stress) via the camera.

[0667] Server: Passes request data and emotion data to an AI algorithm, and calculates the optimal route: "bus → train → walk." Video analysis also confirms bus congestion.

[0668] Server: Automatically book a bus ticket for 8am based on this route.

[0669] Device: Notifies the user of the optimal route from home to office, along with discount information for a cafe near the office.

[0670] Terminal: The user confirms the route, the fare is processed in bulk through the ONEID payment system, and ticket information is received.

[0671] The above is an embodiment of the present invention. This system allows users to travel efficiently and comfortably according to their emotional state, and also allows them to enjoy additional benefits along their travel route.

[0672] The processing flow will be explained below.

[0673] Step 1:

[0674] Device: The user launches the dedicated app and enters their departure and destination. They enter "Home" as the departure point and "Office" as the destination. They then set the departure time to "Now" and press the "Send Request" button.

[0675] Step 2:

[0676] Server: Receives the request data sent from the device and stores it in a database. The stored data includes the departure location "Home", the destination "Office", and the departure time "Now".

[0677] Step 3:

[0678] Device: The user provides their own emotional data using the app's camera or microphone. For example, emotions are recognized by showing facial expressions to the camera or speaking into the microphone.

[0679] Step 4:

[0680] Server: The emotion engine receives facial expressions and voice data sent from the device and performs emotion recognition processing. The emotion engine saves the analysis results in a database and records that the user is feeling "stressed."

[0681] Step 5:

[0682] Server: Retrieves request data and emotion data from the database and passes them to an AI algorithm. The AI ​​calculates the optimal route based on past travel data, real-time traffic conditions, and the user's emotion data. For example, if the user is feeling stressed, it will prioritize routes with less congestion or more beautiful scenery.

[0683] Step 6:

[0684] Server: Obtains real-time video data from the traffic camera API and performs video analysis. Using video analysis technology, congestion status on major roads and in areas with heavy traffic can be grasped in real time.

[0685] Step 7:

[0686] Server: Integrates AI analysis results with real-time video analysis results to generate the optimal combination of transportation methods. For example, if trains are crowded, prioritize bus and walking routes.

[0687] Step 8:

[0688] Server: Makes necessary reservations and arranges tickets. For example, automatically reserves bus tickets using an API and arranges train transfer information at the same time.

[0689] Step 9:

[0690] Server: Generates optimal travel routes and store discount information along the route to notify users. Suggestions include "use the bus as a route from home to the office, with a 10% discount coupon at a cafe along the way."

[0691] Step 10:

[0692] Device: Receives "optimal route notification" and displays the notification content. You can check the suggested route and discount information. For example, it may say, "The optimal route from home to the office is a 30-minute bus ride, and there is a 10% off coupon at the cafe along the way."

[0693] Step 11:

[0694] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay."

[0695] Step 12:

[0696] Server: Sends payment information to the integrated payment system and processes the fee. The payment system processes the credit card information and charges the fee in one lump sum. A success response is returned to the server and sent to the user's device.

[0697] Step 13:

[0698] Terminal: The user's terminal is notified that the fare for each mode of transportation has been paid. In addition, ticket information for each mode of transportation is displayed, allowing the user to smoothly start traveling.

[0699] In this way, the system of the present invention provides an optimal travel route that takes into account the user's emotional state, helping to make the trip more comfortable and efficient, and also improving the overall user experience by utilizing discounts and special offers along the route.

[0700] Example 2

[0701] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0702] While modern mobility assistance systems can provide efficient travel routes, they struggle to provide a personalized travel experience that takes into account the user's emotional state. For example, when a user is feeling stressed, it is desirable to avoid congested routes, but current systems struggle to provide such flexible route planning. Furthermore, they rarely properly integrate real-time traffic information and store discount information to provide users with optimal travel options.

[0703] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving information on a departure point and a destination input by a user, a means for acquiring and analyzing the user's emotional state from a camera or a microphone, a means for calculating an optimal travel route based on past travel data, traffic conditions, and the user's emotional state, a means for analyzing data from traffic cameras and public transportation in real time, a means for generating a travel route by combining optimal means of transportation, a means for acquiring discount information from stores along the travel route and including it in a route proposal, a means for notifying the user of the proposed travel route and discount information, and a means for processing travel fare information using an integrated payment system. This makes it possible to provide a personalized travel experience according to the user's emotional state.

[0704] "User" refers to a person who uses the system to receive travel route suggestions.

[0705] "Start point" refers to the point where the user starts.

[0706] "Destination" refers to the point the user wishes to reach.

[0707] "Emotional state" refers to the user's current mental or emotional state, including stress, joy, and the like.

[0708] "Camera" refers to a device that captures video.

[0709] "Microphone" refers to a device that captures sound.

[0710] "Analysis" refers to the act of analyzing acquired data and extracting useful information.

[0711] "Movement data" refers to data including past movement history and route information.

[0712] "Traffic conditions" refers to information showing the current traffic flow and degree of congestion.

[0713] "Real-time analytics" refers to the process of processing data instantly and obtaining results instantly.

[0714] "Travel route" refers to the route taken from a starting point to a destination.

[0715] "Store discount information" refers to information about discounts and coupons available at stores.

[0716] "Suggestion" refers to the recommended route or information provided to the user by the system.

[0717] "Notification" refers to the act of the system transmitting information to a user terminal.

[0718] "Integrated payment system" refers to a system for processing multiple payment methods at once.

[0719] The present invention relates to a system for assisting a user in efficient and comfortable travel. The system can provide a personalized travel experience by taking into account the emotional state of the user. Specific embodiments for implementing the invention will be described below.

[0720] Accepting user requests

[0721] The user launches the mobility assistance application on their device and enters information about their departure and destination. This information is sent to the server in JSON format. The server stores the received request data in a database.

[0722] Emotion recognition by emotion engine

[0723] Users use their device's camera and microphone to provide facial expressions and voice. The emotion engine analyzes this data to identify the user's emotional state. The analysis results are sent to the server and stored in a database as emotion data.

[0724] Request analysis and route optimization

[0725] The server retrieves the saved request data and emotion data and passes them to an AI algorithm. This AI algorithm calculates the optimal route based on past travel data and real-time traffic conditions. If the user is feeling stressed, it will prioritize routes with less congestion or better scenery.

[0726] Utilizing video analysis

[0727] The server receives real-time video data from traffic cameras and uses video analytics to analyze traffic congestion information, checking congestion at major intersections and public transport, and integrating the results with emotion data to reevaluate the optimal route.

[0728] Utilizing MaaS

[0729] The server combines the results of AI analysis, real-time video analysis information, and emotion data to generate the optimal combination of transportation methods. For example, it can generate a route combining trains and buses, and automatically make reservations for each mode of transportation.

[0730] Route suggestions

[0731] The server obtains the optimal route and discount information for stores along the route, and generates a proposal. The generated route proposal is notified to the device and provided to the user. For example, it may include information such as "Take the train as the route from station A to station B, and a 10% discount coupon is being offered at a cafe along the way."

[0732] ONEID payment execution

[0733] If the user is satisfied with the proposed route, they select "Confirm route and pay." The server then sends payment information to the integrated payment system and processes the credit card information. If the payment is successful, the result is notified to the terminal, and ticket information for each mode of transportation is displayed. The user can then use this information to smoothly begin their journey.

[0734] Specific examples

[0735] For example, a user inputs "home" as the departure point and "office" as the destination, and sets the departure time to "now." In addition, the user provides facial expression data showing stress levels via a camera. The server passes the request data and emotional data to an AI algorithm, which calculates the optimal route of "bus → train → walk." It checks the bus congestion status and automatically reserves a bus ticket for 8:00 AM. The user is notified of the optimal route from "home → office" and information about discounts at cafes. After the route is confirmed, the fare is processed in one lump sum via the ONEID payment system, and the ticket information is displayed.

[0736] In this way, by using this system, users can achieve an efficient and comfortable travel experience that suits their emotional state, and can also enjoy additional benefits along their travel route.

[0737] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0738] Step 1:

[0739] User: The user launches a dedicated mobility assistance application and inputs information about the departure and destination locations.

[0740] Input: Departure and destination information (e.g. "Station A", "Station B")

[0741] Output: Request data in JSON format

[0742] Specific operation: The user enters the departure and destination into the input fields in the application and presses the tap button to generate request data.

[0743] Step 2:

[0744] Terminal: Sends the entered data to the server.

[0745] Input: Request data

[0746] Output: Request data sent to the server

[0747] Specific operation: The terminal sends the generated request data to the "POST / api / requests" endpoint, which the server receives and stores in the database.

[0748] Step 3:

[0749] User: Uses the device's camera and microphone to provide facial expressions and voice.

[0750] Input: Camera video data, microphone audio data

[0751] Output: Input data to the emotion engine

[0752] Specific operation: When the user shows their facial expression to the camera or speaks into the microphone, video and audio are captured in real time.

[0753] Step 4:

[0754] Terminal: The emotion engine analyzes the user's emotional state.

[0755] Input: Camera video data, microphone audio data

[0756] Output: Emotion analysis results

[0757] How it works: The device's emotion engine analyzes video and audio data to identify the user's emotional state. For example, an AI model uses a facial expression recognition algorithm to determine stress levels.

[0758] Step 5:

[0759] Server: Receives the sentiment analysis results and stores them in a database.

[0760] Input: Sentiment analysis results

[0761] Output: Emotion data stored in a database

[0762] Specific operation: The analysis results received from the device are stored in the database as a "user ID-emotion data" pair.

[0763] Step 6:

[0764] Server: Obtains request data and emotion data and passes it to the AI ​​algorithm.

[0765] Input: Origin, Destination, and Emotion Data

[0766] Output: Optimal route

[0767] Specific operation: The server executes the "GET / api / requests / {user ID}" and "GET / api / emotions / {user ID}" APIs and passes the data to the AI ​​algorithm, which then calculates the optimal route using past movement data, traffic conditions, and emotion data.

[0768] Step 7:

[0769] Server: Acquires and analyzes real-time video data from traffic cameras.

[0770] Input: Traffic camera video data

[0771] Output: Traffic congestion information

[0772] Specific operation: The server retrieves video data using "GET / api / traffic-cameras" and inputs it into the video analysis model. The analysis results include congestion levels and traffic conditions.

[0773] Step 8:

[0774] Server: Integrates video analysis results with emotion data and reevaluates the optimal route.

[0775] Input: Traffic congestion information, emotion data

[0776] Output: Updated optimal route

[0777] Specific operation: Traffic congestion information and emotion data are passed to the server's "route reoptimization function" and the route is recalculated.

[0778] Step 9:

[0779] Server: Generates the optimal combination of transportation means and makes the necessary reservations and ticket arrangements.

[0780] Input: Optimal Route

[0781] Output: Booked transportation (e.g. train, bus tickets)

[0782] Specific operation: The server uses "POST / api / tickets" to link with the electronic reservation system and arrange for tickets.

[0783] Step 10:

[0784] Server: Notifies the user of the best route and store discount information.

[0785] Input: Optimal route, store discount information

[0786] Output: Notification message

[0787] Specific operation: The server sends a notification message to "PUSH / api / notifications" and it is displayed on the user's terminal.

[0788] Step 11:

[0789] User: Check the proposed route and make payment.

[0790] Input: Proposed route and discount information

[0791] Output: Payment information

[0792] Specific operation: When the user taps the "Confirm route and pay" button in the application, payment information is sent to the server.

[0793] Step 12:

[0794] Server: Processes payments through the integrated payment system.

[0795] Input: Payment information (credit card information, etc.)

[0796] Output: Payment result

[0797] Specific operation: The server executes "POST / api / payment" and processes the payment through the integrated payment system. If successful, it notifies the user terminal of the result.

[0798] Step 13:

[0799] Terminal: Notifies the user of the payment result and ticket information.

[0800] Input: Payment result, ticket information

[0801] Output: User notification

[0802] Specific operation: The terminal notifies the user that the payment was successful and displays ticket information for each mode of transportation.

[0803] (Application example 2)

[0804] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0805] Conventional mobility assistance systems do not take into account the user's emotional state, which means that comfort and satisfaction during travel are not fully ensured. Furthermore, users may feel stressed when multiple modes of transportation and route selection are required. Furthermore, adjusting the in-car environment during travel requires manual adjustments, which places a burden on the user.

[0806] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0807] In this invention, the server includes means for receiving information on departure and destination input by the user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for analyzing data from traffic observation equipment and public transportation in real time, means for generating a travel route by combining optimal travel modes, means for acquiring discount information for commercial facilities along the travel route and including it in a route proposal, means for notifying the user of the proposed travel route and discount information, means for processing travel mode fares using an integrated payment system, means for recognizing the user's emotional state and selecting an optimal travel route based on the emotion, and means for adjusting the internal environment of the travel mode based on the emotion, thereby enabling automatic adjustment of the optimal travel route and in-vehicle environment according to the user's emotional state.

[0808] A "user" is a person who uses the system and who inputs information about the departure point and destination.

[0809] The "starting point" is the point from which the user starts traveling.

[0810] A "destination" is a point where a user ends their journey.

[0811] "Travel data" refers to information about past travel history and traffic patterns.

[0812] "Traffic conditions" refers to the current road conditions and the operation status of public transportation.

[0813] "Traffic observation equipment" refers to devices such as cameras and sensors that are installed to monitor road and traffic conditions.

[0814] "Public transportation" refers to public means of transportation such as buses, trains, and taxis.

[0815] "Real-time analysis" is a technology that analyzes data instantly and grasps the current situation.

[0816] "Transportation" refers to any means of transportation used by a user to travel.

[0817] "Commercial facilities" refer to stores and service facilities that users can stop by while traveling.

[0818] "Discount information" is information about discounts and special offers that users can use at commercial facilities.

[0819] An "integrated payment system" is a system for processing multiple payment methods together.

[0820] "Fee" means the amount paid by a User for transportation or services provided.

[0821] "Emotional state" refers to the emotions or moods that a user is currently experiencing.

[0822] The "optimal travel route" refers to the route that allows the user to travel most comfortably and efficiently, based on past data and real-time information.

[0823] "In-vehicle environment" refers to environmental factors such as temperature, lighting, and music inside a vehicle.

[0824] "Means for adjusting the internal environment" refers to technologies and devices for automatically adjusting the temperature, music, etc. inside the vehicle.

[0825] The present invention is a system for optimizing a user's travel experience based on their emotional state. This system utilizes past travel data and real-time traffic conditions to provide an optimal travel route. It also recognizes the user's emotional state, selects an appropriate route, and adjusts the internal environment of the vehicle to support a user's comfortable travel.

[0826] Hardware and Software

[0827] Hardware used

[0828] Smartphone or smart glasses (with camera and microphone)

[0829] Self-driving cars (equipped with interior environment sensors and control devices)

[0830] Server (cloud computing environment)

[0831] Software used

[0832] EmotionRecognition: A library for analyzing emotion data in real time.

[0833] RouteOptimizer: An algorithm that selects the optimal route based on the user's emotional state.

[0834] CarControl: A control library for autonomous vehicles to coordinate routing and the in-car environment.

[0835] System processing procedure

[0836] 1. Collecting information from users

[0837] Users input their departure and destination information into the system using a smartphone or smart glasses, and also provide emotion data via a camera and microphone.

[0838] 2. Recognizing emotional states

[0839] The server uses the EmotionRecognition library to analyze facial and voice data obtained from the user in real time to identify the user's emotional state.

[0840] 3. Calculating the optimal route

[0841] The server uses the RouteOptimizer algorithm to calculate the optimal route based on past travel data and real-time traffic data, and selects a route that reflects the user's emotional state, taking into account emotional data.

[0842] 4. Real-time data analysis

[0843] The server collects real-time data from traffic observation equipment and public transport operators and analyzes traffic conditions. The analysis results are integrated with those of the emotion engine to reevaluate the most suitable route for the user.

[0844] 5. Adjusting the in-car environment

[0845] Based on the acquired emotional data, the server uses the CarControl library to adjust the in-car environment, such as temperature, music, and lighting, inside the self-driving car.

[0846] Specific examples

[0847] For example, if a user enters "home" as the departure point and "office" as the destination, they set the departure time as "now" and provide emotional data (e.g., stress) via the camera. The server passes the request data and emotional data to an AI algorithm, which calculates the optimal route: "bus → train → walk." Real-time video analysis also checks the bus congestion status. Based on this route, a bus ticket for 8:00 AM is then automatically reserved. The user is notified of the optimal route from "home to office" along with discount information for a cafe near the office. The user confirms the route, the fare is processed in one lump sum via the integrated payment system, and the ticket information is received.

[0848] Example prompts to input to the generative AI model

[0849] "Generate a program that suggests scenic routes and plays relaxation music to help users who are stressed relax."

[0850] This system allows users to have a comfortable and efficient travel experience while also enjoying additional benefits along the travel route.

[0851] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0852] Step 1:

[0853] The user inputs their departure and destination using a smartphone or smart glasses. Specifically, they launch a dedicated application and set their "home" as the departure point and their "office" as the destination through the UI. This information is sent from the application to the server, where the input data (departure point, destination) is output.

[0854] Step 2:

[0855] The device uses a camera and microphone to collect emotional data from the user. Specifically, the camera analyzes facial expressions and the microphone records voice tone. This data is analyzed in real time using the EmotionRecognition library to recognize the user's emotional state (e.g., stress). The input data (facial expression data, voice data) is output as emotional data (stress).

[0856] Step 3:

[0857] The server collects historical travel data and real-time traffic condition data. Specifically, it collects data from traffic observation equipment and public transportation APIs, and passes it to the RouteOptimizer algorithm. The input data (historical travel data, real-time traffic condition data) is output as the optimal route.

[0858] Step 4:

[0859] The server calculates the optimal travel route based on the emotion data. Specifically, it takes into account the user's emotional state (stress) and uses the RouteOptimizer algorithm to select a route with good scenery that reduces stress. The input data (emotion data, traffic condition data) is output as the emotion-based optimal route.

[0860] Step 5:

[0861] The server analyzes data from traffic observation equipment and public transportation in real time to obtain traffic congestion information. Specifically, it uses real-time video analysis technology to analyze data obtained from traffic cameras and sensors to understand congestion conditions. Input data (real-time video data) is output as traffic congestion information.

[0862] Step 6:

[0863] The server integrates traffic congestion information and emotion data and reevaluates the optimal route. Specifically, when traffic congestion information is acquired, it compares it with emotion data and recalculates the optimal travel route. The input data (emotion data, traffic congestion information) is output as the reevaluated optimal route.

[0864] Step 7:

[0865] The server obtains discount information for commercial facilities along the travel route and includes it in the route proposal. Specifically, it obtains discount information from the commercial facility API and adds it to the user's travel route. The input data (commercial facility data, optimal route) is output as a route proposal.

[0866] Step 8:

[0867] The server notifies the user of the proposed travel route and discount information. Specifically, it sends a push notification to the device, and the optimal route and discount information are displayed on the UI. The input data (route proposal, discount information) is notified to the user's device.

[0868] Step 9:

[0869] The user confirms the route and the fare is processed in bulk in the integrated payment system. Specifically, the user presses the route confirmation button within the application, and the payment system processes the fare for each mode of transportation. The input data (route confirmation data, payment information) is output as payment completion information.

[0870] Step 10:

[0871] The server adjusts the in-car environment based on the emotion data. Specifically, it adjusts the temperature, music, lighting, etc. inside the autonomous vehicle through the CarControl library. The input data (emotion data) is output as the adjusted in-car environment.

[0872] In this way, the system provides an optimal travel route and a comfortable in-car environment according to the user's emotional state.

[0873] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0874] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0875] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0876] [Third embodiment]

[0877] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0878] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0879] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0880] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0881] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0882] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0883] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0884] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0885] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0886] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0887] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0888] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0889] The present invention is a system designed to assist a user in efficient movement, and is implemented as follows.

[0890] 1. Acceptance of customer requests

[0891] Device: The user launches a dedicated mobility assistance application and inputs their starting point and destination. For example, if the user inputs "Station A" as the starting point and "Station B" as the destination, the input data is structured within the app.

[0892] Server: Receives the departure and destination information sent from the device and stores it in a database. This stored data is used in subsequent processes.

[0893] 2. AI-based request analysis

[0894] Server: Retrieves request data from the database and passes it to an AI algorithm, which calculates the optimal route in real time based on past travel data and traffic conditions.

[0895] Server: For example, it learns from past data which means of transportation a user prefers depending on the time of day, and presents routes based on that, taking into account traffic congestion and weather conditions.

[0896] 3. Use of video analysis

[0897] Server: Acquires the latest data from traffic cameras and public transport systems and analyzes it in real time, using image processing technology and machine learning algorithms.

[0898] Server: For example, it analyzes images from traffic cameras installed at major intersections and stations and obtains congestion and delay information in real time, making it possible to take measures such as avoiding routes with heavy traffic.

[0899] 4. Utilizing MaaS

[0900] Server: Integrates AI analysis results with real-time video analytics information to generate optimal travel combinations, including train transfers, bus trips, and taxi reservations.

[0901] Server: Automatically makes the necessary reservations and ticket arrangements. For example, if the system suggests a route that combines buses and trains, it purchases bus tickets on behalf of the user.

[0902] 5. Route suggestions

[0903] Server: Generates the optimal travel route and discount information for stores along that route, and notifies the user.

[0904] On the device: A notification appears in the user's app, showing suggested routes and store discount information. For example, the notification might say, "The best route from Station A to Station B is 30 minutes by train. There's also a 10% discount at the cafe near Station B."

[0905] 6. ONEID Payment Execution

[0906] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay," which enters the user's payment information.

[0907] Server: Sends payment information to the integrated payment system and processes the transportation fare. After receiving a successful response, sends the ticket information for each transportation to the user's terminal.

[0908] Examples:

[0909] Terminal: The user inputs "home" as the departure point and "office" as the destination. The desired departure time is "8:00 AM."

[0910] Server: Passes the request data to an AI algorithm and calculates the optimal route: "bus → train → walk." Video analysis also confirms bus congestion.

[0911] Server: Automatically book a bus ticket for 8am based on this route.

[0912] Device: Notifies the user of the optimal route from home to office, along with discount information for a cafe near the office.

[0913] Terminal: The user confirms the route and the fare is processed in bulk through the ONEID payment system. Ticket information is received, allowing for smooth travel.

[0914] The above is an embodiment of the present invention. This system enables users to travel stress-free and efficiently, and also allows users to take advantage of advantageous information along their travel route without missing it.

[0915] The processing flow will be explained below.

[0916] Step 1:

[0917] Device: The user launches the dedicated app and enters their departure and destination. The departure point is entered as "Station A" and the destination as "Station B." The user then selects "Now" as the departure time and presses the "Send Request" button.

[0918] Step 2:

[0919] Server: Receives the request data sent from the terminal. The received data includes information on the departure point (Station A), destination (Station B), and departure time (Now). The server stores this data in a database.

[0920] Step 3:

[0921] Server: Retrieves request data from the database and passes it to the AI ​​algorithm. The AI ​​calculates the optimal route based on past travel data and traffic conditions. For example, if it determines from past data that the user tends to avoid crowded areas, it will prioritize routes that avoid crowds.

[0922] Step 4:

[0923] Server: Obtains the latest information from traffic cameras and public transportation. Obtains real-time video data from the traffic camera API and analyzes traffic congestion information using video analytics technology. For example, it detects congestion at major intersections from traffic camera footage.

[0924] Step 5:

[0925] Server: Combines the results of AI analysis with the results of real-time video analysis to find the optimal means of transportation. For example, it generates the optimal route combining trains and buses.

[0926] Step 6:

[0927] Server: Makes the necessary reservations and ticket arrangements. For example, automatically reserves train tickets and arranges bus tickets at the same time. In this case, the reservation is made using the public transport company's API.

[0928] Step 7:

[0929] Server: Obtains the optimal travel route and discount information for stores along the route, and generates route suggestions. Suggestions include information such as "Take the train from station A to station B, and get a 10% discount coupon at a cafe along the way."

[0930] Step 8:

[0931] On the device: A notification appears in the user's app. The notification includes a suggested route and store discount information. For example, "The best route is a 30-minute train ride, and there's a 10% off coupon at a cafe along the way."

[0932] Step 9:

[0933] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay."

[0934] Step 10:

[0935] Server: Sends payment information to the integrated payment system. The payment system processes the user's credit card information and other details, and pays the fee in one lump sum. A response indicating successful payment is returned to the server, which then sends the response to the user's device.

[0936] Step 11:

[0937] Terminal: The user's terminal will be notified that the fare for each mode of transportation has been paid. Ticket information for each mode of transportation will also be displayed. The user can use this information to smoothly start their journey.

[0938] The above are the specific processing steps of the program for implementing the present invention. This system enables users to travel efficiently and comfortably, and also allows them to enjoy additional benefits along their travel route.

[0939] Example 1

[0940] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0941] Conventional mobility assistance systems require users to select the optimal route from their departure point to their destination and reserve and arrange multiple modes of transportation, which is cumbersome and hinders efficient travel. Furthermore, the lack of optimal route suggestions that reflect real-time traffic conditions and congestion information makes it difficult for users to travel as planned, and they often miss out on discount information that can be used during travel. To solve these issues, a system is needed that automatically proposes and arranges efficient travel routes, taking into account past travel data and real-time traffic conditions.

[0942] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0943] In this invention, the server includes means for receiving information on departure and destination entered by the user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for acquiring and analyzing data from traffic cameras and public transportation in real time, means for generating a travel route by combining optimal means of transportation and automatically making reservations and ticket arrangements, means for acquiring store discount information along the travel route and including it in the route proposal, means for notifying the user of the proposed travel route and discount information, means for processing travel fares and transmitting payment information using an integrated payment system, and means for distributing ticket information for the user's means of transportation in real time. This allows users to efficiently select and arrange travel routes without stress, and enables optimal travel taking into account real-time congestion information and discount information.

[0944] "User" refers to an individual or organization that uses the services of the mobility assistance system.

[0945] "Start point" refers to the point where the user starts their journey.

[0946] "Destination" refers to the point where the user ends their journey.

[0947] A "travel route" refers to a route from a departure point to a destination, and may include a route that combines multiple modes of transportation.

[0948] "Past journey data" refers to data relating to journeys taken in the past by the user or other users, including information such as preferred modes of transportation and time of day.

[0949] "Traffic conditions" refers to factors that affect travel, such as the current congestion situation on roads and public transportation, delay information, and accident information.

[0950] "Traffic cameras" refer to cameras installed to monitor congestion and congestion on roads and public transport.

[0951] "Public transport" refers to means of transport available to the general public, such as buses, trains and taxis.

[0952] "Real time" refers to ongoing situations and information, and indicates the exact moment an event occurs.

[0953] "Analysis" refers to procedures and methods for processing acquired data to facilitate the extraction of specific information.

[0954] "Reservation" refers to the act of a user promising to use something at a later date, including arranging transportation or services.

[0955] "Ticket" means a permit or ticket required to use a particular means of transportation or service.

[0956] "Store discount information" refers to information about discounts and special offers offered by stores along the route.

[0957] "Notification" refers to providing information to users, and is often done through smartphone applications.

[0958] An "integrated payment system" refers to a system for processing payments for different services and products in one place.

[0959] "Fare" means the amount paid for using a means of transportation or service.

[0960] "Payment information" refers to information related to a user's payment, including credit card information and electronic money details.

[0961] "Transportation ticket information" refers to the tickets and reservation information required to use transportation.

[0962] A "system" refers to a mechanism in which multiple elements work together to achieve a specific purpose.

[0963] The system is designed to efficiently support users' mobility and is implemented using multiple components, including hardware and software such as user devices, a central server, traffic cameras, databases, AI algorithms, and an integrated payment system.

[0964] User terminal

[0965] Users use a smartphone or tablet with a dedicated mobility support application installed. This application provides an interface for users to input their departure and destination locations. Payment information and notifications are also entered on this device.

[0966] Central Server

[0967] The server plays a central role in the system. It receives the origin and destination information sent by the user and stores it in a database. It then retrieves the stored data and passes it to an AI algorithm to calculate the optimal travel route.

[0968] Database

[0969] The database stores information such as user request data, past movement data, and traffic conditions, allowing the server to quickly retrieve and process the necessary data.

[0970] AI algorithms

[0971] The AI ​​algorithm calculates the optimal route based on past travel data and real-time traffic conditions. This algorithm learns the user's travel habits and preferences and can suggest the optimal means of transportation for each time of day. For example, it may suggest a "bus → train → walk" route during the morning commute.

[0972] Traffic Camera

[0973] Traffic cameras are installed at major intersections and stations to monitor traffic conditions in real time. The server acquires video data from these cameras and analyzes it using image processing technology. The analysis results include congestion status and delay information.

[0974] Integrated Payment System

[0975] The integrated payment system is used to process all the fares for the travel route selected by the user. When the user selects "Confirm route and pay," the server sends the payment information to the integrated payment system and processes the fare.

[0976] Route suggestions and notifications

[0977] The server generates an optimal route based on the results of AI algorithms and video analysis, and also integrates discount information for stores along the route. This information is sent to the user's device, where the user can check the proposed route and discount information. For example, a notification may appear saying, "The optimal route from Station A to Station B is 30 minutes by train. There is also a 10% discount at the cafe near Station B."

[0978] Specific examples

[0979] As a concrete example, consider the case where a user inputs "home" as the departure point, "office" as the destination, and the desired departure time as "8:00 AM." The server passes the request data to an AI algorithm, which calculates the optimal route: "bus → train → walk." It also analyzes traffic camera footage to check bus congestion. Based on this route, the system automatically reserves a bus ticket for 8:00 AM. The user's device is notified of the optimal route from "home to office," along with discount information for a cafe near the office. Finally, the user confirms the route and the fare is processed in one lump sum via the ONEID payment system. The ticket information is then received by the device, allowing for a smooth journey.

[0980] The above is a specific embodiment of the present invention. This system allows users to efficiently select and arrange travel routes without stress, and enables optimal travel taking into account real-time congestion information and discount information.

[0981] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0982] Step 1:

[0983] The user starts a dedicated mobility assistance application and inputs their starting point and destination. The input information is converted into JSON format such as {"start":"Station A", "destination":"Station B"}. Input: Starting point and destination from the user. Output: Structured JSON format data.

[0984] Step 2:

[0985] The terminal sends structured JSON data to the server. The HTTPS protocol is used for transmission, and data is encrypted. Input: Structured JSON data. Output: Transmission to server.

[0986] Step 3:

[0987] The server saves the received JSON data in the database. The specific save operation uses an SQL insert statement. Input: JSON data from the terminal. Output: Request information saved in the database.

[0988] Step 4:

[0989] The server retrieves the request data from the database and passes it to the AI ​​algorithm. It retrieves the data using an SQL select statement. Input: Request data from the database. Output: Request data passed to the AI ​​algorithm.

[0990] Step 5:

[0991] The AI ​​algorithm on the server calculates the optimal travel route based on past travel data and traffic conditions. It uses a specific machine learning model (e.g., neural network). Input: Request data, past travel data, and traffic condition data. Output: Optimal travel route information.

[0992] Step 6:

[0993] The server stores the calculation results in temporary storage. For the storage operation, an in-memory database such as Redis or Memcached is used. Input: Optimal travel route information. Output: Travel route information stored in temporary storage.

[0994] Step 7:

[0995] The server retrieves the latest data from traffic cameras and public transport. It uses API calls to retrieve real-time data. Input: API call. Output: Latest data from traffic cameras and public transport.

[0996] Step 8:

[0997] The server analyzes the traffic camera video data using image processing technology. It uses a deep learning model (e.g., YOLO). Input: Traffic camera video data. Output: Analyzed congestion and delay information.

[0998] Step 9:

[0999] The server stores the analysis results in temporary storage. Input: Congestion status and delay information. Output: Analysis results stored in temporary storage.

[1000] Step 10:

[1001] The server combines the results of AI analysis and video analysis to generate the optimal combination of transportation methods. The algorithm uses a linear optimization model. Input: Optimal transportation route information and analysis results. Output: Optimal combination of transportation methods.

[1002] Step 11:

[1003] The server automatically makes the necessary reservations and arranges tickets. It uses API calls to connect with bus and train reservation systems. Input: Information on optimal combinations of transportation methods. Output: Information on completed reservations and ticket arrangements.

[1004] Step 12:

[1005] The server saves the reservation and arrangement results in temporary storage. Input: Completed reservation and ticket arrangement information. Output: Reservation and arrangement information saved in temporary storage.

[1006] Step 13:

[1007] The server generates the optimal travel route and store discount information along that route. The store discount information is obtained from an external promotion API. Input: Optimal travel route information. Output: Combination of travel route and store discount information.

[1008] Step 14:

[1009] The server sends the generated information to the user's device. The HTTPS protocol is used for communication. Input: Travel route and store discount information. Output: Notification to the user's device.

[1010] Step 15:

[1011] The device displays the notification to the user. Specifically, it uses the notification function within the app to display information on the screen. Input: Notification information from the server. Output: Visualized notification to the user.

[1012] Step 16:

[1013] The user selects "Confirm route and pay." The entered payment information is encrypted on the terminal. Input: User's payment information. Output: Encrypted payment information.

[1014] Step 17:

[1015] The terminal sends encrypted payment information to the server. The HTTPS protocol is used for communication. Input: Encrypted payment information. Output: Send to server.

[1016] Step 18:

[1017] The server sends the payment information to the integrated payment system and processes the travel fare. Input: Encrypted payment information. Output: Transmission to the integrated payment system and fare processing.

[1018] Step 19:

[1019] The server receives a success response indicating that the fare processing was successful and sends the ticket information for each means of transportation to the user's terminal. Input: Success response from the integrated payment system. Output: Ticket information sent to the user's terminal.

[1020] Step 20:

[1021] The device displays the ticket information to the user. Specifically, it uses the ticket management function within the app. Input: Ticket information from the server. Output: Ticket information displayed to the user.

[1022] (Application example 1)

[1023] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1024] In recent years, advances in autonomous driving technology have improved the convenience of user travel. However, there are still problems with finding optimal travel routes and updating routes in real time according to traffic congestion and weather conditions. Furthermore, there is a lack of personalized travel routes that meet individual user preferences, so there is room for improvement in order to provide an optimal travel experience. Furthermore, it is necessary to improve user convenience by utilizing discount information and services during travel.

[1025] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1026] In this invention, the server includes means for receiving information on departure and destination points input by a user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for analyzing data from traffic cameras and public transportation in real time, means for combining optimal travel modes to generate a travel route, means for acquiring discount information for stores along the travel route and including it in a route proposal, means for notifying the user of the proposed travel route and discount information, means for processing travel mode fares using an integrated payment system, means for updating the route in real time using an application installed in the autonomous vehicle, means for customizing the route based on user preferences, and means for taking traffic congestion and weather conditions into consideration. This allows users to obtain the optimal travel route based on real-time information, enjoy a personalized travel experience tailored to their individual preferences, and also take advantage of discount information along the travel route.

[1027] The "means for receiving information on the departure point and destination" refers to a device or software that acquires the information on the departure point and destination entered by the user and transmits it to the server.

[1028] A "means for calculating an optimal travel route" refers to an algorithm or program that calculates the most efficient and quickest travel route based on past travel data and traffic conditions.

[1029] "Means for analyzing data from traffic cameras and public transportation in real time" refers to technology and devices that analyze data obtained from traffic cameras and public transportation in real time to determine traffic congestion and delay information.

[1030] "Means for generating a travel route by combining optimal means of transportation" refers to a system or software for combining multiple means of transportation to generate the most suitable travel route for a user.

[1031] "Means for acquiring discount information of stores along a travel route and including it in a route proposal" refers to technology or devices that acquire discount information of stores along a travel route and reflect it in a travel route proposal.

[1032] The "means for notifying the user of the proposed travel route and discount information" refers to a system or application for notifying the user of the optimal travel route and discount information related to that route.

[1033] "Means for processing transportation fares using an integrated payment system" refers to an integrated payment system for processing fares for multiple transportation means in one go.

[1034] "Means for updating routes in real time through an application installed on an autonomous vehicle" refers to technology or devices for updating travel routes in real time through an application installed on an autonomous vehicle.

[1035] "Means for customizing routes based on user preferences" means a system or algorithm for providing personalized travel routes based on a user's past travel history and current preferences.

[1036] "Means that take into account traffic congestion and weather conditions" refers to technologies and devices that acquire and analyze traffic congestion and weather conditions in real time and provide optimal travel routes based on that information.

[1037] The present invention is a system designed to assist users in traveling efficiently, and is implemented as follows using an application installed in an autonomous vehicle.

[1038] First, the user launches a dedicated mobility assistance application and inputs their departure and destination. This information is sent from the user's device to the server and stored in a database. The server then calculates the optimal route based on the received departure and destination information, taking into account past travel data and real-time traffic conditions.

[1039] The calculated route is further optimized by analyzing data from traffic cameras and public transport in real time. The server analyzes the video data acquired in real time and adjusts the route based on congestion and delay information. This is done using Python and OpenCV video analysis.

[1040] Once the optimal combination of transportation modes has been determined, the server automatically makes the necessary reservations and ticket arrangements using MaaS (Mobility as a Service), and processes fares through an integrated payment system. Once all arrangements are complete, the user is notified of the proposed route and discount information for stores along the route.

[1041] One of the features of this system is that the application installed in the autonomous vehicle updates the route in real time. The server takes into account traffic congestion and weather conditions in real time and provides the autonomous vehicle with the optimal route.

[1042] Additionally, it also has the ability to customize routes based on user preferences, which takes into account the user's past travel history and current preferences to provide a more personalized travel experience.

[1043] For example, if a user inputs "home" as the departure point and "sports gym" as the destination, setting the departure time to "5:00 PM," the server will calculate a "bus → train → walk" route based on past travel data and real-time traffic information. The server then analyzes traffic camera footage to check bus congestion and automatically reserves a bus ticket for 5:00 PM. The user receives this route and discount information for cafes near the gym via the app, and the fare is processed in one lump sum via the ONEID payment system.

[1044] An example prompt might be, "Based on the user's input of origin and destination, suggest the best route for travel, taking into account traffic congestion and weather conditions. Also, provide personalized suggestions based on the user's preferences and offer discount information along the route."

[1045] As described above, the present invention is a system that can provide users with efficient, personalized, and optimal travel routes by using autonomous vehicles.

[1046] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1047] Step 1:

[1048] The user starts a dedicated mobility assistance application and inputs their starting point and destination. The starting point (e.g., "home") and destination (e.g., "sports gym") information entered by the user is sent from the terminal to the server.

[1049] Step 2:

[1050] The server receives the departure and destination information and stores it in a database. The server then retrieves past travel data and real-time traffic information from the database and calculates the optimal travel route. The input here is the departure and destination information entered by the user, as well as the past travel data and traffic information in the database, and the output is the optimal travel route.

[1051] Step 3:

[1052] The server acquires real-time data from traffic cameras and public transport and uses video analysis to determine traffic congestion and delay information. The server then uses this data to further optimize the optimal travel route. Specifically, it uses Python and OpenCV to analyze the video data and detect congestion and delays. The input is real-time data, and the output is an updated optimal travel route.

[1053] Step 4:

[1054] The server utilizes MaaS (Mobility as a Service) to combine optimal means of transportation and automatically make the necessary reservations and ticket arrangements. The server processes all transportation fares in one go through an integrated payment system. The input is the optimal travel route and reservation information, and the output is reservation confirmation and payment completion information.

[1055] Step 5:

[1056] The server notifies the user of the optimal travel route and discount information for stores along the route. This includes a function to send notifications to the user's application. The input is the optimal travel route and discount information, and the output is a notification to the user's terminal.

[1057] Step 6:

[1058] The application installed in the autonomous vehicle updates the route of the autonomous vehicle in real time based on the optimal travel route information received from the server. The input is the optimal travel route information, and the output is the updated route setting for the autonomous vehicle.

[1059] Step 7:

[1060] The server personalizes and provides travel routes based on the user's past travel history and current preferences. For example, if a user prefers a particular route based on their past travel history, that route will be preferentially suggested. The input is the user's past travel history and current preferences, and the output is a customized travel route.

[1061] Step 8:

[1062] The server acquires and analyzes traffic congestion and weather conditions in real time and provides the optimal travel route based on that information. For example, if there is a sudden change in weather, the server will suggest a route change based on that information. The input is real-time traffic congestion and weather conditions, and the output is an updated optimal travel route.

[1063] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1064] The present invention provides a personalized travel experience that corresponds to the user's emotional state by combining an emotion engine with a system that supports efficient and comfortable travel for users. Specific embodiments of the present invention are described below.

[1065] 1. Acceptance of customer requests

[1066] Device: The user launches a dedicated mobility assistance application and inputs their starting point and destination. For example, if the user inputs "Station A" as the starting point and "Station B" as the destination, the input data is structured within the app.

[1067] Server: Receives the departure and destination information sent from the device and stores it in a database. This stored data is used in subsequent processes.

[1068] 2. Emotion Recognition by Emotion Engine

[1069] Device: The user provides facial expressions and voice through the device's camera and microphone, which the emotion engine analyzes to determine the user's current emotional state.

[1070] Server: The emotion engine analyzes the user's emotion data and classifies it. For example, if the user is feeling stressed, the emotion data is classified as "stress."

[1071] 3. Request Analysis and Route Optimization

[1072] Server: Retrieves request data from the database and passes it to the AI ​​algorithm. The AI ​​calculates the optimal route based on past travel data and traffic conditions. It also takes into account emotional data and selects a route that reflects the user's emotional state.

[1073] Server: For example, if the user is feeling stressed, the AI ​​will prioritize routes with less populated areas or more scenic views.

[1074] 4. Use of video analysis

[1075] Server: Obtains the latest information from traffic cameras and public transport and analyzes it in real time. Obtains real-time video data from the traffic camera API and uses video analysis technology to analyze traffic congestion information. For example, it detects congestion at major intersections.

[1076] Server: Integrates the analysis results of the emotion engine and the video analysis results, and reevaluates the route that best suits the user's emotions.

[1077] 5. Utilizing MaaS

[1078] Server: Integrates AI analysis results, real-time video analysis information, and emotion data to generate the optimal combination of transportation methods. For example, it generates the optimal travel route combining trains and buses.

[1079] Server: Automatically makes the necessary reservations and ticket arrangements. For example, automatically reserves train tickets and arranges bus tickets at the same time. Reservations are made using the public transport API.

[1080] 6. Route suggestions

[1081] Server: Obtains the optimal travel route and discount information for stores along that route, and generates route suggestions. Suggestions include information such as "Take the train from station A to station B, and get a 10% discount coupon at a cafe along the way."

[1082] On the device: The user is notified with a suggested route and store discount information. For example, the notification might say, "The best route is a 30-minute train ride, and there's a 10% off coupon at a cafe along the way."

[1083] 7. ONEID Payment Execution

[1084] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay."

[1085] Server: Sends payment information to the integrated payment system. The payment system processes the user's credit card information and other details, and pays the fee in one lump sum. A response indicating successful payment is returned to the server, which then sends the response to the user's device.

[1086] Terminal: The user's terminal will be notified that the fare for each mode of transportation has been paid. Ticket information for each mode of transportation will also be displayed. The user can use this information to smoothly start their journey.

[1087] Examples:

[1088] Terminal: The user inputs "home" as the departure point, "office" as the destination, and "immediately" as the departure time. In addition, the user provides emotional data (e.g., stress) via the camera.

[1089] Server: Passes request data and emotion data to an AI algorithm, and calculates the optimal route: "bus → train → walk." Video analysis also confirms bus congestion.

[1090] Server: Automatically book a bus ticket for 8am based on this route.

[1091] Device: Notifies the user of the optimal route from home to office, along with discount information for a cafe near the office.

[1092] Terminal: The user confirms the route, the fare is processed in bulk through the ONEID payment system, and ticket information is received.

[1093] The above is an embodiment of the present invention. This system allows users to travel efficiently and comfortably according to their emotional state, and also allows them to enjoy additional benefits along their travel route.

[1094] The processing flow will be explained below.

[1095] Step 1:

[1096] Device: The user launches the dedicated app and enters their departure and destination. They enter "Home" as the departure point and "Office" as the destination. They then set the departure time to "Now" and press the "Send Request" button.

[1097] Step 2:

[1098] Server: Receives the request data sent from the device and stores it in a database. The stored data includes the departure location "Home", the destination "Office", and the departure time "Now".

[1099] Step 3:

[1100] Device: The user provides their own emotional data using the app's camera or microphone. For example, emotions are recognized by showing facial expressions to the camera or speaking into the microphone.

[1101] Step 4:

[1102] Server: The emotion engine receives facial expressions and voice data sent from the device and performs emotion recognition processing. The emotion engine saves the analysis results in a database and records that the user is feeling "stressed."

[1103] Step 5:

[1104] Server: Retrieves request data and emotion data from the database and passes them to an AI algorithm. The AI ​​calculates the optimal route based on past travel data, real-time traffic conditions, and the user's emotion data. For example, if the user is feeling stressed, it will prioritize routes with less congestion or more beautiful scenery.

[1105] Step 6:

[1106] Server: Obtains real-time video data from the traffic camera API and performs video analysis. Using video analysis technology, congestion status on major roads and in areas with heavy traffic can be grasped in real time.

[1107] Step 7:

[1108] Server: Integrates AI analysis results with real-time video analysis results to generate the optimal combination of transportation methods. For example, if trains are crowded, prioritize bus and walking routes.

[1109] Step 8:

[1110] Server: Makes necessary reservations and arranges tickets. For example, automatically reserves bus tickets using an API and arranges train transfer information at the same time.

[1111] Step 9:

[1112] Server: Generates optimal travel routes and store discount information along the route to notify users. Suggestions include "use the bus as a route from home to the office, with a 10% discount coupon at a cafe along the way."

[1113] Step 10:

[1114] Device: Receives "optimal route notification" and displays the notification content. You can check the suggested route and discount information. For example, it may say, "The optimal route from home to the office is a 30-minute bus ride, and there is a 10% off coupon at the cafe along the way."

[1115] Step 11:

[1116] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay."

[1117] Step 12:

[1118] Server: Sends payment information to the integrated payment system and processes the fee. The payment system processes the credit card information and charges the fee in one lump sum. A success response is returned to the server and sent to the user's device.

[1119] Step 13:

[1120] Terminal: The user's terminal is notified that the fare for each mode of transportation has been paid. In addition, ticket information for each mode of transportation is displayed, allowing the user to smoothly start traveling.

[1121] In this way, the system of the present invention provides an optimal travel route that takes into account the user's emotional state, helping to make the trip more comfortable and efficient, and also improving the overall user experience by utilizing discounts and special offers along the route.

[1122] Example 2

[1123] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1124] While modern mobility assistance systems can provide efficient travel routes, they struggle to provide a personalized travel experience that takes into account the user's emotional state. For example, when a user is feeling stressed, it is desirable to avoid congested routes, but current systems struggle to provide such flexible route planning. Furthermore, they rarely properly integrate real-time traffic information and store discount information to provide users with optimal travel options.

[1125] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving information on a departure point and a destination input by a user, a means for acquiring and analyzing the user's emotional state from a camera or a microphone, a means for calculating an optimal travel route based on past travel data, traffic conditions, and the user's emotional state, a means for analyzing data from traffic cameras and public transportation in real time, a means for generating a travel route by combining optimal means of transportation, a means for acquiring discount information from stores along the travel route and including it in a route proposal, a means for notifying the user of the proposed travel route and discount information, and a means for processing travel fare information using an integrated payment system. This makes it possible to provide a personalized travel experience according to the user's emotional state.

[1126] "User" refers to a person who uses the system to receive travel route suggestions.

[1127] "Start point" refers to the point where the user starts.

[1128] "Destination" refers to the point the user wishes to reach.

[1129] "Emotional state" refers to the user's current mental or emotional state, including stress, joy, and the like.

[1130] "Camera" refers to a device that captures video.

[1131] "Microphone" refers to a device that captures sound.

[1132] "Analysis" refers to the act of analyzing acquired data and extracting useful information.

[1133] "Movement data" refers to data including past movement history and route information.

[1134] "Traffic conditions" refers to information showing the current traffic flow and degree of congestion.

[1135] "Real-time analytics" refers to the process of processing data instantly and obtaining results instantly.

[1136] "Travel route" refers to the route taken from a starting point to a destination.

[1137] "Store discount information" refers to information about discounts and coupons available at stores.

[1138] "Suggestion" refers to the recommended route or information provided to the user by the system.

[1139] "Notification" refers to the act of the system transmitting information to a user terminal.

[1140] "Integrated payment system" refers to a system for processing multiple payment methods at once.

[1141] The present invention relates to a system for assisting a user in efficient and comfortable travel. The system can provide a personalized travel experience by taking into account the emotional state of the user. Specific embodiments for implementing the invention will be described below.

[1142] Accepting user requests

[1143] The user launches the mobility assistance application on their device and enters information about their departure and destination. This information is sent to the server in JSON format. The server stores the received request data in a database.

[1144] Emotion recognition by emotion engine

[1145] Users use their device's camera and microphone to provide facial expressions and voice. The emotion engine analyzes this data to identify the user's emotional state. The analysis results are sent to the server and stored in a database as emotion data.

[1146] Request analysis and route optimization

[1147] The server retrieves the saved request data and emotion data and passes them to an AI algorithm. This AI algorithm calculates the optimal route based on past travel data and real-time traffic conditions. If the user is feeling stressed, it will prioritize routes with less congestion or better scenery.

[1148] Utilizing video analysis

[1149] The server receives real-time video data from traffic cameras and uses video analytics to analyze traffic congestion information, checking congestion at major intersections and public transport, and integrating the results with emotion data to reevaluate the optimal route.

[1150] Utilizing MaaS

[1151] The server combines the results of AI analysis, real-time video analysis information, and emotion data to generate the optimal combination of transportation methods. For example, it can generate a route combining trains and buses, and automatically make reservations for each mode of transportation.

[1152] Route suggestions

[1153] The server obtains the optimal route and discount information for stores along the route, and generates a proposal. The generated route proposal is notified to the device and provided to the user. For example, it may include information such as "Take the train as the route from station A to station B, and a 10% discount coupon is being offered at a cafe along the way."

[1154] ONEID payment execution

[1155] If the user is satisfied with the proposed route, they select "Confirm route and pay." The server then sends payment information to the integrated payment system and processes the credit card information. If the payment is successful, the result is notified to the terminal, and ticket information for each mode of transportation is displayed. The user can then use this information to smoothly begin their journey.

[1156] Specific examples

[1157] For example, a user inputs "home" as the departure point and "office" as the destination, and sets the departure time to "now." In addition, the user provides facial expression data showing stress levels via a camera. The server passes the request data and emotional data to an AI algorithm, which calculates the optimal route of "bus → train → walk." It checks the bus congestion status and automatically reserves a bus ticket for 8:00 AM. The user is notified of the optimal route from "home → office" and information about discounts at cafes. After the route is confirmed, the fare is processed in one lump sum via the ONEID payment system, and the ticket information is displayed.

[1158] In this way, by using this system, users can achieve an efficient and comfortable travel experience that suits their emotional state, and can also enjoy additional benefits along their travel route.

[1159] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1160] Step 1:

[1161] User: The user launches a dedicated mobility assistance application and inputs information about the departure and destination locations.

[1162] Input: Departure and destination information (e.g. "Station A", "Station B")

[1163] Output: Request data in JSON format

[1164] Specific operation: The user enters the departure and destination into the input fields in the application and presses the tap button to generate request data.

[1165] Step 2:

[1166] Terminal: Sends the entered data to the server.

[1167] Input: Request data

[1168] Output: Request data sent to the server

[1169] Specific operation: The terminal sends the generated request data to the "POST / api / requests" endpoint, which the server receives and stores in the database.

[1170] Step 3:

[1171] User: Uses the device's camera and microphone to provide facial expressions and voice.

[1172] Input: Camera video data, microphone audio data

[1173] Output: Input data to the emotion engine

[1174] Specific operation: When the user shows their facial expression to the camera or speaks into the microphone, video and audio are captured in real time.

[1175] Step 4:

[1176] Terminal: The emotion engine analyzes the user's emotional state.

[1177] Input: Camera video data, microphone audio data

[1178] Output: Emotion analysis results

[1179] How it works: The device's emotion engine analyzes video and audio data to identify the user's emotional state. For example, an AI model uses a facial expression recognition algorithm to determine stress levels.

[1180] Step 5:

[1181] Server: Receives the sentiment analysis results and stores them in a database.

[1182] Input: Sentiment analysis results

[1183] Output: Emotion data stored in a database

[1184] Specific operation: The analysis results received from the device are stored in the database as a "user ID-emotion data" pair.

[1185] Step 6:

[1186] Server: Obtains request data and emotion data and passes it to the AI ​​algorithm.

[1187] Input: Origin, Destination, and Emotion Data

[1188] Output: Optimal route

[1189] Specific operation: The server executes the "GET / api / requests / {user ID}" and "GET / api / emotions / {user ID}" APIs and passes the data to the AI ​​algorithm, which then calculates the optimal route using past movement data, traffic conditions, and emotion data.

[1190] Step 7:

[1191] Server: Acquires and analyzes real-time video data from traffic cameras.

[1192] Input: Traffic camera video data

[1193] Output: Traffic congestion information

[1194] Specific operation: The server retrieves video data using "GET / api / traffic-cameras" and inputs it into the video analysis model. The analysis results include congestion levels and traffic conditions.

[1195] Step 8:

[1196] Server: Integrates video analysis results with emotion data and reevaluates the optimal route.

[1197] Input: Traffic congestion information, emotion data

[1198] Output: Updated optimal route

[1199] Specific operation: Traffic congestion information and emotion data are passed to the server's "route reoptimization function" and the route is recalculated.

[1200] Step 9:

[1201] Server: Generates the optimal combination of transportation means and makes the necessary reservations and ticket arrangements.

[1202] Input: Optimal Route

[1203] Output: Booked transportation (e.g. train, bus tickets)

[1204] Specific operation: The server uses "POST / api / tickets" to link with the electronic reservation system and arrange for tickets.

[1205] Step 10:

[1206] Server: Notifies the user of the best route and store discount information.

[1207] Input: Optimal route, store discount information

[1208] Output: Notification message

[1209] Specific operation: The server sends a notification message to "PUSH / api / notifications" and it is displayed on the user's terminal.

[1210] Step 11:

[1211] User: Check the proposed route and make payment.

[1212] Input: Proposed route and discount information

[1213] Output: Payment information

[1214] Specific operation: When the user taps the "Confirm route and pay" button in the application, payment information is sent to the server.

[1215] Step 12:

[1216] Server: Processes payments through the integrated payment system.

[1217] Input: Payment information (credit card information, etc.)

[1218] Output: Payment result

[1219] Specific operation: The server executes "POST / api / payment" and processes the payment through the integrated payment system. If successful, it notifies the user terminal of the result.

[1220] Step 13:

[1221] Terminal: Notifies the user of the payment result and ticket information.

[1222] Input: Payment result, ticket information

[1223] Output: User notification

[1224] Specific operation: The terminal notifies the user that the payment was successful and displays ticket information for each mode of transportation.

[1225] (Application example 2)

[1226] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1227] Conventional mobility assistance systems do not take into account the user's emotional state, which means that comfort and satisfaction during travel are not fully ensured. Furthermore, users may feel stressed when multiple modes of transportation and route selection are required. Furthermore, adjusting the in-car environment during travel requires manual adjustments, which places a burden on the user.

[1228] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1229] In this invention, the server includes means for receiving information on departure and destination input by the user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for analyzing data from traffic observation equipment and public transportation in real time, means for generating a travel route by combining optimal travel modes, means for acquiring discount information for commercial facilities along the travel route and including it in a route proposal, means for notifying the user of the proposed travel route and discount information, means for processing travel mode fares using an integrated payment system, means for recognizing the user's emotional state and selecting an optimal travel route based on the emotion, and means for adjusting the internal environment of the travel mode based on the emotion, thereby enabling automatic adjustment of the optimal travel route and in-vehicle environment according to the user's emotional state.

[1230] A "user" is a person who uses the system and who inputs information about the departure point and destination.

[1231] The "starting point" is the point from which the user starts traveling.

[1232] A "destination" is a point where a user ends their journey.

[1233] "Travel data" refers to information about past travel history and traffic patterns.

[1234] "Traffic conditions" refers to the current road conditions and the operation status of public transportation.

[1235] "Traffic observation equipment" refers to devices such as cameras and sensors that are installed to monitor road and traffic conditions.

[1236] "Public transportation" refers to public means of transportation such as buses, trains, and taxis.

[1237] "Real-time analysis" is a technology that analyzes data instantly and grasps the current situation.

[1238] "Transportation" refers to any means of transportation used by a user to travel.

[1239] "Commercial facilities" refer to stores and service facilities that users can stop by while traveling.

[1240] "Discount information" is information about discounts and special offers that users can use at commercial facilities.

[1241] An "integrated payment system" is a system for processing multiple payment methods together.

[1242] "Fee" means the amount paid by a User for transportation or services provided.

[1243] "Emotional state" refers to the emotions or moods that a user is currently experiencing.

[1244] The "optimal travel route" refers to the route that allows the user to travel most comfortably and efficiently, based on past data and real-time information.

[1245] "In-vehicle environment" refers to environmental factors such as temperature, lighting, and music inside a vehicle.

[1246] "Means for adjusting the internal environment" refers to technologies and devices for automatically adjusting the temperature, music, etc. inside the vehicle.

[1247] The present invention is a system for optimizing a user's travel experience based on their emotional state. This system utilizes past travel data and real-time traffic conditions to provide an optimal travel route. It also recognizes the user's emotional state, selects an appropriate route, and adjusts the internal environment of the vehicle to support a user's comfortable travel.

[1248] Hardware and Software

[1249] Hardware used

[1250] Smartphone or smart glasses (with camera and microphone)

[1251] Self-driving cars (equipped with interior environment sensors and control devices)

[1252] Server (cloud computing environment)

[1253] Software used

[1254] EmotionRecognition: A library for analyzing emotion data in real time.

[1255] RouteOptimizer: An algorithm that selects the optimal route based on the user's emotional state.

[1256] CarControl: A control library for autonomous vehicles to coordinate routing and the in-car environment.

[1257] System processing procedure

[1258] 1. Collecting information from users

[1259] Users input their departure and destination information into the system using a smartphone or smart glasses, and also provide emotion data via a camera and microphone.

[1260] 2. Recognizing emotional states

[1261] The server uses the EmotionRecognition library to analyze facial and voice data obtained from the user in real time to identify the user's emotional state.

[1262] 3. Calculating the optimal route

[1263] The server uses the RouteOptimizer algorithm to calculate the optimal route based on past travel data and real-time traffic data, and selects a route that reflects the user's emotional state, taking into account emotional data.

[1264] 4. Real-time data analysis

[1265] The server collects real-time data from traffic observation equipment and public transport operators and analyzes traffic conditions. The analysis results are integrated with those of the emotion engine to reevaluate the most suitable route for the user.

[1266] 5. Adjusting the in-car environment

[1267] Based on the acquired emotional data, the server uses the CarControl library to adjust the in-car environment, such as temperature, music, and lighting, inside the self-driving car.

[1268] Specific examples

[1269] For example, if a user enters "home" as the departure point and "office" as the destination, they set the departure time as "now" and provide emotional data (e.g., stress) via the camera. The server passes the request data and emotional data to an AI algorithm, which calculates the optimal route: "bus → train → walk." Real-time video analysis also checks the bus congestion status. Based on this route, a bus ticket for 8:00 AM is then automatically reserved. The user is notified of the optimal route from "home to office" along with discount information for a cafe near the office. The user confirms the route, the fare is processed in one lump sum via the integrated payment system, and the ticket information is received.

[1270] Example prompts to input to the generative AI model

[1271] "Generate a program that suggests scenic routes and plays relaxation music to help users who are stressed relax."

[1272] This system allows users to have a comfortable and efficient travel experience while also enjoying additional benefits along the travel route.

[1273] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1274] Step 1:

[1275] The user inputs their departure and destination using a smartphone or smart glasses. Specifically, they launch a dedicated application and set their "home" as the departure point and their "office" as the destination through the UI. This information is sent from the application to the server, where the input data (departure point, destination) is output.

[1276] Step 2:

[1277] The device uses a camera and microphone to collect emotional data from the user. Specifically, the camera analyzes facial expressions and the microphone records voice tone. This data is analyzed in real time using the EmotionRecognition library to recognize the user's emotional state (e.g., stress). The input data (facial expression data, voice data) is output as emotional data (stress).

[1278] Step 3:

[1279] The server collects historical travel data and real-time traffic condition data. Specifically, it collects data from traffic observation equipment and public transportation APIs, and passes it to the RouteOptimizer algorithm. The input data (historical travel data, real-time traffic condition data) is output as the optimal route.

[1280] Step 4:

[1281] The server calculates the optimal travel route based on the emotion data. Specifically, it takes into account the user's emotional state (stress) and uses the RouteOptimizer algorithm to select a route with good scenery that reduces stress. The input data (emotion data, traffic condition data) is output as the emotion-based optimal route.

[1282] Step 5:

[1283] The server analyzes data from traffic observation equipment and public transportation in real time to obtain traffic congestion information. Specifically, it uses real-time video analysis technology to analyze data obtained from traffic cameras and sensors to understand congestion conditions. Input data (real-time video data) is output as traffic congestion information.

[1284] Step 6:

[1285] The server integrates traffic congestion information and emotion data and reevaluates the optimal route. Specifically, when traffic congestion information is acquired, it compares it with emotion data and recalculates the optimal travel route. The input data (emotion data, traffic congestion information) is output as the reevaluated optimal route.

[1286] Step 7:

[1287] The server obtains discount information for commercial facilities along the travel route and includes it in the route proposal. Specifically, it obtains discount information from the commercial facility API and adds it to the user's travel route. The input data (commercial facility data, optimal route) is output as a route proposal.

[1288] Step 8:

[1289] The server notifies the user of the proposed travel route and discount information. Specifically, it sends a push notification to the device, and the optimal route and discount information are displayed on the UI. The input data (route proposal, discount information) is notified to the user's device.

[1290] Step 9:

[1291] The user confirms the route and the fare is processed in bulk in the integrated payment system. Specifically, the user presses the route confirmation button within the application, and the payment system processes the fare for each mode of transportation. The input data (route confirmation data, payment information) is output as payment completion information.

[1292] Step 10:

[1293] The server adjusts the in-car environment based on the emotion data. Specifically, it adjusts the temperature, music, lighting, etc. inside the autonomous vehicle through the CarControl library. The input data (emotion data) is output as the adjusted in-car environment.

[1294] In this way, the system provides an optimal travel route and a comfortable in-car environment according to the user's emotional state.

[1295] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1296] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1297] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1298] [Fourth embodiment]

[1299] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1300] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1301] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1302] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1303] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1304] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1305] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1306] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1307] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1308] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1309] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1310] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1311] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1312] The present invention is a system designed to assist a user in efficient movement, and is implemented as follows.

[1313] 1. Acceptance of customer requests

[1314] Device: The user launches a dedicated mobility assistance application and inputs their starting point and destination. For example, if the user inputs "Station A" as the starting point and "Station B" as the destination, the input data is structured within the app.

[1315] Server: Receives the departure and destination information sent from the device and stores it in a database. This stored data is used in subsequent processes.

[1316] 2. AI-based request analysis

[1317] Server: Retrieves request data from the database and passes it to an AI algorithm, which calculates the optimal route in real time based on past travel data and traffic conditions.

[1318] Server: For example, it learns from past data which means of transportation a user prefers depending on the time of day, and presents routes based on that, taking into account traffic congestion and weather conditions.

[1319] 3. Use of video analysis

[1320] Server: Acquires the latest data from traffic cameras and public transport systems and analyzes it in real time, using image processing technology and machine learning algorithms.

[1321] Server: For example, it analyzes images from traffic cameras installed at major intersections and stations and obtains congestion and delay information in real time, making it possible to take measures such as avoiding routes with heavy traffic.

[1322] 4. Utilizing MaaS

[1323] Server: Integrates AI analysis results with real-time video analytics information to generate optimal travel combinations, including train transfers, bus trips, and taxi reservations.

[1324] Server: Automatically makes the necessary reservations and ticket arrangements. For example, if the system suggests a route that combines buses and trains, it purchases bus tickets on behalf of the user.

[1325] 5. Route suggestions

[1326] Server: Generates the optimal travel route and discount information for stores along that route, and notifies the user.

[1327] On the device: A notification appears in the user's app, showing suggested routes and store discount information. For example, the notification might say, "The best route from Station A to Station B is 30 minutes by train. There's also a 10% discount at the cafe near Station B."

[1328] 6. ONEID Payment Execution

[1329] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay," which enters the user's payment information.

[1330] Server: Sends payment information to the integrated payment system and processes the transportation fare. After receiving a successful response, sends the ticket information for each transportation to the user's terminal.

[1331] Examples:

[1332] Terminal: The user inputs "home" as the departure point and "office" as the destination. The desired departure time is "8:00 AM."

[1333] Server: Passes the request data to an AI algorithm and calculates the optimal route: "bus → train → walk." Video analysis also confirms bus congestion.

[1334] Server: Automatically book a bus ticket for 8am based on this route.

[1335] Device: Notifies the user of the optimal route from home to office, along with discount information for a cafe near the office.

[1336] Terminal: The user confirms the route and the fare is processed in bulk through the ONEID payment system. Ticket information is received, allowing for smooth travel.

[1337] The above is an embodiment of the present invention. This system enables users to travel stress-free and efficiently, and also allows users to take advantage of advantageous information along their travel route without missing it.

[1338] The processing flow will be explained below.

[1339] Step 1:

[1340] Device: The user launches the dedicated app and enters their departure and destination. The departure point is entered as "Station A" and the destination as "Station B." The user then selects "Now" as the departure time and presses the "Send Request" button.

[1341] Step 2:

[1342] Server: Receives the request data sent from the terminal. The received data includes information on the departure point (Station A), destination (Station B), and departure time (Now). The server stores this data in a database.

[1343] Step 3:

[1344] Server: Retrieves request data from the database and passes it to the AI ​​algorithm. The AI ​​calculates the optimal route based on past travel data and traffic conditions. For example, if it determines from past data that the user tends to avoid crowded areas, it will prioritize routes that avoid crowds.

[1345] Step 4:

[1346] Server: Obtains the latest information from traffic cameras and public transportation. Obtains real-time video data from the traffic camera API and analyzes traffic congestion information using video analytics technology. For example, it detects congestion at major intersections from traffic camera footage.

[1347] Step 5:

[1348] Server: Combines the results of AI analysis with the results of real-time video analysis to find the optimal means of transportation. For example, it generates the optimal route combining trains and buses.

[1349] Step 6:

[1350] Server: Makes the necessary reservations and ticket arrangements. For example, automatically reserves train tickets and arranges bus tickets at the same time. In this case, the reservation is made using the public transport company's API.

[1351] Step 7:

[1352] Server: Obtains the optimal travel route and discount information for stores along the route, and generates route suggestions. Suggestions include information such as "Take the train from station A to station B, and get a 10% discount coupon at a cafe along the way."

[1353] Step 8:

[1354] On the device: A notification appears in the user's app. The notification includes a suggested route and store discount information. For example, "The best route is a 30-minute train ride, and there's a 10% off coupon at a cafe along the way."

[1355] Step 9:

[1356] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay."

[1357] Step 10:

[1358] Server: Sends payment information to the integrated payment system. The payment system processes the user's credit card information and other details, and pays the fee in one lump sum. A response indicating successful payment is returned to the server, which then sends the response to the user's device.

[1359] Step 11:

[1360] Terminal: The user's terminal will be notified that the fare for each mode of transportation has been paid. Ticket information for each mode of transportation will also be displayed. The user can use this information to smoothly start their journey.

[1361] The above are the specific processing steps of the program for implementing the present invention. This system enables users to travel efficiently and comfortably, and also allows them to enjoy additional benefits along their travel route.

[1362] Example 1

[1363] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1364] Conventional mobility assistance systems require users to select the optimal route from their departure point to their destination and reserve and arrange multiple modes of transportation, which is cumbersome and hinders efficient travel. Furthermore, the lack of optimal route suggestions that reflect real-time traffic conditions and congestion information makes it difficult for users to travel as planned, and they often miss out on discount information that can be used during travel. To solve these issues, a system is needed that automatically proposes and arranges efficient travel routes, taking into account past travel data and real-time traffic conditions.

[1365] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1366] In this invention, the server includes means for receiving information on departure and destination entered by the user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for acquiring and analyzing data from traffic cameras and public transportation in real time, means for generating a travel route by combining optimal means of transportation and automatically making reservations and ticket arrangements, means for acquiring store discount information along the travel route and including it in the route proposal, means for notifying the user of the proposed travel route and discount information, means for processing travel fares and transmitting payment information using an integrated payment system, and means for distributing ticket information for the user's means of transportation in real time. This allows users to efficiently select and arrange travel routes without stress, and enables optimal travel taking into account real-time congestion information and discount information.

[1367] "User" refers to an individual or organization that uses the services of the mobility assistance system.

[1368] "Start point" refers to the point where the user starts their journey.

[1369] "Destination" refers to the point where the user ends their journey.

[1370] A "travel route" refers to a route from a departure point to a destination, and may include a route that combines multiple modes of transportation.

[1371] "Past journey data" refers to data relating to journeys taken in the past by the user or other users, including information such as preferred modes of transportation and time of day.

[1372] "Traffic conditions" refers to factors that affect travel, such as the current congestion situation on roads and public transportation, delay information, and accident information.

[1373] "Traffic cameras" refer to cameras installed to monitor congestion and congestion on roads and public transport.

[1374] "Public transport" refers to means of transport available to the general public, such as buses, trains and taxis.

[1375] "Real time" refers to ongoing situations and information, and indicates the exact moment an event occurs.

[1376] "Analysis" refers to procedures and methods for processing acquired data to facilitate the extraction of specific information.

[1377] "Reservation" refers to the act of a user promising to use something at a later date, including arranging transportation or services.

[1378] "Ticket" means a permit or ticket required to use a particular means of transportation or service.

[1379] "Store discount information" refers to information about discounts and special offers offered by stores along the route.

[1380] "Notification" refers to providing information to users, and is often done through smartphone applications.

[1381] An "integrated payment system" refers to a system for processing payments for different services and products in one place.

[1382] "Fare" means the amount paid for using a means of transportation or service.

[1383] "Payment information" refers to information related to a user's payment, including credit card information and electronic money details.

[1384] "Transportation ticket information" refers to the tickets and reservation information required to use transportation.

[1385] A "system" refers to a mechanism in which multiple elements work together to achieve a specific purpose.

[1386] The system is designed to efficiently support users' mobility and is implemented using multiple components, including hardware and software such as user devices, a central server, traffic cameras, databases, AI algorithms, and an integrated payment system.

[1387] User terminal

[1388] Users use a smartphone or tablet with a dedicated mobility support application installed. This application provides an interface for users to input their departure and destination locations. Payment information and notifications are also entered on this device.

[1389] Central Server

[1390] The server plays a central role in the system. It receives the origin and destination information sent by the user and stores it in a database. It then retrieves the stored data and passes it to an AI algorithm to calculate the optimal travel route.

[1391] Database

[1392] The database stores information such as user request data, past movement data, and traffic conditions, allowing the server to quickly retrieve and process the necessary data.

[1393] AI algorithms

[1394] The AI ​​algorithm calculates the optimal route based on past travel data and real-time traffic conditions. This algorithm learns the user's travel habits and preferences and can suggest the optimal means of transportation for each time of day. For example, it may suggest a "bus → train → walk" route during the morning commute.

[1395] Traffic Camera

[1396] Traffic cameras are installed at major intersections and stations to monitor traffic conditions in real time. The server acquires video data from these cameras and analyzes it using image processing technology. The analysis results include congestion status and delay information.

[1397] Integrated Payment System

[1398] The integrated payment system is used to process all the fares for the travel route selected by the user. When the user selects "Confirm route and pay," the server sends the payment information to the integrated payment system and processes the fare.

[1399] Route suggestions and notifications

[1400] The server generates an optimal route based on the results of AI algorithms and video analysis, and also integrates discount information for stores along the route. This information is sent to the user's device, where the user can check the proposed route and discount information. For example, a notification may appear saying, "The optimal route from Station A to Station B is 30 minutes by train. There is also a 10% discount at the cafe near Station B."

[1401] Specific examples

[1402] As a concrete example, consider the case where a user inputs "home" as the departure point, "office" as the destination, and the desired departure time as "8:00 AM." The server passes the request data to an AI algorithm, which calculates the optimal route: "bus → train → walk." It also analyzes traffic camera footage to check bus congestion. Based on this route, the system automatically reserves a bus ticket for 8:00 AM. The user's device is notified of the optimal route from "home to office," along with discount information for a cafe near the office. Finally, the user confirms the route and the fare is processed in one lump sum via the ONEID payment system. The ticket information is then received by the device, allowing for a smooth journey.

[1403] The above is a specific embodiment of the present invention. This system allows users to efficiently select and arrange travel routes without stress, and enables optimal travel taking into account real-time congestion information and discount information.

[1404] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1405] Step 1:

[1406] The user starts a dedicated mobility assistance application and inputs their starting point and destination. The input information is converted into JSON format such as {"start":"Station A", "destination":"Station B"}. Input: Starting point and destination from the user. Output: Structured JSON format data.

[1407] Step 2:

[1408] The terminal sends structured JSON data to the server. The HTTPS protocol is used for transmission, and data is encrypted. Input: Structured JSON data. Output: Transmission to server.

[1409] Step 3:

[1410] The server saves the received JSON data in the database. The specific save operation uses an SQL insert statement. Input: JSON data from the terminal. Output: Request information saved in the database.

[1411] Step 4:

[1412] The server retrieves the request data from the database and passes it to the AI ​​algorithm. It retrieves the data using an SQL select statement. Input: Request data from the database. Output: Request data passed to the AI ​​algorithm.

[1413] Step 5:

[1414] The AI ​​algorithm on the server calculates the optimal travel route based on past travel data and traffic conditions. It uses a specific machine learning model (e.g., neural network). Input: Request data, past travel data, and traffic condition data. Output: Optimal travel route information.

[1415] Step 6:

[1416] The server stores the calculation results in temporary storage. For the storage operation, an in-memory database such as Redis or Memcached is used. Input: Optimal travel route information. Output: Travel route information stored in temporary storage.

[1417] Step 7:

[1418] The server retrieves the latest data from traffic cameras and public transport. It uses API calls to retrieve real-time data. Input: API call. Output: Latest data from traffic cameras and public transport.

[1419] Step 8:

[1420] The server analyzes the traffic camera video data using image processing technology. It uses a deep learning model (e.g., YOLO). Input: Traffic camera video data. Output: Analyzed congestion and delay information.

[1421] Step 9:

[1422] The server stores the analysis results in temporary storage. Input: Congestion status and delay information. Output: Analysis results stored in temporary storage.

[1423] Step 10:

[1424] The server combines the results of AI analysis and video analysis to generate the optimal combination of transportation methods. The algorithm uses a linear optimization model. Input: Optimal transportation route information and analysis results. Output: Optimal combination of transportation methods.

[1425] Step 11:

[1426] The server automatically makes the necessary reservations and arranges tickets. It uses API calls to connect with bus and train reservation systems. Input: Information on optimal combinations of transportation methods. Output: Information on completed reservations and ticket arrangements.

[1427] Step 12:

[1428] The server saves the reservation and arrangement results in temporary storage. Input: Completed reservation and ticket arrangement information. Output: Reservation and arrangement information saved in temporary storage.

[1429] Step 13:

[1430] The server generates the optimal travel route and store discount information along that route. The store discount information is obtained from an external promotion API. Input: Optimal travel route information. Output: Combination of travel route and store discount information.

[1431] Step 14:

[1432] The server sends the generated information to the user's device. The HTTPS protocol is used for communication. Input: Travel route and store discount information. Output: Notification to the user's device.

[1433] Step 15:

[1434] The device displays the notification to the user. Specifically, it uses the notification function within the app to display information on the screen. Input: Notification information from the server. Output: Visualized notification to the user.

[1435] Step 16:

[1436] The user selects "Confirm route and pay." The entered payment information is encrypted on the terminal. Input: User's payment information. Output: Encrypted payment information.

[1437] Step 17:

[1438] The terminal sends encrypted payment information to the server. The HTTPS protocol is used for communication. Input: Encrypted payment information. Output: Send to server.

[1439] Step 18:

[1440] The server sends the payment information to the integrated payment system and processes the travel fare. Input: Encrypted payment information. Output: Transmission to the integrated payment system and fare processing.

[1441] Step 19:

[1442] The server receives a success response indicating that the fare processing was successful and sends the ticket information for each means of transportation to the user's terminal. Input: Success response from the integrated payment system. Output: Ticket information sent to the user's terminal.

[1443] Step 20:

[1444] The device displays the ticket information to the user. Specifically, it uses the ticket management function within the app. Input: Ticket information from the server. Output: Ticket information displayed to the user.

[1445] (Application example 1)

[1446] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1447] In recent years, advances in autonomous driving technology have improved the convenience of user travel. However, there are still problems with finding optimal travel routes and updating routes in real time according to traffic congestion and weather conditions. Furthermore, there is a lack of personalized travel routes that meet individual user preferences, so there is room for improvement in order to provide an optimal travel experience. Furthermore, it is necessary to improve user convenience by utilizing discount information and services during travel.

[1448] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1449] In this invention, the server includes means for receiving information on departure and destination points input by a user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for analyzing data from traffic cameras and public transportation in real time, means for combining optimal travel modes to generate a travel route, means for acquiring discount information for stores along the travel route and including it in a route proposal, means for notifying the user of the proposed travel route and discount information, means for processing travel mode fares using an integrated payment system, means for updating the route in real time using an application installed in the autonomous vehicle, means for customizing the route based on user preferences, and means for taking traffic congestion and weather conditions into consideration. This allows users to obtain the optimal travel route based on real-time information, enjoy a personalized travel experience tailored to their individual preferences, and also take advantage of discount information along the travel route.

[1450] The "means for receiving information on the departure point and destination" refers to a device or software that acquires the information on the departure point and destination entered by the user and transmits it to the server.

[1451] A "means for calculating an optimal travel route" refers to an algorithm or program that calculates the most efficient and quickest travel route based on past travel data and traffic conditions.

[1452] "Means for analyzing data from traffic cameras and public transportation in real time" refers to technology and devices that analyze data obtained from traffic cameras and public transportation in real time to determine traffic congestion and delay information.

[1453] "Means for generating a travel route by combining optimal means of transportation" refers to a system or software for combining multiple means of transportation to generate the most suitable travel route for a user.

[1454] "Means for acquiring discount information of stores along a travel route and including it in a route proposal" refers to technology or devices that acquire discount information of stores along a travel route and reflect it in a travel route proposal.

[1455] The "means for notifying the user of the proposed travel route and discount information" refers to a system or application for notifying the user of the optimal travel route and discount information related to that route.

[1456] "Means for processing transportation fares using an integrated payment system" refers to an integrated payment system for processing fares for multiple transportation means in one go.

[1457] "Means for updating routes in real time through an application installed on an autonomous vehicle" refers to technology or devices for updating travel routes in real time through an application installed on an autonomous vehicle.

[1458] "Means for customizing routes based on user preferences" means a system or algorithm for providing personalized travel routes based on a user's past travel history and current preferences.

[1459] "Means that take into account traffic congestion and weather conditions" refers to technologies and devices that acquire and analyze traffic congestion and weather conditions in real time and provide optimal travel routes based on that information.

[1460] The present invention is a system designed to assist users in traveling efficiently, and is implemented as follows using an application installed in an autonomous vehicle.

[1461] First, the user launches a dedicated mobility assistance application and inputs their departure and destination. This information is sent from the user's device to the server and stored in a database. The server then calculates the optimal route based on the received departure and destination information, taking into account past travel data and real-time traffic conditions.

[1462] The calculated route is further optimized by analyzing data from traffic cameras and public transport in real time. The server analyzes the video data acquired in real time and adjusts the route based on congestion and delay information. This is done using Python and OpenCV video analysis.

[1463] Once the optimal combination of transportation modes has been determined, the server automatically makes the necessary reservations and ticket arrangements using MaaS (Mobility as a Service), and processes fares through an integrated payment system. Once all arrangements are complete, the user is notified of the proposed route and discount information for stores along the route.

[1464] One of the features of this system is that the application installed in the autonomous vehicle updates the route in real time. The server takes into account traffic congestion and weather conditions in real time and provides the autonomous vehicle with the optimal route.

[1465] Additionally, it also has the ability to customize routes based on user preferences, which takes into account the user's past travel history and current preferences to provide a more personalized travel experience.

[1466] For example, if a user inputs "home" as the departure point and "sports gym" as the destination, setting the departure time to "5:00 PM," the server will calculate a "bus → train → walk" route based on past travel data and real-time traffic information. The server then analyzes traffic camera footage to check bus congestion and automatically reserves a bus ticket for 5:00 PM. The user receives this route and discount information for cafes near the gym via the app, and the fare is processed in one lump sum via the ONEID payment system.

[1467] An example prompt might be, "Based on the user's input of origin and destination, suggest the best route for travel, taking into account traffic congestion and weather conditions. Also, provide personalized suggestions based on the user's preferences and offer discount information along the route."

[1468] As described above, the present invention is a system that can provide users with efficient, personalized, and optimal travel routes by using autonomous vehicles.

[1469] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1470] Step 1:

[1471] The user starts a dedicated mobility assistance application and inputs their starting point and destination. The starting point (e.g., "home") and destination (e.g., "sports gym") information entered by the user is sent from the terminal to the server.

[1472] Step 2:

[1473] The server receives the departure and destination information and stores it in a database. The server then retrieves past travel data and real-time traffic information from the database and calculates the optimal travel route. The input here is the departure and destination information entered by the user, as well as the past travel data and traffic information in the database, and the output is the optimal travel route.

[1474] Step 3:

[1475] The server acquires real-time data from traffic cameras and public transport and uses video analysis to determine traffic congestion and delay information. The server then uses this data to further optimize the optimal travel route. Specifically, it uses Python and OpenCV to analyze the video data and detect congestion and delays. The input is real-time data, and the output is an updated optimal travel route.

[1476] Step 4:

[1477] The server utilizes MaaS (Mobility as a Service) to combine optimal means of transportation and automatically make the necessary reservations and ticket arrangements. The server processes all transportation fares in one go through an integrated payment system. The input is the optimal travel route and reservation information, and the output is reservation confirmation and payment completion information.

[1478] Step 5:

[1479] The server notifies the user of the optimal travel route and discount information for stores along the route. This includes a function to send notifications to the user's application. The input is the optimal travel route and discount information, and the output is a notification to the user's terminal.

[1480] Step 6:

[1481] The application installed in the autonomous vehicle updates the route of the autonomous vehicle in real time based on the optimal travel route information received from the server. The input is the optimal travel route information, and the output is the updated route setting for the autonomous vehicle.

[1482] Step 7:

[1483] The server personalizes and provides travel routes based on the user's past travel history and current preferences. For example, if a user prefers a particular route based on their past travel history, that route will be preferentially suggested. The input is the user's past travel history and current preferences, and the output is a customized travel route.

[1484] Step 8:

[1485] The server acquires and analyzes traffic congestion and weather conditions in real time and provides the optimal travel route based on that information. For example, if there is a sudden change in weather, the server will suggest a route change based on that information. The input is real-time traffic congestion and weather conditions, and the output is an updated optimal travel route.

[1486] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1487] The present invention provides a personalized travel experience that corresponds to the user's emotional state by combining an emotion engine with a system that supports efficient and comfortable travel for users. Specific embodiments of the present invention are described below.

[1488] 1. Acceptance of customer requests

[1489] Device: The user launches a dedicated mobility assistance application and inputs their starting point and destination. For example, if the user inputs "Station A" as the starting point and "Station B" as the destination, the input data is structured within the app.

[1490] Server: Receives the departure and destination information sent from the device and stores it in a database. This stored data is used in subsequent processes.

[1491] 2. Emotion Recognition by Emotion Engine

[1492] Device: The user provides facial expressions and voice through the device's camera and microphone, which the emotion engine analyzes to determine the user's current emotional state.

[1493] Server: The emotion engine analyzes the user's emotion data and classifies it. For example, if the user is feeling stressed, the emotion data is classified as "stress."

[1494] 3. Request Analysis and Route Optimization

[1495] Server: Retrieves request data from the database and passes it to the AI ​​algorithm. The AI ​​calculates the optimal route based on past travel data and traffic conditions. It also takes into account emotional data and selects a route that reflects the user's emotional state.

[1496] Server: For example, if the user is feeling stressed, the AI ​​will prioritize routes with less populated areas or more scenic views.

[1497] 4. Use of video analysis

[1498] Server: Obtains the latest information from traffic cameras and public transport and analyzes it in real time. Obtains real-time video data from the traffic camera API and uses video analysis technology to analyze traffic congestion information. For example, it detects congestion at major intersections.

[1499] Server: Integrates the analysis results of the emotion engine and the video analysis results, and reevaluates the route that best suits the user's emotions.

[1500] 5. Utilizing MaaS

[1501] Server: Integrates AI analysis results, real-time video analysis information, and emotion data to generate the optimal combination of transportation methods. For example, it generates the optimal travel route combining trains and buses.

[1502] Server: Automatically makes the necessary reservations and ticket arrangements. For example, automatically reserves train tickets and arranges bus tickets at the same time. Reservations are made using the public transport API.

[1503] 6. Route suggestions

[1504] Server: Obtains the optimal travel route and discount information for stores along that route, and generates route suggestions. Suggestions include information such as "Take the train from station A to station B, and get a 10% discount coupon at a cafe along the way."

[1505] On the device: The user is notified with a suggested route and store discount information. For example, the notification might say, "The best route is a 30-minute train ride, and there's a 10% off coupon at a cafe along the way."

[1506] 7. ONEID Payment Execution

[1507] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay."

[1508] Server: Sends payment information to the integrated payment system. The payment system processes the user's credit card information and other details, and pays the fee in one lump sum. A response indicating successful payment is returned to the server, which then sends the response to the user's device.

[1509] Terminal: The user's terminal will be notified that the fare for each mode of transportation has been paid. Ticket information for each mode of transportation will also be displayed. The user can use this information to smoothly start their journey.

[1510] Examples:

[1511] Terminal: The user inputs "home" as the departure point, "office" as the destination, and "immediately" as the departure time. In addition, the user provides emotional data (e.g., stress) via the camera.

[1512] Server: Passes request data and emotion data to an AI algorithm, and calculates the optimal route: "bus → train → walk." Video analysis also confirms bus congestion.

[1513] Server: Automatically book a bus ticket for 8am based on this route.

[1514] Device: Notifies the user of the optimal route from home to office, along with discount information for a cafe near the office.

[1515] Terminal: The user confirms the route, the fare is processed in bulk through the ONEID payment system, and ticket information is received.

[1516] The above is an embodiment of the present invention. This system allows users to travel efficiently and comfortably according to their emotional state, and also allows them to enjoy additional benefits along their travel route.

[1517] The processing flow will be explained below.

[1518] Step 1:

[1519] Device: The user launches the dedicated app and enters their departure and destination. They enter "Home" as the departure point and "Office" as the destination. They then set the departure time to "Now" and press the "Send Request" button.

[1520] Step 2:

[1521] Server: Receives the request data sent from the device and stores it in a database. The stored data includes the departure location "Home", the destination "Office", and the departure time "Now".

[1522] Step 3:

[1523] Device: The user provides their own emotional data using the app's camera or microphone. For example, emotions are recognized by showing facial expressions to the camera or speaking into the microphone.

[1524] Step 4:

[1525] Server: The emotion engine receives facial expressions and voice data sent from the device and performs emotion recognition processing. The emotion engine saves the analysis results in a database and records that the user is feeling "stressed."

[1526] Step 5:

[1527] Server: Retrieves request data and emotion data from the database and passes them to an AI algorithm. The AI ​​calculates the optimal route based on past travel data, real-time traffic conditions, and the user's emotion data. For example, if the user is feeling stressed, it will prioritize routes with less congestion or more beautiful scenery.

[1528] Step 6:

[1529] Server: Obtains real-time video data from the traffic camera API and performs video analysis. Using video analysis technology, congestion status on major roads and in areas with heavy traffic can be grasped in real time.

[1530] Step 7:

[1531] Server: Integrates AI analysis results with real-time video analysis results to generate the optimal combination of transportation methods. For example, if trains are crowded, prioritize bus and walking routes.

[1532] Step 8:

[1533] Server: Makes necessary reservations and arranges tickets. For example, automatically reserves bus tickets using an API and arranges train transfer information at the same time.

[1534] Step 9:

[1535] Server: Generates optimal travel routes and store discount information along the route to notify users. Suggestions include "use the bus as a route from home to the office, with a 10% discount coupon at a cafe along the way."

[1536] Step 10:

[1537] Device: Receives "optimal route notification" and displays the notification content. You can check the suggested route and discount information. For example, it may say, "The optimal route from home to the office is a 30-minute bus ride, and there is a 10% off coupon at the cafe along the way."

[1538] Step 11:

[1539] Terminal: The user is satisfied with the proposed route and selects "Confirm route and pay."

[1540] Step 12:

[1541] Server: Sends payment information to the integrated payment system and processes the fee. The payment system processes the credit card information and charges the fee in one lump sum. A success response is returned to the server and sent to the user's device.

[1542] Step 13:

[1543] Terminal: The user's terminal is notified that the fare for each mode of transportation has been paid. In addition, ticket information for each mode of transportation is displayed, allowing the user to smoothly start traveling.

[1544] In this way, the system of the present invention provides an optimal travel route that takes into account the user's emotional state, helping to make the trip more comfortable and efficient, and also improving the overall user experience by utilizing discounts and special offers along the route.

[1545] Example 2

[1546] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1547] While modern mobility assistance systems can provide efficient travel routes, they struggle to provide a personalized travel experience that takes into account the user's emotional state. For example, when a user is feeling stressed, it is desirable to avoid congested routes, but current systems struggle to provide such flexible route planning. Furthermore, they rarely properly integrate real-time traffic information and store discount information to provide users with optimal travel options.

[1548] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving information on a departure point and a destination input by a user, a means for acquiring and analyzing the user's emotional state from a camera or a microphone, a means for calculating an optimal travel route based on past travel data, traffic conditions, and the user's emotional state, a means for analyzing data from traffic cameras and public transportation in real time, a means for generating a travel route by combining optimal means of transportation, a means for acquiring discount information from stores along the travel route and including it in a route proposal, a means for notifying the user of the proposed travel route and discount information, and a means for processing travel fare information using an integrated payment system. This makes it possible to provide a personalized travel experience according to the user's emotional state.

[1549] "User" refers to a person who uses the system to receive travel route suggestions.

[1550] "Start point" refers to the point where the user starts.

[1551] "Destination" refers to the point the user wishes to reach.

[1552] "Emotional state" refers to the user's current mental or emotional state, including stress, joy, and the like.

[1553] "Camera" refers to a device that captures video.

[1554] "Microphone" refers to a device that captures sound.

[1555] "Analysis" refers to the act of analyzing acquired data and extracting useful information.

[1556] "Movement data" refers to data including past movement history and route information.

[1557] "Traffic conditions" refers to information showing the current traffic flow and degree of congestion.

[1558] "Real-time analytics" refers to the process of processing data instantly and obtaining results instantly.

[1559] "Travel route" refers to the route taken from a starting point to a destination.

[1560] "Store discount information" refers to information about discounts and coupons available at stores.

[1561] "Suggestion" refers to the recommended route or information provided to the user by the system.

[1562] "Notification" refers to the act of the system transmitting information to a user terminal.

[1563] "Integrated payment system" refers to a system for processing multiple payment methods at once.

[1564] The present invention relates to a system for assisting a user in efficient and comfortable travel. The system can provide a personalized travel experience by taking into account the emotional state of the user. Specific embodiments for implementing the invention will be described below.

[1565] Accepting user requests

[1566] The user launches the mobility assistance application on their device and enters information about their departure and destination. This information is sent to the server in JSON format. The server stores the received request data in a database.

[1567] Emotion recognition by emotion engine

[1568] Users use their device's camera and microphone to provide facial expressions and voice. The emotion engine analyzes this data to identify the user's emotional state. The analysis results are sent to the server and stored in a database as emotion data.

[1569] Request analysis and route optimization

[1570] The server retrieves the saved request data and emotion data and passes them to an AI algorithm. This AI algorithm calculates the optimal route based on past travel data and real-time traffic conditions. If the user is feeling stressed, it will prioritize routes with less congestion or better scenery.

[1571] Utilizing video analysis

[1572] The server receives real-time video data from traffic cameras and uses video analytics to analyze traffic congestion information, checking congestion at major intersections and public transport, and integrating the results with emotion data to reevaluate the optimal route.

[1573] Utilizing MaaS

[1574] The server combines the results of AI analysis, real-time video analysis information, and emotion data to generate the optimal combination of transportation methods. For example, it can generate a route combining trains and buses, and automatically make reservations for each mode of transportation.

[1575] Route suggestions

[1576] The server obtains the optimal route and discount information for stores along the route, and generates a proposal. The generated route proposal is notified to the device and provided to the user. For example, it may include information such as "Take the train as the route from station A to station B, and a 10% discount coupon is being offered at a cafe along the way."

[1577] ONEID payment execution

[1578] If the user is satisfied with the proposed route, they select "Confirm route and pay." The server then sends payment information to the integrated payment system and processes the credit card information. If the payment is successful, the result is notified to the terminal, and ticket information for each mode of transportation is displayed. The user can then use this information to smoothly begin their journey.

[1579] Specific examples

[1580] For example, a user inputs "home" as the departure point and "office" as the destination, and sets the departure time to "now." In addition, the user provides facial expression data showing stress levels via a camera. The server passes the request data and emotional data to an AI algorithm, which calculates the optimal route of "bus → train → walk." It checks the bus congestion status and automatically reserves a bus ticket for 8:00 AM. The user is notified of the optimal route from "home → office" and information about discounts at cafes. After the route is confirmed, the fare is processed in one lump sum via the ONEID payment system, and the ticket information is displayed.

[1581] In this way, by using this system, users can achieve an efficient and comfortable travel experience that suits their emotional state, and can also enjoy additional benefits along their travel route.

[1582] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1583] Step 1:

[1584] User: The user launches a dedicated mobility assistance application and inputs information about the departure and destination locations.

[1585] Input: Departure and destination information (e.g. "Station A", "Station B")

[1586] Output: Request data in JSON format

[1587] Specific operation: The user enters the departure and destination into the input fields in the application and presses the tap button to generate request data.

[1588] Step 2:

[1589] Terminal: Sends the entered data to the server.

[1590] Input: Request data

[1591] Output: Request data sent to the server

[1592] Specific operation: The terminal sends the generated request data to the "POST / api / requests" endpoint, which the server receives and stores in the database.

[1593] Step 3:

[1594] User: Uses the device's camera and microphone to provide facial expressions and voice.

[1595] Input: Camera video data, microphone audio data

[1596] Output: Input data to the emotion engine

[1597] Specific operation: When the user shows their facial expression to the camera or speaks into the microphone, video and audio are captured in real time.

[1598] Step 4:

[1599] Terminal: The emotion engine analyzes the user's emotional state.

[1600] Input: Camera video data, microphone audio data

[1601] Output: Emotion analysis results

[1602] How it works: The device's emotion engine analyzes video and audio data to identify the user's emotional state. For example, an AI model uses a facial expression recognition algorithm to determine stress levels.

[1603] Step 5:

[1604] Server: Receives the sentiment analysis results and stores them in a database.

[1605] Input: Sentiment analysis results

[1606] Output: Emotion data stored in a database

[1607] Specific operation: The analysis results received from the device are stored in the database as a "user ID-emotion data" pair.

[1608] Step 6:

[1609] Server: Obtains request data and emotion data and passes it to the AI ​​algorithm.

[1610] Input: Origin, Destination, and Emotion Data

[1611] Output: Optimal route

[1612] Specific operation: The server executes the "GET / api / requests / {user ID}" and "GET / api / emotions / {user ID}" APIs and passes the data to the AI ​​algorithm, which then calculates the optimal route using past movement data, traffic conditions, and emotion data.

[1613] Step 7:

[1614] Server: Acquires and analyzes real-time video data from traffic cameras.

[1615] Input: Traffic camera video data

[1616] Output: Traffic congestion information

[1617] Specific operation: The server retrieves video data using "GET / api / traffic-cameras" and inputs it into the video analysis model. The analysis results include congestion levels and traffic conditions.

[1618] Step 8:

[1619] Server: Integrates video analysis results with emotion data and reevaluates the optimal route.

[1620] Input: Traffic congestion information, emotion data

[1621] Output: Updated optimal route

[1622] Specific operation: Traffic congestion information and emotion data are passed to the server's "route reoptimization function" and the route is recalculated.

[1623] Step 9:

[1624] Server: Generates the optimal combination of transportation means and makes the necessary reservations and ticket arrangements.

[1625] Input: Optimal Route

[1626] Output: Booked transportation (e.g. train, bus tickets)

[1627] Specific operation: The server uses "POST / api / tickets" to link with the electronic reservation system and arrange for tickets.

[1628] Step 10:

[1629] Server: Notifies the user of the best route and store discount information.

[1630] Input: Optimal route, store discount information

[1631] Output: Notification message

[1632] Specific operation: The server sends a notification message to "PUSH / api / notifications" and it is displayed on the user's terminal.

[1633] Step 11:

[1634] User: Check the proposed route and make payment.

[1635] Input: Proposed route and discount information

[1636] Output: Payment information

[1637] Specific operation: When the user taps the "Confirm route and pay" button in the application, payment information is sent to the server.

[1638] Step 12:

[1639] Server: Processes payments through the integrated payment system.

[1640] Input: Payment information (credit card information, etc.)

[1641] Output: Payment result

[1642] Specific operation: The server executes "POST / api / payment" and processes the payment through the integrated payment system. If successful, it notifies the user terminal of the result.

[1643] Step 13:

[1644] Terminal: Notifies the user of the payment result and ticket information.

[1645] Input: Payment result, ticket information

[1646] Output: User notification

[1647] Specific operation: The terminal notifies the user that the payment was successful and displays ticket information for each mode of transportation.

[1648] (Application example 2)

[1649] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1650] Conventional mobility assistance systems do not take into account the user's emotional state, which means that comfort and satisfaction during travel are not fully ensured. Furthermore, users may feel stressed when multiple modes of transportation and route selection are required. Furthermore, adjusting the in-car environment during travel requires manual adjustments, which places a burden on the user.

[1651] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1652] In this invention, the server includes means for receiving information on departure and destination input by the user, means for calculating an optimal travel route based on past travel data and traffic conditions, means for analyzing data from traffic observation equipment and public transportation in real time, means for generating a travel route by combining optimal travel modes, means for acquiring discount information for commercial facilities along the travel route and including it in a route proposal, means for notifying the user of the proposed travel route and discount information, means for processing travel mode fares using an integrated payment system, means for recognizing the user's emotional state and selecting an optimal travel route based on the emotion, and means for adjusting the internal environment of the travel mode based on the emotion, thereby enabling automatic adjustment of the optimal travel route and in-vehicle environment according to the user's emotional state.

[1653] A "user" is a person who uses the system and who inputs information about the departure point and destination.

[1654] The "starting point" is the point from which the user starts traveling.

[1655] A "destination" is a point where a user ends their journey.

[1656] "Travel data" refers to information about past travel history and traffic patterns.

[1657] "Traffic conditions" refers to the current road conditions and the operation status of public transportation.

[1658] "Traffic observation equipment" refers to devices such as cameras and sensors that are installed to monitor road and traffic conditions.

[1659] "Public transportation" refers to public means of transportation such as buses, trains, and taxis.

[1660] "Real-time analysis" is a technology that analyzes data instantly and grasps the current situation.

[1661] "Transportation" refers to any means of transportation used by a user to travel.

[1662] "Commercial facilities" refer to stores and service facilities that users can stop by while traveling.

[1663] "Discount information" is information about discounts and special offers that users can use at commercial facilities.

[1664] An "integrated payment system" is a system for processing multiple payment methods together.

[1665] "Fee" means the amount paid by a User for transportation or services provided.

[1666] "Emotional state" refers to the emotions or moods that a user is currently experiencing.

[1667] The "optimal travel route" refers to the route that allows the user to travel most comfortably and efficiently, based on past data and real-time information.

[1668] "In-vehicle environment" refers to environmental factors such as temperature, lighting, and music inside a vehicle.

[1669] "Means for adjusting the internal environment" refers to technologies and devices for automatically adjusting the temperature, music, etc. inside the vehicle.

[1670] The present invention is a system for optimizing a user's travel experience based on their emotional state. This system utilizes past travel data and real-time traffic conditions to provide an optimal travel route. It also recognizes the user's emotional state, selects an appropriate route, and adjusts the internal environment of the vehicle to support a user's comfortable travel.

[1671] Hardware and Software

[1672] Hardware used

[1673] Smartphone or smart glasses (with camera and microphone)

[1674] Self-driving cars (equipped with interior environment sensors and control devices)

[1675] Server (cloud computing environment)

[1676] Software used

[1677] EmotionRecognition: A library for analyzing emotion data in real time.

[1678] RouteOptimizer: An algorithm that selects the optimal route based on the user's emotional state.

[1679] CarControl: A control library for autonomous vehicles to coordinate routing and the in-car environment.

[1680] System processing procedure

[1681] 1. Collecting information from users

[1682] Users input their departure and destination information into the system using a smartphone or smart glasses, and also provide emotion data via a camera and microphone.

[1683] 2. Recognizing emotional states

[1684] The server uses the EmotionRecognition library to analyze facial and voice data obtained from the user in real time to identify the user's emotional state.

[1685] 3. Calculating the optimal route

[1686] The server uses the RouteOptimizer algorithm to calculate the optimal route based on past travel data and real-time traffic data, and selects a route that reflects the user's emotional state, taking into account emotional data.

[1687] 4. Real-time data analysis

[1688] The server collects real-time data from traffic observation equipment and public transport operators and analyzes traffic conditions. The analysis results are integrated with those of the emotion engine to reevaluate the most suitable route for the user.

[1689] 5. Adjusting the in-car environment

[1690] Based on the acquired emotional data, the server uses the CarControl library to adjust the in-car environment, such as temperature, music, and lighting, inside the self-driving car.

[1691] Specific examples

[1692] For example, if a user enters "home" as the departure point and "office" as the destination, they set the departure time as "now" and provide emotional data (e.g., stress) via the camera. The server passes the request data and emotional data to an AI algorithm, which calculates the optimal route: "bus → train → walk." Real-time video analysis also checks the bus congestion status. Based on this route, a bus ticket for 8:00 AM is then automatically reserved. The user is notified of the optimal route from "home to office" along with discount information for a cafe near the office. The user confirms the route, the fare is processed in one lump sum via the integrated payment system, and the ticket information is received.

[1693] Example prompts to input to the generative AI model

[1694] "Generate a program that suggests scenic routes and plays relaxation music to help users who are stressed relax."

[1695] This system allows users to have a comfortable and efficient travel experience while also enjoying additional benefits along the travel route.

[1696] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1697] Step 1:

[1698] The user inputs their departure and destination using a smartphone or smart glasses. Specifically, they launch a dedicated application and set their "home" as the departure point and their "office" as the destination through the UI. This information is sent from the application to the server, where the input data (departure point, destination) is output.

[1699] Step 2:

[1700] The device uses a camera and microphone to collect emotional data from the user. Specifically, the camera analyzes facial expressions and the microphone records voice tone. This data is analyzed in real time using the EmotionRecognition library to recognize the user's emotional state (e.g., stress). The input data (facial expression data, voice data) is output as emotional data (stress).

[1701] Step 3:

[1702] The server collects historical travel data and real-time traffic condition data. Specifically, it collects data from traffic observation equipment and public transportation APIs, and passes it to the RouteOptimizer algorithm. The input data (historical travel data, real-time traffic condition data) is output as the optimal route.

[1703] Step 4:

[1704] The server calculates the optimal travel route based on the emotion data. Specifically, it takes into account the user's emotional state (stress) and uses the RouteOptimizer algorithm to select a route with good scenery that reduces stress. The input data (emotion data, traffic condition data) is output as the emotion-based optimal route.

[1705] Step 5:

[1706] The server analyzes data from traffic observation equipment and public transportation in real time to obtain traffic congestion information. Specifically, it uses real-time video analysis technology to analyze data obtained from traffic cameras and sensors to understand congestion conditions. Input data (real-time video data) is output as traffic congestion information.

[1707] Step 6:

[1708] The server integrates traffic congestion information and emotion data and reevaluates the optimal route. Specifically, when traffic congestion information is acquired, it compares it with emotion data and recalculates the optimal travel route. The input data (emotion data, traffic congestion information) is output as the reevaluated optimal route.

[1709] Step 7:

[1710] The server obtains discount information for commercial facilities along the travel route and includes it in the route proposal. Specifically, it obtains discount information from the commercial facility API and adds it to the user's travel route. The input data (commercial facility data, optimal route) is output as a route proposal.

[1711] Step 8:

[1712] The server notifies the user of the proposed travel route and discount information. Specifically, it sends a push notification to the device, and the optimal route and discount information are displayed on the UI. The input data (route proposal, discount information) is notified to the user's device.

[1713] Step 9:

[1714] The user confirms the route and the fare is processed in bulk in the integrated payment system. Specifically, the user presses the route confirmation button within the application, and the payment system processes the fare for each mode of transportation. The input data (route confirmation data, payment information) is output as payment completion information.

[1715] Step 10:

[1716] The server adjusts the in-car environment based on the emotion data. Specifically, it adjusts the temperature, music, lighting, etc. inside the autonomous vehicle through the CarControl library. The input data (emotion data) is output as the adjusted in-car environment.

[1717] In this way, the system provides an optimal travel route and a comfortable in-car environment according to the user's emotional state.

[1718] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1719] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1720] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1721] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1722] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1723] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1724] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1725] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1726] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1727] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1728] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1729] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1730] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1731] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the networ...

Claims

1. means for receiving origin and destination information input by a user; A method for calculating the optimal travel route based on past travel data and traffic conditions, A means of analyzing traffic camera and public transport data in real time, A means for generating a travel route by combining optimal means of transportation; A means for acquiring discount information from stores along a travel route and including the information in a route proposal; means for notifying the user of the suggested travel route and discount information; a means for processing transportation fares using an integrated payment system; A system including:

2. 2. The system according to claim 1, further comprising means for calculating an optimal travel route taking into account the user's past travel history.

3. The system according to claim 1, further comprising means for acquiring traffic congestion information using real-time video analysis technology.

Images