system

Abstract

We provide the system. [Solution] The information processing system provides a means for obtaining schedule information from the schedule management system, A means for automatically identifying the departure point and destination point based on the information of places to visit included in the aforementioned itinerary information, A means for searching for a route between the departure point and the destination point and calculating the travel cost, A means for automatically registering the calculated travel expenses in the expense settlement system, A means of automatically calculating transportation costs using planned route data, by searching for the optimal travel distance between the departure point and destination point using a route search service, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including: receiving a user utterance; adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot; encoding the prompt; and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional reimbursement of transportation expenses, people with many business trips or outings need to manually input transportation expense information into a separate expense reimbursement system based on the visit schedule already entered in the scheduling device, and this duplication of effort results in a waste of time and labor, which is a problem. Also, input errors during transportation expense calculation and reimbursement, as well as the trouble of setting approvers, are major issues.

Means for Solving the Problems

[0005] This invention provides a system that automatically acquires schedule information from a schedule management device and identifies the departure and arrival points based on the destination information, thereby eliminating manual input by the user. Furthermore, it automatically searches for travel routes, calculates travel expenses, and registers them in the expense settlement system. In addition, it streamlines the approval process with a function that automatically sets approvers. The registered information is notified to the user's terminal, and the user can check and modify the content, significantly improving work efficiency.

[0006] An "information processing device" is a device that acquires information from a schedule management device and has the function of automatically processing that information.

[0007] A "schedule management device" refers to a system or software that allows users to input and manage their schedules, and is a device that provides schedule information.

[0008] "Scheduled information" refers to data that includes details about the planned visit, such as the name of the company to be visited, its address, and the date and time of the visit.

[0009] "Destination information" refers to information related to the destination, and primarily includes data such as company names and addresses.

[0010] The "starting point" refers to the location where the user begins their visit or travel, and is usually the user's office location.

[0011] "Destination" refers to the destination that the user is aiming for in order to visit or perform their duties.

[0012] A "travel route" is information that shows the optimal route for moving from a point of origin to a point of destination, and may include multiple modes of transportation or routes.

[0013] "Travel expenses" refer to the financial costs incurred when using a travel route, such as transportation fees.

[0014] A "expense reimbursement system" is a system or software used to manage expenses and travel expenses and to carry out the reimbursement process.

[0015] "Approver" refers to a person who has the authority to review an application and determine approval or rejection in the expense settlement process.

[0016] "User terminal" refers to a device such as a computer or smartphone used by a user, which receives notifications from an information processing device and has functions for confirmation and correction.

Brief Description of Drawings

[0017] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12]It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the language used in the following description will be explained.

[0020] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] This invention is a system that uses an information processing device to acquire scheduled visit information from a schedule management device and automatically settle travel expenses. The system uses the schedule information entered by the user in advance to identify the departure and arrival points and calculate the optimal travel route and its cost. This significantly reduces the user's input work and minimizes errors.

[0039] The information processing device acquires data from the schedule management device via an API and uses an external geocoding service to obtain accurate location information based on the name and address of the visited company. Furthermore, it uses a route search service to collect the optimal travel route from the starting point to the destination and its transportation costs.

[0040] For example, suppose a user enters "XYZ Corporation, Minato-ku, Tokyo" as a planned visit in their calendar. The server retrieves this information and searches for the optimal route from the user's office in Shinjuku-ku, Tokyo. The calculated travel expenses, along with the resulting route data, are automatically registered in the expense reimbursement system.

[0041] Furthermore, the expense reimbursement system can automatically assign approvers based on the user's department and position. This ensures a quick and reliable approval process. Users receive notifications on their devices, can review the details of registered travel expenses, and make corrections as needed. Finally, once the user selects "Submit," the travel expense claim is finalized, and the approval flow begins.

[0042] This system frees users from complex settlement procedures and improves operational efficiency. Because the entire system is cloud-based, it offers flexible scalability and can be customized to meet the diverse needs of various companies.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The server retrieves schedule information from the schedule management device. This schedule information includes the name, address, and date and time of the visit. This data is periodically retrieved via an API and prepared for use within the system.

[0046] Step 2:

[0047] Based on the schedule information acquired by the server, the address of the destination is used to call the Geocoding service to obtain location information. The address data is sent to the API to obtain the precise latitude and longitude, which are then recorded as the departure and arrival points.

[0048] Step 3:

[0049] The server uses location information of the departure point and destination to perform a route search on a route search service. To obtain the optimal travel route and the means of transportation to be used, it calculates the route and travel time using an external transportation API.

[0050] Step 4:

[0051] The server calculates travel expenses based on the route search results. It aggregates the fares for each mode of transport and generates the total amount as settlement data.

[0052] Step 5:

[0053] The server calculates travel expense data and registers it in the expense reimbursement system. The visit date, destination information, and calculated travel expenses are converted into the system's appropriate data format and sent.

[0054] Step 6:

[0055] The server automatically configures the approver information required for the approval flow based on the user's department information. The system identifies appropriate approvers based on the user's job title and authority, and integrates them into the workflow.

[0056] Step 7:

[0057] The device sends a notification to the user informing them that travel expenses have been automatically registered. A confirmation screen is displayed on the device, allowing the user to review the details and make corrections if necessary.

[0058] Step 8:

[0059] The user receives a notification and reviews the travel expense request. The user chooses either "revise" or "submit," and finally submits the request to initiate the approval process, thus completing the travel expense request.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] Existing scheduling management systems require users to manually calculate travel expenses and submit separate applications for each visit, which is time-consuming and raises concerns about errors. Furthermore, manual approval processes hinder efficiency and make quick decision-making difficult.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for acquiring schedule data from a schedule management device, means for acquiring coordinate data using a location information service, means for searching for a route between a departure point and a destination point and calculating the fare, and means for automatically registering it in the expense settlement system. This enables automatic calculation and registration of transportation expenses, reduces the workload on users, and realizes a quick and accurate approval process.

[0065] An "information processing device" is a device that acquires data from a schedule management device and performs tasks such as identifying location information and calculating routes.

[0066] A "schedule management device" is a platform for users to input their planned visits and destinations.

[0067] "Schedule data" refers to information such as destinations and dates entered into the schedule management device.

[0068] "Destination data" refers to information included in the schedule data that indicates the address or company name of the destination.

[0069] "Starting point" refers to location information indicating the starting point of the movement.

[0070] "Destination" refers to location information that indicates the destination of a journey.

[0071] A "location information service" is an external service used to obtain latitude and longitude coordinate data from information such as addresses.

[0072] "Coordinate data" refers to numerical data that indicates the latitude and longitude of a specific location.

[0073] A "route" is information that shows the path traveled between a starting point and a destination.

[0074] "Fare" refers to the amount of money spent on transportation.

[0075] An "expense reimbursement system" is a system that registers calculated fares and manages the approval and reimbursement process.

[0076] An "approval officer" is a person responsible for reviewing and approving submitted expense applications.

[0077] A "user terminal" is a device used by a user to receive notifications and to review and correct expense claims.

[0078] This invention is a system that automatically settles travel expenses based on schedule data acquired from a schedule management device. The main components of the system are an information processing device, a location information service, a route search service, and an expense settlement system. Each component is described below.

[0079] The server functions as an information processing device, retrieving schedule data entered by the user into the schedule management device via an API. The retrieved schedule data includes the name and address of the company to be visited. Based on this, the server uses external location services, such as Google® Maps API or OpenStreetMap API, to obtain precise coordinate data. This process converts the address of the destination into latitude and longitude, clearly identifying the location of the destination.

[0080] Next, the server uses a route search service to calculate the optimal route from the starting point (e.g., the user's office) to the destination (the place to be visited). Services such as the Google Maps Directions API are used. Based on this route data, the server calculates the fare. In this process, accurate fare calculations are performed by referring to public transport fare information and taxi fare tables.

[0081] The calculated fare is immediately registered in the expense reimbursement system. The system automatically assigns an approver based on the user's affiliation information. This allows the approval process to begin quickly, resulting in increased operational efficiency.

[0082] Users receive a notification via their device that their travel expense claim has been registered. At that time, they can check details such as fares and destinations and make corrections as needed. Once all information has been confirmed and they click "Submit," the application is officially completed and the approval process begins.

[0083] As a concrete example, suppose a user enters "ABC Company, Minato Ward" into their calendar. Based on this data, the server searches for the optimal travel route from the user's office in Shinjuku Ward and calculates the fare. This information is then used to automate expense reporting. This process can also be verified by using prompts such as "Tell me how long this route takes and how much the transportation costs."

[0084] This system significantly reduces the effort required for expense reimbursement and improves operational efficiency. Furthermore, because it is provided as a cloud-based system, it allows for flexible expansion and customization.

[0085] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0086] Step 1:

[0087] Users enter data about their planned visits into the schedule management device. This data includes the name and address of the place they are visiting. The schedules entered by the user serve as basic information for subsequent processing.

[0088] Step 2:

[0089] The server retrieves schedule data from the schedule management device via an API. The input data retrieved includes the address and company name of the destination. The server uses this input data to query an external location information service, converts the address into latitude and longitude coordinate data, and accurately identifies the destination information.

[0090] Step 3:

[0091] The server uses a route search service to calculate the optimal travel route, taking the user's starting point (office location) and the specified destination coordinates as input. This is done using services such as the Google Maps Directions API. This calculation considers time and distance, selecting the most efficient route from multiple options.

[0092] Step 4:

[0093] The server references fare information to calculate fares based on the calculated travel route. Route information is input, and based on this, it calculates public transport fares and, if necessary, taxi fares, and outputs fare data.

[0094] Step 5:

[0095] The server automatically registers the calculated fare data into the expense reimbursement system. The data entered includes fare information and destination information, and payment information is then registered and managed in the reimbursement system based on this data.

[0096] Step 6:

[0097] The terminal notifies the user about registered travel expenses and visit information. The user receives this information, reviews the details, and makes corrections as needed. This action is a final confirmation step before the user actually completes the application.

[0098] Step 7:

[0099] Once the user selects "Submit" after reviewing, the device initiates the approval process. The user's final action confirms the application without requiring any further input, allowing the approval flow to proceed efficiently.

[0100] (Application Example 1)

[0101] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0102] In traditional operations, managing travel schedules and processing travel expenses for client visits were cumbersome and time-consuming. In particular, the calculation and approval processes for travel expenses were prone to human error, highlighting the need for increased efficiency. Furthermore, identifying optimal travel routes and managing costs became a significant burden without adequate tools and information. There is a need for technology that can address these challenges and enable efficient and accurate travel and expense reimbursement.

[0103] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0104] In this invention, the server includes means for acquiring schedule information from a schedule management system, means for automatically identifying departure and arrival points based on visit location information, and means for searching for the optimal travel distance using a route search service and automatically calculating transportation costs using the planned route data. This automates schedule management and transportation expense settlement, enabling users to improve work efficiency and accurate processing.

[0105] An "information processing system" is a system that uses computers to acquire, process, and manage data.

[0106] A "schedule management system" is software or a platform that records and manages a user's schedule and appointments.

[0107] "Schedule information" refers to data about appointments registered by the user, including the date, time, and location.

[0108] "Place information" refers to information about places the user plans to visit, specifically including addresses and facility names.

[0109] The "starting point" refers to the location where the user begins their journey, and usually refers to the user's office or home.

[0110] A "destination point" refers to the final destination a user must reach to reach their destination.

[0111] A "route search service" is an online or offline service that calculates and provides the optimal route from a starting point to a destination.

[0112] "Distance traveled" refers to the distance between the starting point and the destination, and is usually calculated by walking or traveling by vehicle.

[0113] "Transportation expenses" refer to the costs incurred for travel, and specifically include train fares, bus fares, and taxi fares.

[0114] An "expense settlement system" is a management system designed to streamline the application and settlement of expenses within a company.

[0115] "User terminal" refers to a device used by the user for operation, such as a smartphone or personal computer.

[0116] "User" refers to an individual who operates this system and inputs or verifies information.

[0117] The system that realizes this invention consists of an information processing system, a schedule management system, a route search service, and an expense settlement system. This system, in particular, enables the efficient management of a user's schedule and the accurate calculation and settlement of transportation expenses related to travel to visited locations, using a mobile information terminal such as a smartphone.

[0118] The server connects to the scheduling management system to retrieve the user's schedule information. For example, the Google Calendar API or Outlook API can be used for this purpose. Based on the retrieved schedule information, the Google Maps Geocoding API can be used to identify the departure and arrival points based on the visited locations.

[0119] The server then uses a route search service to find the optimal route from the departure point to the destination point. During this process, the Google Maps Directions API is used to obtain route information, including the shortest distance and estimated travel time. Furthermore, the Google Maps Distance Matrix API is used to automatically calculate travel expenses.

[0120] The user's mobile device receives calculation results in real time and provides an interface that allows the user to review this information and make corrections as needed. Ultimately, the calculated travel expenses are automatically registered in the expense settlement system (e.g., SAP Concur), and the supervisor approval process is made visible.

[0121] As a concrete example, when a user visits a specific city for sales activities, they enter the address information of their destination into a terminal. The system automatically calculates the route from the user's starting point (for example, an office in Tokyo) to the destination and calculates the transportation expenses. Through this series of operations, the user can avoid errors when submitting transportation expense claims, resulting in efficient business operations.

[0122] An example of a prompt for the generated AI model would be: "Explain the procedure for calculating the optimal route and transportation costs to the next destination and automatically registering them in the expense settlement system."

[0123] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0124] Step 1:

[0125] The server retrieves schedule information from the schedule management system. Specifically, it retrieves the schedule entered by the user via API and collects data including the address of the destination. At this point, the input is the user's schedule, and the output is the address information of the destination.

[0126] Step 2:

[0127] The server uses a geocoding service to determine the latitude and longitude of the departure and destination points based on the destination information. It uses the Google Maps Geocoding API to obtain precise location information from the address. The input is the address of the destination, and the output provides the latitude and longitude information of the departure and destination points.

[0128] Step 3:

[0129] The server uses a route search service to find the optimal route. It obtains optimal route information using the Google Maps Directions API and calculates the travel distance and travel time between the departure point and the destination point. The input is the latitude and longitude information of the departure point and destination point, and the output is the optimal route information.

[0130] Step 4:

[0131] The server calculates transportation costs based on distance and mode of transport. It uses the Google Maps Distance Matrix API to calculate costs for various modes of transport (e.g., public transport, taxi). The input is optimal route information, and the output is the calculated transportation cost.

[0132] Step 5:

[0133] The terminal receives the calculation results from the server. The user checks the details of the transportation expenses on their smartphone or computer screen and makes corrections as needed. The input is the calculated transportation expenses, and the output is obtained with the necessary corrections made by the user.

[0134] Step 6:

[0135] The server registers the finalized travel expenses in the expense settlement system. The expenses are automatically sent to the system, and the application is displayed to the approver. The input is travel expenses confirmed by the user, and the output is expense application information awaiting approval.

[0136] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0137] This invention is a system that combines an optimized travel expense settlement system in an information processing device with an emotion engine that analyzes user emotions. When a user enters a visit schedule into a schedule management device, the system can collect and utilize user emotion data simultaneously with that information.

[0138] Specifically, the server retrieves the name, address, and date and time of the visit as scheduled information. Furthermore, an emotion engine analyzes the user's emotions and adjusts the system's response to match their real-time emotional state. For example, if the user is feeling stressed, the system adjusts to send notifications in a gentler manner to ensure that important information is not missed.

[0139] The emotion engine analyzes the user's emotions from their voice tone and facial expressions, and adjusts the visit schedule based on that analysis. It can optimize visit priorities and offer suggestions to reduce the user's stress level.

[0140] Furthermore, in the travel expense reimbursement process, the server uses visit information to identify the departure and arrival points, searches for the travel route and expenses, and automatically registers them in the reimbursement system. The registered information is notified to the user via a terminal, and the user can review and modify the application details through an emotionally responsive interface.

[0141] For example, when a user enters "a certain company, a certain location" as a planned visit, if the emotion engine detects a positive emotion from the user's facial expression, the system will notify the user of the registration in a cheerful tone. Conversely, if a negative emotion is detected, the notification will be presented more cautiously than usual, providing a confirmation process that takes the user's emotions into consideration.

[0142] In this way, the system can recognize the user's emotions and use that information to optimally adjust each step of the travel expense reimbursement process, thereby improving the user experience.

[0143] The following describes the processing flow.

[0144] Step 1:

[0145] The server retrieves visit schedules from the schedule management device. This includes the name and address of the company to be visited, as well as the date and time of the visit. It also periodically retrieves data via API and prepares it for use within the system.

[0146] Step 2:

[0147] The server activates the emotion engine and analyzes the user's current emotions. During this process, the user's camera and microphone are used to transmit facial expressions and voice tone to the emotion engine, thereby acquiring emotion data in real time.

[0148] Step 3:

[0149] The emotion engine analyzes the user's emotional state and sends the results back to the server. It determines whether the emotion is positive, neutral, or negative, and based on that, decides on the appropriate response for subsequent processes.

[0150] Step 4:

[0151] The server performs route searching based on information obtained from the origin and sentiment analysis. Using external transportation APIs, it selects a route that minimizes user stress and detects the travel time and cost.

[0152] Step 5:

[0153] The server automatically registers the calculated route and cost into the expense reimbursement system. During registration, it generates a detailed message tailored to the user's emotional state and sends it to the expense reimbursement system.

[0154] Step 6:

[0155] The device displays a notification to the user. If the user is relaxed, the notification is presented in a normal format; if the user is tense or stressed, the notification is presented in a softer tone and interface, providing a careful review process.

[0156] Step 7:

[0157] The user receives a notification on their device, reviews the travel expense request, and makes corrections if necessary. The UI guides the user according to their emotional state, and the user confirms the request by selecting "Submit," at which point the approval flow automatically begins.

[0158] (Example 2)

[0159] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0160] In modern information management, there is a demand for both efficient expense reimbursement and responses that are sensitive to the user's feelings. While conventional systems have the functionality to automate expense reimbursement, they lack consideration for the user's emotional state, limiting their ability to improve the user experience. As a result, the burden on users, especially in busy schedules or stressful environments, is not reduced.

[0161] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0162] In this invention, the server includes means for acquiring schedule information from a schedule management device, means for analyzing user emotion data related to the schedule information and adjusting the system's response according to the emotion, and means for generating schedule adjustment proposals based on the user's emotional state using a generative AI model. This enables a travel expense settlement process and schedule adjustment that takes the user's emotional state into consideration.

[0163] An "information processing device" is an electronic device used for collecting, analyzing, and processing data.

[0164] A "schedule management device" is a device or software that provides an interface for users to input and manage their visit schedules and appointments.

[0165] "Place information" refers to data about places that users record as planned visits, specifically including the name of the place, address, and date and time of visit.

[0166] "Departure point" refers to the point where the user begins their journey.

[0167] "Destination" refers to the point that the user is expected to reach through their travels.

[0168] "Travel route" refers to route information that shows the means of transportation and the route taken from the departure point to the destination point.

[0169] "Travel expenses" refers to the amount of money required for transportation and related expenses for travel between the departure point and the destination.

[0170] A "expense reimbursement system" is a system for recording travel expenses and automatically processing reimbursements.

[0171] "Emotional data" refers to the results of an analysis that indicates the user's emotional state, and is obtained through methods such as voice tone and facial expression analysis.

[0172] A "generative AI model" is an artificial intelligence model trained for data analysis and inference, and is used for analyzing sentiment data.

[0173] A "schedule adjustment proposal" is information that shows a new schedule suggestion that has been adjusted according to the user's schedule and emotional state.

[0174] This invention is a system that combines travel expense settlement and user sentiment analysis using an information processing device. The system includes a server, a terminal, and user behavior-based interactions.

[0175] The server retrieves schedule information entered by the user from the schedule management device and analyzes destination information based on this information. Destination information includes the destination name and detailed address information registered by the user as a scheduled visit. Furthermore, the server drives an emotion engine to analyze the user's voice tone and facial expressions in real time, collecting user emotion data.

[0176] Generative AI models are used to analyze emotional data. These models identify the user's emotional state and generate optimal schedule adjustments and system responses based on that state. To improve the accuracy of the emotional analysis, a large dataset is used as a pre-trained model.

[0177] As a concrete example, consider a scenario where a user enters "Company X, City Y, Date and Time Z" as destination information. If the emotion engine determines that the user's facial expression is positive, the server sets the system's notification tone to a brighter one and presents the schedule. Conversely, if the user shows signs of stress, the notification will be delivered in a cautious and gentle tone. This ensures that the user does not miss important information and can use the system comfortably.

[0178] An example of a prompt message is: "The user has entered a scheduled visit: Company A, Tokyo Office, December 25, 2023, 10:00 AM. Please consider the user's current emotional state." Based on this, the system provides the user with an appropriate response and adjustment suggestions.

[0179] In this way, the present invention aims to improve the user experience by realizing travel expense settlement and schedule adjustment that take user emotions into consideration.

[0180] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0181] Step 1:

[0182] The user enters their visit schedule into the schedule management device. The user enters the name of the destination, address, and date and time of the visit, and this information is collected by the schedule management device. The entered information is sent from the terminal to the server.

[0183] Step 2:

[0184] The server receives the schedule information sent from the terminal and stores it in the database. Based on this information, the server automatically identifies the departure and arrival points. To identify these points, the server performs a cross-referencing process with a map database and calculates the travel route and travel costs.

[0185] Step 3:

[0186] The server drives the emotion engine, analyzing the user's voice tone and facial expression data. Real-time audio and video data acquired from the user's device is input, and the generating AI model analyzes the user's emotional state. As a result of the analysis, the user's emotional data is generated.

[0187] Step 4:

[0188] The server uses an AI model to generate appropriate system responses and schedule adjustments based on the user's emotional data. For example, if the user is feeling stressed, the urgency of the schedule is re-evaluated, and adjustments to reduce the burden are output.

[0189] Step 5:

[0190] The server automatically registers the calculated travel expenses into the travel expense reimbursement system. Based on the departure point, arrival point, and travel route information, it registers expense items and prepares the data to be sent to the terminal.

[0191] Step 6:

[0192] The server generates a schedule adjustment plan and travel expense reimbursement information, which is then sent to the terminal and notified to the user. The notification displays the information using an interface tailored to the user's emotional state. Through this emotionally responsive UI, the user can review and, if necessary, modify the information provided by the system.

[0193] (Application Example 2)

[0194] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0195] Modern travel expense reimbursement and scheduling systems lack features that consider user emotions, leading to situations where users are likely to experience stress. In particular, as there is a need to improve the user experience in managing the prioritization of scheduled visits and information exchange, these systems are unable to respond flexibly in accordance with user emotions, which poses a significant challenge.

[0196] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0197] In this invention, the server includes means for acquiring time information from a time information manager, means for automatically determining the departure and arrival points based on destination information, and emotion analysis means for analyzing the user's emotions and adjusting the response of the information processing device according to the user's real-time emotional state. This enables efficient management of the user's visit schedule and expense settlement in a manner that takes the user's emotions into consideration, thereby reducing their mental burden.

[0198] An "information processing device" is a device that has the functions of acquiring, analyzing, storing, and communicating data, and that provides a response according to the user's request.

[0199] A "time information manager" is a device or software that manages a user's schedule and plans, and makes necessary data available in conjunction with other systems.

[0200] "Destination information" refers to information about the place the user plans to visit, and specifically includes data such as the name, address, and estimated arrival time of the destination.

[0201] "Departure point and arrival point" refer to the place where a journey begins and the place where it ends, respectively, and serve as the time and spatial reference points in a travel plan.

[0202] "Emotion analysis methods" are technologies that analyze emotions from data such as the tone of a user's voice and facial expressions, and understand the user's emotional state in real time.

[0203] A "cost settlement system" is a mechanism that calculates and records travel expenses and manages and communicates that information to users and approvers.

[0204] To implement this invention, it is necessary to build a system in which a server, acting as an information processing device, is central and operates in conjunction with the user's terminal. The server obtains the user's schedule information from a time information manager and analyzes destination information based on that data. Based on the destination information, the server automatically identifies the departure and arrival points. In this process, the server uses map information services such as the Google Maps API to search for a specific travel route and calculate an estimated travel cost.

[0205] Furthermore, as a means of emotion analysis, the server collaborates with home robots or smart devices equipped with microphones and cameras to analyze the user's voice tone and facial expressions in real time. In this process, generative AI models such as OpenAI's GPT are used to generate appropriate responses that match the user's emotional state. For example, if the server determines that the user is feeling stressed, it instructs the robot to provide content to help the user relax, and the robot then makes suggestions to the user.

[0206] When settling expenses, the server notifies the user's terminal of travel costs, and the user can review the details and make corrections as needed on their terminal. As an example of a prompt, by inputting a sentence such as, "Perform sentiment analysis regarding the next scheduled visit and suggest rescheduling if the user is feeling stressed, include a function that analyzes emotions based on the user's tone of voice and facial expressions and suggests relaxing music," an appropriate response can be obtained.

[0207] For example, if a user schedules a work meeting and the system observes that they are busy just before the meeting, it will provide a gentle voice reminder and recommend relaxing activities after the meeting. This reduces the user's mental burden while enabling smooth schedule management and expense reimbursement.

[0208] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0209] Step 1:

[0210] The server retrieves schedule information from the user's terminal via a time information manager. The retrieved information includes the name of the destination, address, and scheduled date and time of the visit. This aggregates the data that forms the basis for the server's next processing.

[0211] Step 2:

[0212] The server automatically identifies the departure and arrival points using the Google Maps API and other tools based on the acquired destination information. The output of this process confirms the specific departure and arrival points, which are then used in the next process.

[0213] Step 3:

[0214] The server searches for a travel route between the specified origin and destination points and calculates the associated travel costs. Map information is used to search for the travel route, and travel costs are calculated based on a set pricing structure. This results in the travel costs being calculated as cost data.

[0215] Step 4:

[0216] The user's device acquires data on the user's voice tone and facial expressions through robots and smart devices. This data is sent to a server to analyze the user's real-time emotional state.

[0217] Step 5:

[0218] The server uses a generative AI model to analyze the user's emotions based on the transmitted data and generates a response based on the results. This response is appropriate to the user's emotional state. For example, if stress is detected, the server instructs the robot to provide content to help the user relax.

[0219] Step 6:

[0220] The server automatically registers the calculated travel expenses with the expense settlement system and notifies the user's terminal of the result. The user can then review this information on their terminal and make corrections as needed. This ensures the accuracy and convenience of the settlement data.

[0221] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0222] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0223] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0224] [Second Embodiment]

[0225] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0226] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0227] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0228] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0229] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0230] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0231] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0232] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0233] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0234] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0235] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0236] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0237] This invention is a system that uses an information processing device to acquire scheduled visit information from a schedule management device and automatically settle travel expenses. The system uses the schedule information entered by the user in advance to identify the departure and arrival points and calculate the optimal travel route and its cost. This significantly reduces the user's input work and minimizes errors.

[0238] The information processing device acquires data from the schedule management device via an API and uses an external geocoding service to obtain accurate location information based on the name and address of the visited company. Furthermore, it uses a route search service to collect the optimal travel route from the starting point to the destination and its transportation costs.

[0239] For example, suppose a user enters "XYZ Corporation, Minato-ku, Tokyo" as a planned visit in their calendar. The server retrieves this information and searches for the optimal route from the user's office in Shinjuku-ku, Tokyo. The calculated travel expenses, along with the resulting route data, are automatically registered in the expense reimbursement system.

[0240] Furthermore, the expense reimbursement system can automatically assign approvers based on the user's department and position. This ensures a quick and reliable approval process. Users receive notifications on their devices, can review the details of registered travel expenses, and make corrections as needed. Finally, once the user selects "Submit," the travel expense claim is finalized, and the approval flow begins.

[0241] This system frees users from complex settlement procedures and improves operational efficiency. Because the entire system is cloud-based, it offers flexible scalability and can be customized to meet the diverse needs of various companies.

[0242] The following describes the processing flow.

[0243] Step 1:

[0244] The server retrieves schedule information from the schedule management device. This schedule information includes the name, address, and date and time of the visit. This data is periodically retrieved via an API and prepared for use within the system.

[0245] Step 2:

[0246] Based on the schedule information acquired by the server, the address of the destination is used to call the Geocoding service to obtain location information. The address data is sent to the API to obtain the precise latitude and longitude, which are then recorded as the departure and arrival points.

[0247] Step 3:

[0248] The server uses location information of the departure point and destination to perform a route search on a route search service. To obtain the optimal travel route and the means of transportation to be used, it calculates the route and travel time using an external transportation API.

[0249] Step 4:

[0250] The server calculates travel expenses based on the route search results. It aggregates the fares for each mode of transport and generates the total amount as settlement data.

[0251] Step 5:

[0252] The server calculates travel expense data and registers it in the expense reimbursement system. The visit date, destination information, and calculated travel expenses are converted into the system's appropriate data format and sent.

[0253] Step 6:

[0254] The server automatically configures the approver information required for the approval flow based on the user's department information. The system identifies appropriate approvers based on the user's job title and authority, and integrates them into the workflow.

[0255] Step 7:

[0256] The device sends a notification to the user informing them that travel expenses have been automatically registered. A confirmation screen is displayed on the device, allowing the user to review the details and make corrections if necessary.

[0257] Step 8:

[0258] The user receives a notification and reviews the travel expense request. The user chooses either "revise" or "submit," and finally submits the request to initiate the approval process, thus completing the travel expense request.

[0259] (Example 1)

[0260] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0261] Existing scheduling management systems require users to manually calculate travel expenses and submit separate applications for each visit, which is time-consuming and raises concerns about errors. Furthermore, manual approval processes hinder efficiency and make quick decision-making difficult.

[0262] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0263] In this invention, the server includes means for acquiring schedule data from a schedule management device, means for acquiring coordinate data using a location information service, means for searching for a route between a departure point and a destination point and calculating the fare, and means for automatically registering it in the expense settlement system. This enables automatic calculation and registration of transportation expenses, reduces the workload on users, and realizes a quick and accurate approval process.

[0264] An "information processing device" is a device that acquires data from a schedule management device and performs tasks such as identifying location information and calculating routes.

[0265] A "schedule management device" is a platform for users to input their planned visits and destinations.

[0266] "Schedule data" refers to information such as destinations and dates entered into the schedule management device.

[0267] "Destination data" refers to information included in the schedule data that indicates the address or company name of the destination.

[0268] "Starting point" refers to location information indicating the starting point of the movement.

[0269] "Destination" refers to location information that indicates the destination of a journey.

[0270] A "location information service" is an external service used to obtain latitude and longitude coordinate data from information such as addresses.

[0271] "Coordinate data" refers to numerical data that indicates the latitude and longitude of a specific location.

[0272] A "route" is information that shows the path traveled between a starting point and a destination.

[0273] "Fare" refers to the amount of money spent on transportation.

[0274] An "expense reimbursement system" is a system that registers calculated fares and manages the approval and reimbursement process.

[0275] An "approval officer" is a person responsible for reviewing and approving submitted expense applications.

[0276] A "user terminal" is a device used by a user to receive notifications and to review and correct expense claims.

[0277] This invention is a system that automatically settles travel expenses based on schedule data acquired from a schedule management device. The main components of the system are an information processing device, a location information service, a route search service, and an expense settlement system. Each component is described below.

[0278] The server functions as an information processing device, retrieving schedule data entered by the user into the schedule management device via an API. This retrieved schedule data includes the name and address of the visited company. Based on this, the server uses external location services, such as the Google Maps API or OpenStreetMap API, to obtain precise coordinate data. This process converts the address of the visited location into latitude and longitude, clearly identifying the destination's location.

[0279] Next, the server uses a route search service to calculate the optimal route from the starting point (e.g., the user's office) to the destination (the place to be visited). Services such as the Google Maps Directions API are used. Based on this route data, the server calculates the fare. In this process, accurate fare calculations are performed by referring to public transport fare information and taxi fare tables.

[0280] The calculated fare is immediately registered in the expense reimbursement system. The system automatically assigns an approver based on the user's affiliation information. This allows the approval process to begin quickly, resulting in increased operational efficiency.

[0281] Users receive a notification via their device that their travel expense claim has been registered. At that time, they can check details such as fares and destinations and make corrections as needed. Once all information has been confirmed and they click "Submit," the application is officially completed and the approval process begins.

[0282] As a concrete example, suppose a user enters "ABC Company, Minato Ward" into their calendar. Based on this data, the server searches for the optimal travel route from the user's office in Shinjuku Ward and calculates the fare. This information is then used to automate expense reporting. This process can also be verified by using prompts such as "Tell me how long this route takes and how much the transportation costs."

[0283] With this system, users can significantly reduce the time and effort required for transportation expense settlement and improve work efficiency. In addition, since it is provided as a cloud-based system, flexible expansion and customization are possible.

[0284] The flow of the specific process in Example 1 will be described using FIG. 11.

[0285] Step 1:

[0286] The user inputs the data of the planned visit into the schedule management device. This data includes the name of the destination and its address. The schedule input by the user serves as the basic information for subsequent processing.

[0287] Step 2:

[0288] The server obtains the schedule data from the schedule management device via the API. The input data obtained includes the address of the destination and the company name, etc. The server uses this input data to query an external location information service, converts the address into coordinate data of latitude and longitude, and accurately identifies the destination information.

[0289] Step 3:

[0290] The server uses the route search service with the coordinate information of the office location as the departure point of the user and the identified arrival point as the input to calculate the optimal travel route. Here, services such as Google Maps Directions API are used. This calculation takes into account time and distance and selects the most efficient route from multiple routes.

[0291] Step 4:

[0292] The server refers to the freight information to calculate the freight based on the calculated travel route. The input is the route information, and based on this, the freight of public transportation and the taxi fare as required are calculated, and the freight data is output.

[0293] Step 5:

[0294] The server automatically registers the calculated fare data into the expense reimbursement system. The data entered includes fare information and destination information, and payment information is then registered and managed in the reimbursement system based on this data.

[0295] Step 6:

[0296] The terminal notifies the user about registered travel expenses and visit information. The user receives this information, reviews the details, and makes corrections as needed. This action is a final confirmation step before the user actually completes the application.

[0297] Step 7:

[0298] Once the user selects "Submit" after reviewing, the device initiates the approval process. The user's final action confirms the application without requiring any further input, allowing the approval flow to proceed efficiently.

[0299] (Application Example 1)

[0300] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0301] In traditional operations, managing travel schedules and processing travel expenses for client visits were cumbersome and time-consuming. In particular, the calculation and approval processes for travel expenses were prone to human error, highlighting the need for increased efficiency. Furthermore, identifying optimal travel routes and managing costs became a significant burden without adequate tools and information. There is a need for technology that can address these challenges and enable efficient and accurate travel and expense reimbursement.

[0302] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0303] In this invention, the server includes means for obtaining schedule information from a schedule management system, means for automatically identifying a departure point and an arrival point based on visit location information, and means for searching for an optimal travel distance using a route search service and automatically calculating transportation costs using the planned route data. As a result, schedule management and transportation cost settlement are automated, enabling users to improve work efficiency and perform accurate processing.

[0304] An "information processing system" is a system that uses a computer to acquire, process, and manage data.

[0305] A "schedule management system" is software or a platform that records and manages a user's schedule and appointments.

[0306] "Schedule information" is data on a user's registered schedule, including date, time, and location.

[0307] "Visit location information" is information about the location a user plans to visit, specifically referring to an address or facility name.

[0308] A "departure point" refers to the starting point of a user's movement, usually referring to the user's office or home.

[0309] An "arrival point" refers to the final destination point a user needs to reach.

[0310] A "route search service" is an online or offline service that calculates and provides the optimal route from a departure location to a destination.

[0311] "Travel distance" refers to the distance between a departure point and an arrival point, usually calculated based on walking or vehicle movement.

[0312] "Transportation cost" refers to the expenses incurred during movement, specifically including train fares, bus fares, taxi fares, etc.

[0313] An "expense settlement system" is a management system designed to streamline the application and settlement of expenses within a company.

[0314] "User terminal" refers to a device used by the user for operation, such as a smartphone or personal computer.

[0315] "User" refers to an individual who operates this system and inputs or verifies information.

[0316] The system that realizes this invention consists of an information processing system, a schedule management system, a route search service, and an expense settlement system. This system, in particular, enables the efficient management of a user's schedule and the accurate calculation and settlement of transportation expenses related to travel to visited locations, using a mobile information terminal such as a smartphone.

[0317] The server connects to the scheduling management system to retrieve the user's schedule information. For example, the Google Calendar API or Outlook API can be used for this purpose. Based on the retrieved schedule information, the Google Maps Geocoding API can be used to identify the departure and arrival points based on the visited locations.

[0318] The server then uses a route search service to find the optimal route from the departure point to the destination point. During this process, the Google Maps Directions API is used to obtain route information, including the shortest distance and estimated travel time. Furthermore, the Google Maps Distance Matrix API is used to automatically calculate travel expenses.

[0319] The user's mobile device receives calculation results in real time and provides an interface that allows the user to review this information and make corrections as needed. Ultimately, the calculated travel expenses are automatically registered in the expense settlement system (e.g., SAP Concur), and the supervisor approval process is made visible.

[0320] As a concrete example, when a user visits a specific city for sales activities, they enter the address information of their destination into a terminal. The system automatically calculates the route from the user's starting point (for example, an office in Tokyo) to the destination and calculates the transportation expenses. Through this series of operations, the user can avoid errors when submitting transportation expense claims, resulting in efficient business operations.

[0321] An example of a prompt for the generated AI model would be: "Explain the procedure for calculating the optimal route and transportation costs to the next destination and automatically registering them in the expense settlement system."

[0322] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0323] Step 1:

[0324] The server retrieves schedule information from the schedule management system. Specifically, it retrieves the schedule entered by the user via API and collects data including the address of the destination. At this point, the input is the user's schedule, and the output is the address information of the destination.

[0325] Step 2:

[0326] The server uses a geocoding service to determine the latitude and longitude of the departure and destination points based on the destination information. It uses the Google Maps Geocoding API to obtain precise location information from the address. The input is the address of the destination, and the output provides the latitude and longitude information of the departure and destination points.

[0327] Step 3:

[0328] The server uses a route search service to find the optimal route. It obtains optimal route information using the Google Maps Directions API and calculates the travel distance and travel time between the departure point and the destination point. The input is the latitude and longitude information of the departure point and destination point, and the output is the optimal route information.

[0329] Step 4:

[0330] The server calculates transportation costs based on distance and mode of transport. It uses the Google Maps Distance Matrix API to calculate costs for various modes of transport (e.g., public transport, taxi). The input is optimal route information, and the output is the calculated transportation cost.

[0331] Step 5:

[0332] The terminal receives the calculation results from the server. The user checks the details of the transportation expenses on their smartphone or computer screen and makes corrections as needed. The input is the calculated transportation expenses, and the output is obtained with the necessary corrections made by the user.

[0333] Step 6:

[0334] The server registers the finalized travel expenses in the expense settlement system. The expenses are automatically sent to the system, and the application is displayed to the approver. The input is travel expenses confirmed by the user, and the output is expense application information awaiting approval.

[0335] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0336] This invention is a system that combines an optimized travel expense settlement system in an information processing device with an emotion engine that analyzes user emotions. When a user enters a visit schedule into a schedule management device, the system can collect and utilize user emotion data simultaneously with that information.

[0337] Specifically, the server retrieves the name, address, and date and time of the visit as scheduled information. Furthermore, an emotion engine analyzes the user's emotions and adjusts the system's response to match their real-time emotional state. For example, if the user is feeling stressed, the system adjusts to send notifications in a gentler manner to ensure that important information is not missed.

[0338] The emotion engine analyzes the user's emotions from their voice tone and facial expressions, and adjusts the visit schedule based on that analysis. It can optimize visit priorities and offer suggestions to reduce the user's stress level.

[0339] Furthermore, in the travel expense reimbursement process, the server uses visit information to identify the departure and arrival points, searches for the travel route and expenses, and automatically registers them in the reimbursement system. The registered information is notified to the user via a terminal, and the user can review and modify the application details through an emotionally responsive interface.

[0340] For example, when a user enters "a certain company, a certain location" as a planned visit, if the emotion engine detects a positive emotion from the user's facial expression, the system will notify the user of the registration in a cheerful tone. Conversely, if a negative emotion is detected, the notification will be presented more cautiously than usual, providing a confirmation process that takes the user's emotions into consideration.

[0341] In this way, the system can recognize the user's emotions and use that information to optimally adjust each step of the travel expense reimbursement process, thereby improving the user experience.

[0342] The following describes the processing flow.

[0343] Step 1:

[0344] The server retrieves visit schedules from the schedule management device. This includes the name and address of the company to be visited, as well as the date and time of the visit. It also periodically retrieves data via API and prepares it for use within the system.

[0345] Step 2:

[0346] The server activates the emotion engine and analyzes the user's current emotions. During this process, the user's camera and microphone are used to transmit facial expressions and voice tone to the emotion engine, thereby acquiring emotion data in real time.

[0347] Step 3:

[0348] The emotion engine analyzes the user's emotional state and sends the results back to the server. It determines whether the emotion is positive, neutral, or negative, and based on that, decides on the appropriate response for subsequent processes.

[0349] Step 4:

[0350] The server performs route searching based on information obtained from the origin and sentiment analysis. Using external transportation APIs, it selects a route that minimizes user stress and detects the travel time and cost.

[0351] Step 5:

[0352] The server automatically registers the calculated route and cost into the expense reimbursement system. During registration, it generates a detailed message tailored to the user's emotional state and sends it to the expense reimbursement system.

[0353] Step 6:

[0354] The device displays a notification to the user. If the user is relaxed, the notification is presented in a normal format; if the user is tense or stressed, the notification is presented in a softer tone and interface, providing a careful review process.

[0355] Step 7:

[0356] The user receives a notification on their device, reviews the travel expense request, and makes corrections if necessary. The UI guides the user according to their emotional state, and the user confirms the request by selecting "Submit," at which point the approval flow automatically begins.

[0357] (Example 2)

[0358] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0359] In modern information management, there is a demand for both efficient expense reimbursement and responses that are sensitive to the user's feelings. While conventional systems have the functionality to automate expense reimbursement, they lack consideration for the user's emotional state, limiting their ability to improve the user experience. As a result, the burden on users, especially in busy schedules or stressful environments, is not reduced.

[0360] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0361] In this invention, the server includes means for acquiring schedule information from a schedule management device, means for analyzing user emotion data related to the schedule information and adjusting the system's response according to the emotion, and means for generating schedule adjustment proposals based on the user's emotional state using a generative AI model. This enables a travel expense settlement process and schedule adjustment that takes the user's emotional state into consideration.

[0362] An "information processing device" is an electronic device used for collecting, analyzing, and processing data.

[0363] A "schedule management device" is a device or software that provides an interface for users to input and manage their visit schedules and appointments.

[0364] "Place information" refers to data about places that users record as planned visits, specifically including the name of the place, address, and date and time of visit.

[0365] "Departure point" refers to the point where the user begins their journey.

[0366] "Destination" refers to the point that the user is expected to reach through their travels.

[0367] "Travel route" refers to route information that shows the means of transportation and the route taken from the departure point to the destination point.

[0368] "Travel expenses" refers to the amount of money required for transportation and related expenses for travel between the departure point and the destination.

[0369] A "expense reimbursement system" is a system for recording travel expenses and automatically processing reimbursements.

[0370] "Emotional data" refers to the results of an analysis that indicates the user's emotional state, and is obtained through methods such as voice tone and facial expression analysis.

[0371] A "generative AI model" is an artificial intelligence model trained for data analysis and inference, and is used for analyzing sentiment data.

[0372] A "schedule adjustment proposal" is information that shows a new schedule suggestion that has been adjusted according to the user's schedule and emotional state.

[0373] This invention is a system that combines travel expense settlement and user sentiment analysis using an information processing device. The system includes a server, a terminal, and user behavior-based interactions.

[0374] The server retrieves schedule information entered by the user from the schedule management device and analyzes destination information based on this information. Destination information includes the destination name and detailed address information registered by the user as a scheduled visit. Furthermore, the server drives an emotion engine to analyze the user's voice tone and facial expressions in real time, collecting user emotion data.

[0375] Generative AI models are used to analyze emotional data. These models identify the user's emotional state and generate optimal schedule adjustments and system responses based on that state. To improve the accuracy of the emotional analysis, a large dataset is used as a pre-trained model.

[0376] As a concrete example, consider a scenario where a user enters "Company X, City Y, Date and Time Z" as destination information. If the emotion engine determines that the user's facial expression is positive, the server sets the system's notification tone to a brighter one and presents the schedule. Conversely, if the user shows signs of stress, the notification will be delivered in a cautious and gentle tone. This ensures that the user does not miss important information and can use the system comfortably.

[0377] An example of a prompt message is: "The user has entered a scheduled visit: Company A, Tokyo Office, December 25, 2023, 10:00 AM. Please consider the user's current emotional state." Based on this, the system provides the user with an appropriate response and adjustment suggestions.

[0378] In this way, the present invention aims to improve the user experience by realizing travel expense settlement and schedule adjustment that take user emotions into consideration.

[0379] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0380] Step 1:

[0381] The user enters their visit schedule into the schedule management device. The user enters the name of the destination, address, and date and time of the visit, and this information is collected by the schedule management device. The entered information is sent from the terminal to the server.

[0382] Step 2:

[0383] The server receives the schedule information sent from the terminal and stores it in the database. Based on this information, the server automatically identifies the departure and arrival points. To identify these points, the server performs a cross-referencing process with a map database and calculates the travel route and travel costs.

[0384] Step 3:

[0385] The server drives the emotion engine, analyzing the user's voice tone and facial expression data. Real-time audio and video data acquired from the user's device is input, and the generating AI model analyzes the user's emotional state. As a result of the analysis, the user's emotional data is generated.

[0386] Step 4:

[0387] The server uses an AI model to generate appropriate system responses and schedule adjustments based on the user's emotional data. For example, if the user is feeling stressed, the urgency of the schedule is re-evaluated, and adjustments to reduce the burden are output.

[0388] Step 5:

[0389] The server automatically registers the calculated travel expenses into the travel expense reimbursement system. Based on the departure point, arrival point, and travel route information, it registers expense items and prepares the data to be sent to the terminal.

[0390] Step 6:

[0391] The server generates a schedule adjustment plan and travel expense reimbursement information, which is then sent to the terminal and notified to the user. The notification displays the information using an interface tailored to the user's emotional state. Through this emotionally responsive UI, the user can review and, if necessary, modify the information provided by the system.

[0392] (Application Example 2)

[0393] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0394] Modern travel expense reimbursement and scheduling systems lack features that consider user emotions, leading to situations where users are likely to experience stress. In particular, as there is a need to improve the user experience in managing the prioritization of scheduled visits and information exchange, these systems are unable to respond flexibly in accordance with user emotions, which poses a significant challenge.

[0395] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0396] In this invention, the server includes means for acquiring time information from a time information manager, means for automatically determining the departure and arrival points based on destination information, and emotion analysis means for analyzing the user's emotions and adjusting the response of the information processing device according to the user's real-time emotional state. This enables efficient management of the user's visit schedule and expense settlement in a manner that takes the user's emotions into consideration, thereby reducing their mental burden.

[0397] An "information processing device" is a device that has the functions of acquiring, analyzing, storing, and communicating data, and that provides a response according to the user's request.

[0398] A "time information manager" is a device or software that manages a user's schedule and plans, and makes necessary data available in conjunction with other systems.

[0399] "Destination information" refers to information about the place the user plans to visit, and specifically includes data such as the name, address, and estimated arrival time of the destination.

[0400] "Departure point and arrival point" refer to the place where a journey begins and the place where it ends, respectively, and serve as the time and spatial reference points in a travel plan.

[0401] "Emotion analysis methods" are technologies that analyze emotions from data such as the tone of a user's voice and facial expressions, and understand the user's emotional state in real time.

[0402] A "cost settlement system" is a mechanism that calculates and records travel expenses and manages and communicates that information to users and approvers.

[0403] To implement this invention, it is necessary to build a system in which a server, acting as an information processing device, is central and operates in conjunction with the user's terminal. The server obtains the user's schedule information from a time information manager and analyzes destination information based on that data. Based on the destination information, the server automatically identifies the departure and arrival points. In this process, the server uses map information services such as the Google Maps API to search for a specific travel route and calculate an estimated travel cost.

[0404] Furthermore, as a means of emotion analysis, the server collaborates with home robots or smart devices equipped with microphones and cameras to analyze the user's voice tone and facial expressions in real time. In this process, generative AI models such as OpenAI's GPT are used to generate appropriate responses that match the user's emotional state. For example, if the server determines that the user is feeling stressed, it instructs the robot to provide content to help the user relax, and the robot then makes suggestions to the user.

[0405] When settling expenses, the server notifies the user's terminal of travel costs, and the user can review the details and make corrections as needed on their terminal. As an example of a prompt, by inputting a sentence such as, "Perform sentiment analysis regarding the next scheduled visit and suggest rescheduling if the user is feeling stressed, include a function that analyzes emotions based on the user's tone of voice and facial expressions and suggests relaxing music," an appropriate response can be obtained.

[0406] For example, if a user schedules a work meeting and the system observes that they are busy just before the meeting, it will provide a gentle voice reminder and recommend relaxing activities after the meeting. This reduces the user's mental burden while enabling smooth schedule management and expense reimbursement.

[0407] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0408] Step 1:

[0409] The server retrieves schedule information from the user's terminal via a time information manager. The retrieved information includes the name of the destination, address, and scheduled date and time of the visit. This aggregates the data that forms the basis for the server's next processing.

[0410] Step 2:

[0411] The server automatically identifies the departure and arrival points using the Google Maps API and other tools based on the acquired destination information. The output of this process confirms the specific departure and arrival points, which are then used in the next process.

[0412] Step 3:

[0413] The server searches for a travel route between the specified origin and destination points and calculates the associated travel costs. Map information is used to search for the travel route, and travel costs are calculated based on a set pricing structure. This results in the travel costs being calculated as cost data.

[0414] Step 4:

[0415] The user's device acquires data on the user's voice tone and facial expressions through robots and smart devices. This data is sent to a server to analyze the user's real-time emotional state.

[0416] Step 5:

[0417] The server uses a generative AI model to analyze the user's emotions based on the transmitted data and generates a response based on the results. This response is appropriate to the user's emotional state. For example, if stress is detected, the server instructs the robot to provide content to help the user relax.

[0418] Step 6:

[0419] The server automatically registers the calculated travel expenses with the expense settlement system and notifies the user's terminal of the result. The user can then review this information on their terminal and make corrections as needed. This ensures the accuracy and convenience of the settlement data.

[0420] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0421] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0422] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0423] [Third Embodiment]

[0424] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0425] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0426] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0427] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0428] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0429] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0430] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0431] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0432] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0433] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0434] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0435] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0436] This invention is a system that uses an information processing device to acquire scheduled visit information from a schedule management device and automatically settle travel expenses. The system uses the schedule information entered by the user in advance to identify the departure and arrival points and calculate the optimal travel route and its cost. This significantly reduces the user's input work and minimizes errors.

[0437] The information processing device acquires data from the schedule management device via an API and uses an external geocoding service to obtain accurate location information based on the name and address of the visited company. Furthermore, it uses a route search service to collect the optimal travel route from the starting point to the destination and its transportation costs.

[0438] For example, suppose a user enters "XYZ Corporation, Minato-ku, Tokyo" as a planned visit in their calendar. The server retrieves this information and searches for the optimal route from the user's office in Shinjuku-ku, Tokyo. The calculated travel expenses, along with the resulting route data, are automatically registered in the expense reimbursement system.

[0439] Furthermore, the expense reimbursement system can automatically assign approvers based on the user's department and position. This ensures a quick and reliable approval process. Users receive notifications on their devices, can review the details of registered travel expenses, and make corrections as needed. Finally, once the user selects "Submit," the travel expense claim is finalized, and the approval flow begins.

[0440] This system frees users from complex settlement procedures and improves operational efficiency. Because the entire system is cloud-based, it offers flexible scalability and can be customized to meet the diverse needs of various companies.

[0441] The following describes the processing flow.

[0442] Step 1:

[0443] The server retrieves schedule information from the schedule management device. This schedule information includes the name, address, and date and time of the visit. This data is periodically retrieved via an API and prepared for use within the system.

[0444] Step 2:

[0445] Based on the schedule information acquired by the server, the address of the destination is used to call the Geocoding service to obtain location information. The address data is sent to the API to obtain the precise latitude and longitude, which are then recorded as the departure and arrival points.

[0446] Step 3:

[0447] The server uses location information of the departure point and destination to perform a route search on a route search service. To obtain the optimal travel route and the means of transportation to be used, it calculates the route and travel time using an external transportation API.

[0448] Step 4:

[0449] The server calculates travel expenses based on the route search results. It aggregates the fares for each mode of transport and generates the total amount as settlement data.

[0450] Step 5:

[0451] The server calculates travel expense data and registers it in the expense reimbursement system. The visit date, destination information, and calculated travel expenses are converted into the system's appropriate data format and sent.

[0452] Step 6:

[0453] The server automatically configures the approver information required for the approval flow based on the user's department information. The system identifies appropriate approvers based on the user's job title and authority, and integrates them into the workflow.

[0454] Step 7:

[0455] The device sends a notification to the user informing them that travel expenses have been automatically registered. A confirmation screen is displayed on the device, allowing the user to review the details and make corrections if necessary.

[0456] Step 8:

[0457] The user receives a notification and reviews the travel expense request. The user chooses either "revise" or "submit," and finally submits the request to initiate the approval process, thus completing the travel expense request.

[0458] (Example 1)

[0459] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0460] Existing scheduling management systems require users to manually calculate travel expenses and submit separate applications for each visit, which is time-consuming and raises concerns about errors. Furthermore, manual approval processes hinder efficiency and make quick decision-making difficult.

[0461] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0462] In this invention, the server includes means for acquiring schedule data from a schedule management device, means for acquiring coordinate data using a location information service, means for searching for a route between a departure point and a destination point and calculating the fare, and means for automatically registering it in the expense settlement system. This enables automatic calculation and registration of transportation expenses, reduces the workload on users, and realizes a quick and accurate approval process.

[0463] An "information processing device" is a device that acquires data from a schedule management device and performs tasks such as identifying location information and calculating routes.

[0464] A "schedule management device" is a platform for users to input their planned visits and destinations.

[0465] "Schedule data" refers to information such as destinations and dates entered into the schedule management device.

[0466] "Destination data" refers to information included in the schedule data that indicates the address or company name of the destination.

[0467] "Starting point" refers to location information indicating the starting point of the movement.

[0468] "Destination" refers to location information that indicates the destination of a journey.

[0469] A "location information service" is an external service used to obtain latitude and longitude coordinate data from information such as addresses.

[0470] "Coordinate data" refers to numerical data that indicates the latitude and longitude of a specific location.

[0471] A "route" is information that shows the path traveled between a starting point and a destination.

[0472] "Fare" refers to the amount of money spent on transportation.

[0473] An "expense reimbursement system" is a system that registers calculated fares and manages the approval and reimbursement process.

[0474] An "approval officer" is a person responsible for reviewing and approving submitted expense applications.

[0475] A "user terminal" is a device used by a user to receive notifications and to review and correct expense claims.

[0476] This invention is a system that automatically settles travel expenses based on schedule data acquired from a schedule management device. The main components of the system are an information processing device, a location information service, a route search service, and an expense settlement system. Each component is described below.

[0477] The server functions as an information processing device, retrieving schedule data entered by the user into the schedule management device via an API. This retrieved schedule data includes the name and address of the visited company. Based on this, the server uses external location services, such as the Google Maps API or OpenStreetMap API, to obtain precise coordinate data. This process converts the address of the visited location into latitude and longitude, clearly identifying the destination's location.

[0478] Next, the server uses a route search service to calculate the optimal route from the starting point (e.g., the user's office) to the destination (the place to be visited). Services such as the Google Maps Directions API are used. Based on this route data, the server calculates the fare. In this process, accurate fare calculations are performed by referring to public transport fare information and taxi fare tables.

[0479] The calculated fare is immediately registered in the expense reimbursement system. The system automatically assigns an approver based on the user's affiliation information. This allows the approval process to begin quickly, resulting in increased operational efficiency.

[0480] Users receive a notification via their device that their travel expense claim has been registered. At that time, they can check details such as fares and destinations and make corrections as needed. Once all information has been confirmed and they click "Submit," the application is officially completed and the approval process begins.

[0481] As a concrete example, suppose a user enters "ABC Company, Minato Ward" into their calendar. Based on this data, the server searches for the optimal travel route from the user's office in Shinjuku Ward and calculates the fare. This information is then used to automate expense reporting. This process can also be verified by using prompts such as "Tell me how long this route takes and how much the transportation costs."

[0482] This system significantly reduces the effort required for expense reimbursement and improves operational efficiency. Furthermore, because it is provided as a cloud-based system, it allows for flexible expansion and customization.

[0483] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0484] Step 1:

[0485] Users enter data about their planned visits into the schedule management device. This data includes the name and address of the place they are visiting. The schedules entered by the user serve as basic information for subsequent processing.

[0486] Step 2:

[0487] The server retrieves schedule data from the schedule management device via an API. The input data retrieved includes the address and company name of the destination. The server uses this input data to query an external location information service, converts the address into latitude and longitude coordinate data, and accurately identifies the destination information.

[0488] Step 3:

[0489] The server uses a route search service to calculate the optimal travel route, taking the user's starting point (office location) and the specified destination coordinates as input. This is done using services such as the Google Maps Directions API. This calculation considers time and distance, selecting the most efficient route from multiple options.

[0490] Step 4:

[0491] The server references fare information to calculate fares based on the calculated travel route. Route information is input, and based on this, it calculates public transport fares and, if necessary, taxi fares, and outputs fare data.

[0492] Step 5:

[0493] The server automatically registers the calculated fare data into the expense reimbursement system. The data entered includes fare information and destination information, and payment information is then registered and managed in the reimbursement system based on this data.

[0494] Step 6:

[0495] The terminal notifies the user about registered travel expenses and visit information. The user receives this information, reviews the details, and makes corrections as needed. This action is a final confirmation step before the user actually completes the application.

[0496] Step 7:

[0497] Once the user selects "Submit" after reviewing, the device initiates the approval process. The user's final action confirms the application without requiring any further input, allowing the approval flow to proceed efficiently.

[0498] (Application Example 1)

[0499] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0500] In traditional operations, managing travel schedules and processing travel expenses for client visits were cumbersome and time-consuming. In particular, the calculation and approval processes for travel expenses were prone to human error, highlighting the need for increased efficiency. Furthermore, identifying optimal travel routes and managing costs became a significant burden without adequate tools and information. There is a need for technology that can address these challenges and enable efficient and accurate travel and expense reimbursement.

[0501] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0502] In this invention, the server includes means for acquiring schedule information from a schedule management system, means for automatically identifying departure and arrival points based on visit location information, and means for searching for the optimal travel distance using a route search service and automatically calculating transportation costs using the planned route data. This automates schedule management and transportation expense settlement, enabling users to improve work efficiency and accurate processing.

[0503] An "information processing system" is a system that uses computers to acquire, process, and manage data.

[0504] A "schedule management system" is software or a platform that records and manages a user's schedule and appointments.

[0505] "Schedule information" refers to data about appointments registered by the user, including the date, time, and location.

[0506] "Place information" refers to information about places the user plans to visit, specifically including addresses and facility names.

[0507] The "starting point" refers to the location where the user begins their journey, and usually refers to the user's office or home.

[0508] A "destination point" refers to the final destination a user must reach to reach their destination.

[0509] A "route search service" is an online or offline service that calculates and provides the optimal route from a starting point to a destination.

[0510] "Distance traveled" refers to the distance between the starting point and the destination, and is usually calculated by walking or traveling by vehicle.

[0511] "Transportation expenses" refer to the costs incurred for travel, and specifically include train fares, bus fares, and taxi fares.

[0512] An "expense settlement system" is a management system designed to streamline the application and settlement of expenses within a company.

[0513] "User terminal" refers to a device used by the user for operation, such as a smartphone or personal computer.

[0514] "User" refers to an individual who operates this system and inputs or verifies information.

[0515] The system that realizes this invention consists of an information processing system, a schedule management system, a route search service, and an expense settlement system. This system, in particular, enables the efficient management of a user's schedule and the accurate calculation and settlement of transportation expenses related to travel to visited locations, using a mobile information terminal such as a smartphone.

[0516] The server connects to the scheduling management system to retrieve the user's schedule information. For example, the Google Calendar API or Outlook API can be used for this purpose. Based on the retrieved schedule information, the Google Maps Geocoding API can be used to identify the departure and arrival points based on the visited locations.

[0517] The server then uses a route search service to find the optimal route from the departure point to the destination point. During this process, the Google Maps Directions API is used to obtain route information, including the shortest distance and estimated travel time. Furthermore, the Google Maps Distance Matrix API is used to automatically calculate travel expenses.

[0518] The user's mobile device receives calculation results in real time and provides an interface that allows the user to review this information and make corrections as needed. Ultimately, the calculated travel expenses are automatically registered in the expense settlement system (e.g., SAP Concur), and the supervisor approval process is made visible.

[0519] As a concrete example, when a user visits a specific city for sales activities, they enter the address information of their destination into a terminal. The system automatically calculates the route from the user's starting point (for example, an office in Tokyo) to the destination and calculates the transportation expenses. Through this series of operations, the user can avoid errors when submitting transportation expense claims, resulting in efficient business operations.

[0520] An example of a prompt for the generated AI model would be: "Explain the procedure for calculating the optimal route and transportation costs to the next destination and automatically registering them in the expense settlement system."

[0521] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0522] Step 1:

[0523] The server retrieves schedule information from the schedule management system. Specifically, it retrieves the schedule entered by the user via API and collects data including the address of the destination. At this point, the input is the user's schedule, and the output is the address information of the destination.

[0524] Step 2:

[0525] The server uses a geocoding service to determine the latitude and longitude of the departure and destination points based on the destination information. It uses the Google Maps Geocoding API to obtain precise location information from the address. The input is the address of the destination, and the output provides the latitude and longitude information of the departure and destination points.

[0526] Step 3:

[0527] The server uses a route search service to find the optimal route. It obtains optimal route information using the Google Maps Directions API and calculates the travel distance and travel time between the departure point and the destination point. The input is the latitude and longitude information of the departure point and destination point, and the output is the optimal route information.

[0528] Step 4:

[0529] The server calculates transportation costs based on distance and mode of transport. It uses the Google Maps Distance Matrix API to calculate costs for various modes of transport (e.g., public transport, taxi). The input is optimal route information, and the output is the calculated transportation cost.

[0530] Step 5:

[0531] The terminal receives the calculation results from the server. The user checks the details of the transportation expenses on their smartphone or computer screen and makes corrections as needed. The input is the calculated transportation expenses, and the output is obtained with the necessary corrections made by the user.

[0532] Step 6:

[0533] The server registers the finalized travel expenses in the expense settlement system. The expenses are automatically sent to the system, and the application is displayed to the approver. The input is travel expenses confirmed by the user, and the output is expense application information awaiting approval.

[0534] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0535] This invention is a system that combines an optimized travel expense settlement system in an information processing device with an emotion engine that analyzes user emotions. When a user enters a visit schedule into a schedule management device, the system can collect and utilize user emotion data simultaneously with that information.

[0536] Specifically, the server retrieves the name, address, and date and time of the visit as scheduled information. Furthermore, an emotion engine analyzes the user's emotions and adjusts the system's response to match their real-time emotional state. For example, if the user is feeling stressed, the system adjusts to send notifications in a gentler manner to ensure that important information is not missed.

[0537] The emotion engine analyzes the user's emotions from their voice tone and facial expressions, and adjusts the visit schedule based on that analysis. It can optimize visit priorities and offer suggestions to reduce the user's stress level.

[0538] Furthermore, in the travel expense reimbursement process, the server uses visit information to identify the departure and arrival points, searches for the travel route and expenses, and automatically registers them in the reimbursement system. The registered information is notified to the user via a terminal, and the user can review and modify the application details through an emotionally responsive interface.

[0539] For example, when a user enters "a certain company, a certain location" as a planned visit, if the emotion engine detects a positive emotion from the user's facial expression, the system will notify the user of the registration in a cheerful tone. Conversely, if a negative emotion is detected, the notification will be presented more cautiously than usual, providing a confirmation process that takes the user's emotions into consideration.

[0540] In this way, the system can recognize the user's emotions and use that information to optimally adjust each step of the travel expense reimbursement process, thereby improving the user experience.

[0541] The following describes the processing flow.

[0542] Step 1:

[0543] The server retrieves visit schedules from the schedule management device. This includes the name and address of the company to be visited, as well as the date and time of the visit. It also periodically retrieves data via API and prepares it for use within the system.

[0544] Step 2:

[0545] The server activates the emotion engine and analyzes the user's current emotions. During this process, the user's camera and microphone are used to transmit facial expressions and voice tone to the emotion engine, thereby acquiring emotion data in real time.

[0546] Step 3:

[0547] The emotion engine analyzes the user's emotional state and sends the results back to the server. It determines whether the emotion is positive, neutral, or negative, and based on that, decides on the appropriate response for subsequent processes.

[0548] Step 4:

[0549] The server performs route searching based on information obtained from the origin and sentiment analysis. Using external transportation APIs, it selects a route that minimizes user stress and detects the travel time and cost.

[0550] Step 5:

[0551] The server automatically registers the calculated route and cost into the expense reimbursement system. During registration, it generates a detailed message tailored to the user's emotional state and sends it to the expense reimbursement system.

[0552] Step 6:

[0553] The device displays a notification to the user. If the user is relaxed, the notification is presented in a normal format; if the user is tense or stressed, the notification is presented in a softer tone and interface, providing a careful review process.

[0554] Step 7:

[0555] The user receives a notification on their device, reviews the travel expense request, and makes corrections if necessary. The UI guides the user according to their emotional state, and the user confirms the request by selecting "Submit," at which point the approval flow automatically begins.

[0556] (Example 2)

[0557] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0558] In modern information management, there is a demand for both efficient expense reimbursement and responses that are sensitive to the user's feelings. While conventional systems have the functionality to automate expense reimbursement, they lack consideration for the user's emotional state, limiting their ability to improve the user experience. As a result, the burden on users, especially in busy schedules or stressful environments, is not reduced.

[0559] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0560] In this invention, the server includes means for acquiring schedule information from a schedule management device, means for analyzing user emotion data related to the schedule information and adjusting the system's response according to the emotion, and means for generating schedule adjustment proposals based on the user's emotional state using a generative AI model. This enables a travel expense settlement process and schedule adjustment that takes the user's emotional state into consideration.

[0561] An "information processing device" is an electronic device used for collecting, analyzing, and processing data.

[0562] A "schedule management device" is a device or software that provides an interface for users to input and manage their visit schedules and appointments.

[0563] "Place information" refers to data about places that users record as planned visits, specifically including the name of the place, address, and date and time of visit.

[0564] "Departure point" refers to the point where the user begins their journey.

[0565] "Destination" refers to the point that the user is expected to reach through their travels.

[0566] "Travel route" refers to route information that shows the means of transportation and the route taken from the departure point to the destination point.

[0567] "Travel expenses" refers to the amount of money required for transportation and related expenses for travel between the departure point and the destination.

[0568] A "expense reimbursement system" is a system for recording travel expenses and automatically processing reimbursements.

[0569] "Emotional data" refers to the results of an analysis that indicates the user's emotional state, and is obtained through methods such as voice tone and facial expression analysis.

[0570] A "generative AI model" is an artificial intelligence model trained for data analysis and inference, and is used for analyzing sentiment data.

[0571] A "schedule adjustment proposal" is information that shows a new schedule suggestion that has been adjusted according to the user's schedule and emotional state.

[0572] This invention is a system that combines travel expense settlement and user sentiment analysis using an information processing device. The system includes a server, a terminal, and user behavior-based interactions.

[0573] The server retrieves schedule information entered by the user from the schedule management device and analyzes destination information based on this information. Destination information includes the destination name and detailed address information registered by the user as a scheduled visit. Furthermore, the server drives an emotion engine to analyze the user's voice tone and facial expressions in real time, collecting user emotion data.

[0574] Generative AI models are used to analyze emotional data. These models identify the user's emotional state and generate optimal schedule adjustments and system responses based on that state. To improve the accuracy of the emotional analysis, a large dataset is used as a pre-trained model.

[0575] As a concrete example, consider a scenario where a user enters "Company X, City Y, Date and Time Z" as destination information. If the emotion engine determines that the user's facial expression is positive, the server sets the system's notification tone to a brighter one and presents the schedule. Conversely, if the user shows signs of stress, the notification will be delivered in a cautious and gentle tone. This ensures that the user does not miss important information and can use the system comfortably.

[0576] An example of a prompt message is: "The user has entered a scheduled visit: Company A, Tokyo Office, December 25, 2023, 10:00 AM. Please consider the user's current emotional state." Based on this, the system provides the user with an appropriate response and adjustment suggestions.

[0577] In this way, the present invention aims to improve the user experience by realizing travel expense settlement and schedule adjustment that take user emotions into consideration.

[0578] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0579] Step 1:

[0580] The user enters their visit schedule into the schedule management device. The user enters the name of the destination, address, and date and time of the visit, and this information is collected by the schedule management device. The entered information is sent from the terminal to the server.

[0581] Step 2:

[0582] The server receives the schedule information sent from the terminal and stores it in the database. Based on this information, the server automatically identifies the departure and arrival points. To identify these points, the server performs a cross-referencing process with a map database and calculates the travel route and travel costs.

[0583] Step 3:

[0584] The server drives the emotion engine, analyzing the user's voice tone and facial expression data. Real-time audio and video data acquired from the user's device is input, and the generating AI model analyzes the user's emotional state. As a result of the analysis, the user's emotional data is generated.

[0585] Step 4:

[0586] The server uses an AI model to generate appropriate system responses and schedule adjustments based on the user's emotional data. For example, if the user is feeling stressed, the urgency of the schedule is re-evaluated, and adjustments to reduce the burden are output.

[0587] Step 5:

[0588] The server automatically registers the calculated travel expenses into the travel expense reimbursement system. Based on the departure point, arrival point, and travel route information, it registers expense items and prepares the data to be sent to the terminal.

[0589] Step 6:

[0590] The server generates a schedule adjustment plan and travel expense reimbursement information, which is then sent to the terminal and notified to the user. The notification displays the information using an interface tailored to the user's emotional state. Through this emotionally responsive UI, the user can review and, if necessary, modify the information provided by the system.

[0591] (Application Example 2)

[0592] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0593] Modern travel expense reimbursement and scheduling systems lack features that consider user emotions, leading to situations where users are likely to experience stress. In particular, as there is a need to improve the user experience in managing the prioritization of scheduled visits and information exchange, these systems are unable to respond flexibly in accordance with user emotions, which poses a significant challenge.

[0594] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0595] In this invention, the server includes means for acquiring time information from a time information manager, means for automatically determining the departure and arrival points based on destination information, and emotion analysis means for analyzing the user's emotions and adjusting the response of the information processing device according to the user's real-time emotional state. This enables efficient management of the user's visit schedule and expense settlement in a manner that takes the user's emotions into consideration, thereby reducing their mental burden.

[0596] An "information processing device" is a device that has the functions of acquiring, analyzing, storing, and communicating data, and that provides a response according to the user's request.

[0597] A "time information manager" is a device or software that manages a user's schedule and plans, and makes necessary data available in conjunction with other systems.

[0598] "Destination information" refers to information about the place the user plans to visit, and specifically includes data such as the name, address, and estimated arrival time of the destination.

[0599] "Departure point and arrival point" refer to the place where a journey begins and the place where it ends, respectively, and serve as the time and spatial reference points in a travel plan.

[0600] "Emotion analysis methods" are technologies that analyze emotions from data such as the tone of a user's voice and facial expressions, and understand the user's emotional state in real time.

[0601] A "cost settlement system" is a mechanism that calculates and records travel expenses and manages and communicates that information to users and approvers.

[0602] To implement this invention, it is necessary to build a system in which a server, acting as an information processing device, is central and operates in conjunction with the user's terminal. The server obtains the user's schedule information from a time information manager and analyzes destination information based on that data. Based on the destination information, the server automatically identifies the departure and arrival points. In this process, the server uses map information services such as the Google Maps API to search for a specific travel route and calculate an estimated travel cost.

[0603] Furthermore, as a means of emotion analysis, the server collaborates with home robots or smart devices equipped with microphones and cameras to analyze the user's voice tone and facial expressions in real time. In this process, generative AI models such as OpenAI's GPT are used to generate appropriate responses that match the user's emotional state. For example, if the server determines that the user is feeling stressed, it instructs the robot to provide content to help the user relax, and the robot then makes suggestions to the user.

[0604] When settling expenses, the server notifies the user's terminal of travel costs, and the user can review the details and make corrections as needed on their terminal. As an example of a prompt, by inputting a sentence such as, "Perform sentiment analysis regarding the next scheduled visit and suggest rescheduling if the user is feeling stressed, include a function that analyzes emotions based on the user's tone of voice and facial expressions and suggests relaxing music," an appropriate response can be obtained.

[0605] For example, if a user schedules a work meeting and the system observes that they are busy just before the meeting, it will provide a gentle voice reminder and recommend relaxing activities after the meeting. This reduces the user's mental burden while enabling smooth schedule management and expense reimbursement.

[0606] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0607] Step 1:

[0608] The server retrieves schedule information from the user's terminal via a time information manager. The retrieved information includes the name of the destination, address, and scheduled date and time of the visit. This aggregates the data that forms the basis for the server's next processing.

[0609] Step 2:

[0610] The server automatically identifies the departure and arrival points using the Google Maps API and other tools based on the acquired destination information. The output of this process confirms the specific departure and arrival points, which are then used in the next process.

[0611] Step 3:

[0612] The server searches for a travel route between the specified origin and destination points and calculates the associated travel costs. Map information is used to search for the travel route, and travel costs are calculated based on a set pricing structure. This results in the travel costs being calculated as cost data.

[0613] Step 4:

[0614] The user's device acquires data on the user's voice tone and facial expressions through robots and smart devices. This data is sent to a server to analyze the user's real-time emotional state.

[0615] Step 5:

[0616] The server uses a generative AI model to analyze the user's emotions based on the transmitted data and generates a response based on the results. This response is appropriate to the user's emotional state. For example, if stress is detected, the server instructs the robot to provide content to help the user relax.

[0617] Step 6:

[0618] The server automatically registers the calculated travel expenses with the expense settlement system and notifies the user's terminal of the result. The user can then review this information on their terminal and make corrections as needed. This ensures the accuracy and convenience of the settlement data.

[0619] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0620] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0621] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0622] [Fourth Embodiment]

[0623] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0624] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0625] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0626] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0627] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0628] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0629] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0630] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0631] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0632] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0633] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0634] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0635] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0636] This invention is a system that uses an information processing device to acquire scheduled visit information from a schedule management device and automatically settle travel expenses. The system uses the schedule information entered by the user in advance to identify the departure and arrival points and calculate the optimal travel route and its cost. This significantly reduces the user's input work and minimizes errors.

[0637] The information processing device acquires data from the schedule management device via an API and uses an external geocoding service to obtain accurate location information based on the name and address of the visited company. Furthermore, it uses a route search service to collect the optimal travel route from the starting point to the destination and its transportation costs.

[0638] For example, suppose a user enters "XYZ Corporation, Minato-ku, Tokyo" as a planned visit in their calendar. The server retrieves this information and searches for the optimal route from the user's office in Shinjuku-ku, Tokyo. The calculated travel expenses, along with the resulting route data, are automatically registered in the expense reimbursement system.

[0639] Furthermore, the expense reimbursement system can automatically assign approvers based on the user's department and position. This ensures a quick and reliable approval process. Users receive notifications on their devices, can review the details of registered travel expenses, and make corrections as needed. Finally, once the user selects "Submit," the travel expense claim is finalized, and the approval flow begins.

[0640] This system frees users from complex settlement procedures and improves operational efficiency. Because the entire system is cloud-based, it offers flexible scalability and can be customized to meet the diverse needs of various companies.

[0641] The following describes the processing flow.

[0642] Step 1:

[0643] The server retrieves schedule information from the schedule management device. This schedule information includes the name, address, and date and time of the visit. This data is periodically retrieved via an API and prepared for use within the system.

[0644] Step 2:

[0645] Based on the schedule information acquired by the server, the address of the destination is used to call the Geocoding service to obtain location information. The address data is sent to the API to obtain the precise latitude and longitude, which are then recorded as the departure and arrival points.

[0646] Step 3:

[0647] The server uses location information of the departure point and destination to perform a route search on a route search service. To obtain the optimal travel route and the means of transportation to be used, it calculates the route and travel time using an external transportation API.

[0648] Step 4:

[0649] The server calculates travel expenses based on the route search results. It aggregates the fares for each mode of transport and generates the total amount as settlement data.

[0650] Step 5:

[0651] The server calculates travel expense data and registers it in the expense reimbursement system. The visit date, destination information, and calculated travel expenses are converted into the system's appropriate data format and sent.

[0652] Step 6:

[0653] The server automatically configures the approver information required for the approval flow based on the user's department information. The system identifies appropriate approvers based on the user's job title and authority, and integrates them into the workflow.

[0654] Step 7:

[0655] The device sends a notification to the user informing them that travel expenses have been automatically registered. A confirmation screen is displayed on the device, allowing the user to review the details and make corrections if necessary.

[0656] Step 8:

[0657] The user receives a notification and reviews the travel expense request. The user chooses either "revise" or "submit," and finally submits the request to initiate the approval process, thus completing the travel expense request.

[0658] (Example 1)

[0659] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0660] Existing scheduling management systems require users to manually calculate travel expenses and submit separate applications for each visit, which is time-consuming and raises concerns about errors. Furthermore, manual approval processes hinder efficiency and make quick decision-making difficult.

[0661] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0662] In this invention, the server includes means for acquiring schedule data from a schedule management device, means for acquiring coordinate data using a location information service, means for searching for a route between a departure point and a destination point and calculating the fare, and means for automatically registering it in the expense settlement system. This enables automatic calculation and registration of transportation expenses, reduces the workload on users, and realizes a quick and accurate approval process.

[0663] An "information processing device" is a device that acquires data from a schedule management device and performs tasks such as identifying location information and calculating routes.

[0664] A "schedule management device" is a platform for users to input their planned visits and destinations.

[0665] "Schedule data" refers to information such as destinations and dates entered into the schedule management device.

[0666] "Destination data" refers to information included in the schedule data that indicates the address or company name of the destination.

[0667] "Starting point" refers to location information indicating the starting point of the movement.

[0668] "Destination" refers to location information that indicates the destination of a journey.

[0669] A "location information service" is an external service used to obtain latitude and longitude coordinate data from information such as addresses.

[0670] "Coordinate data" refers to numerical data that indicates the latitude and longitude of a specific location.

[0671] A "route" is information that shows the path traveled between a starting point and a destination.

[0672] "Fare" refers to the amount of money spent on transportation.

[0673] An "expense reimbursement system" is a system that registers calculated fares and manages the approval and reimbursement process.

[0674] An "approval officer" is a person responsible for reviewing and approving submitted expense applications.

[0675] A "user terminal" is a device used by a user to receive notifications and to review and correct expense claims.

[0676] This invention is a system that automatically settles travel expenses based on schedule data acquired from a schedule management device. The main components of the system are an information processing device, a location information service, a route search service, and an expense settlement system. Each component is described below.

[0677] The server functions as an information processing device, retrieving schedule data entered by the user into the schedule management device via an API. This retrieved schedule data includes the name and address of the visited company. Based on this, the server uses external location services, such as the Google Maps API or OpenStreetMap API, to obtain precise coordinate data. This process converts the address of the visited location into latitude and longitude, clearly identifying the destination's location.

[0678] Next, the server uses a route search service to calculate the optimal route from the starting point (e.g., the user's office) to the destination (the place to be visited). Services such as the Google Maps Directions API are used. Based on this route data, the server calculates the fare. In this process, accurate fare calculations are performed by referring to public transport fare information and taxi fare tables.

[0679] The calculated fare is immediately registered in the expense reimbursement system. The system automatically assigns an approver based on the user's affiliation information. This allows the approval process to begin quickly, resulting in increased operational efficiency.

[0680] Users receive a notification via their device that their travel expense claim has been registered. At that time, they can check details such as fares and destinations and make corrections as needed. Once all information has been confirmed and they click "Submit," the application is officially completed and the approval process begins.

[0681] As a concrete example, suppose a user enters "ABC Company, Minato Ward" into their calendar. Based on this data, the server searches for the optimal travel route from the user's office in Shinjuku Ward and calculates the fare. This information is then used to automate expense reporting. This process can also be verified by using prompts such as "Tell me how long this route takes and how much the transportation costs."

[0682] This system significantly reduces the effort required for expense reimbursement and improves operational efficiency. Furthermore, because it is provided as a cloud-based system, it allows for flexible expansion and customization.

[0683] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0684] Step 1:

[0685] Users enter data about their planned visits into the schedule management device. This data includes the name and address of the place they are visiting. The schedules entered by the user serve as basic information for subsequent processing.

[0686] Step 2:

[0687] The server retrieves schedule data from the schedule management device via an API. The input data retrieved includes the address and company name of the destination. The server uses this input data to query an external location information service, converts the address into latitude and longitude coordinate data, and accurately identifies the destination information.

[0688] Step 3:

[0689] The server uses a route search service to calculate the optimal travel route, taking the user's starting point (office location) and the specified destination coordinates as input. This is done using services such as the Google Maps Directions API. This calculation considers time and distance, selecting the most efficient route from multiple options.

[0690] Step 4:

[0691] The server references fare information to calculate fares based on the calculated travel route. Route information is input, and based on this, it calculates public transport fares and, if necessary, taxi fares, and outputs fare data.

[0692] Step 5:

[0693] The server automatically registers the calculated fare data into the expense reimbursement system. The data entered includes fare information and destination information, and payment information is then registered and managed in the reimbursement system based on this data.

[0694] Step 6:

[0695] The terminal notifies the user about registered travel expenses and visit information. The user receives this information, reviews the details, and makes corrections as needed. This action is a final confirmation step before the user actually completes the application.

[0696] Step 7:

[0697] Once the user selects "Submit" after reviewing, the device initiates the approval process. The user's final action confirms the application without requiring any further input, allowing the approval flow to proceed efficiently.

[0698] (Application Example 1)

[0699] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0700] In traditional operations, managing travel schedules and processing travel expenses for client visits were cumbersome and time-consuming. In particular, the calculation and approval processes for travel expenses were prone to human error, highlighting the need for increased efficiency. Furthermore, identifying optimal travel routes and managing costs became a significant burden without adequate tools and information. There is a need for technology that can address these challenges and enable efficient and accurate travel and expense reimbursement.

[0701] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0702] In this invention, the server includes means for acquiring schedule information from a schedule management system, means for automatically identifying departure and arrival points based on visit location information, and means for searching for the optimal travel distance using a route search service and automatically calculating transportation costs using the planned route data. This automates schedule management and transportation expense settlement, enabling users to improve work efficiency and accurate processing.

[0703] An "information processing system" is a system that uses computers to acquire, process, and manage data.

[0704] A "schedule management system" is software or a platform that records and manages a user's schedule and appointments.

[0705] "Schedule information" refers to data about appointments registered by the user, including the date, time, and location.

[0706] "Place information" refers to information about places the user plans to visit, specifically including addresses and facility names.

[0707] The "starting point" refers to the location where the user begins their journey, and usually refers to the user's office or home.

[0708] A "destination point" refers to the final destination a user must reach to reach their destination.

[0709] A "route search service" is an online or offline service that calculates and provides the optimal route from a starting point to a destination.

[0710] "Distance traveled" refers to the distance between the starting point and the destination, and is usually calculated by walking or traveling by vehicle.

[0711] "Transportation expenses" refer to the costs incurred for travel, and specifically include train fares, bus fares, and taxi fares.

[0712] An "expense settlement system" is a management system designed to streamline the application and settlement of expenses within a company.

[0713] "User terminal" refers to a device used by the user for operation, such as a smartphone or personal computer.

[0714] "User" refers to an individual who operates this system and inputs or verifies information.

[0715] The system that realizes this invention consists of an information processing system, a schedule management system, a route search service, and an expense settlement system. This system, in particular, enables the efficient management of a user's schedule and the accurate calculation and settlement of transportation expenses related to travel to visited locations, using a mobile information terminal such as a smartphone.

[0716] The server connects to the scheduling management system to retrieve the user's schedule information. For example, the Google Calendar API or Outlook API can be used for this purpose. Based on the retrieved schedule information, the Google Maps Geocoding API can be used to identify the departure and arrival points based on the visited locations.

[0717] The server then uses a route search service to find the optimal route from the departure point to the destination point. During this process, the Google Maps Directions API is used to obtain route information, including the shortest distance and estimated travel time. Furthermore, the Google Maps Distance Matrix API is used to automatically calculate travel expenses.

[0718] The user's mobile device receives calculation results in real time and provides an interface that allows the user to review this information and make corrections as needed. Ultimately, the calculated travel expenses are automatically registered in the expense settlement system (e.g., SAP Concur), and the supervisor approval process is made visible.

[0719] As a concrete example, when a user visits a specific city for sales activities, they enter the address information of their destination into a terminal. The system automatically calculates the route from the user's starting point (for example, an office in Tokyo) to the destination and calculates the transportation expenses. Through this series of operations, the user can avoid errors when submitting transportation expense claims, resulting in efficient business operations.

[0720] An example of a prompt for the generated AI model would be: "Explain the procedure for calculating the optimal route and transportation costs to the next destination and automatically registering them in the expense settlement system."

[0721] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0722] Step 1:

[0723] The server retrieves schedule information from the schedule management system. Specifically, it retrieves the schedule entered by the user via API and collects data including the address of the destination. At this point, the input is the user's schedule, and the output is the address information of the destination.

[0724] Step 2:

[0725] The server uses a geocoding service to determine the latitude and longitude of the departure and destination points based on the destination information. It uses the Google Maps Geocoding API to obtain precise location information from the address. The input is the address of the destination, and the output provides the latitude and longitude information of the departure and destination points.

[0726] Step 3:

[0727] The server uses a route search service to find the optimal route. It obtains optimal route information using the Google Maps Directions API and calculates the travel distance and travel time between the departure point and the destination point. The input is the latitude and longitude information of the departure point and destination point, and the output is the optimal route information.

[0728] Step 4:

[0729] The server calculates transportation costs based on distance and mode of transport. It uses the Google Maps Distance Matrix API to calculate costs for various modes of transport (e.g., public transport, taxi). The input is optimal route information, and the output is the calculated transportation cost.

[0730] Step 5:

[0731] The terminal receives the calculation results from the server. The user checks the details of the transportation expenses on their smartphone or computer screen and makes corrections as needed. The input is the calculated transportation expenses, and the output is obtained with the necessary corrections made by the user.

[0732] Step 6:

[0733] The server registers the finalized travel expenses in the expense settlement system. The expenses are automatically sent to the system, and the application is displayed to the approver. The input is travel expenses confirmed by the user, and the output is expense application information awaiting approval.

[0734] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0735] This invention is a system that combines an optimized travel expense settlement system in an information processing device with an emotion engine that analyzes user emotions. When a user enters a visit schedule into a schedule management device, the system can collect and utilize user emotion data simultaneously with that information.

[0736] Specifically, the server retrieves the name, address, and date and time of the visit as scheduled information. Furthermore, an emotion engine analyzes the user's emotions and adjusts the system's response to match their real-time emotional state. For example, if the user is feeling stressed, the system adjusts to send notifications in a gentler manner to ensure that important information is not missed.

[0737] The emotion engine analyzes the user's emotions from their voice tone and facial expressions, and adjusts the visit schedule based on that analysis. It can optimize visit priorities and offer suggestions to reduce the user's stress level.

[0738] Furthermore, in the travel expense reimbursement process, the server uses visit information to identify the departure and arrival points, searches for the travel route and expenses, and automatically registers them in the reimbursement system. The registered information is notified to the user via a terminal, and the user can review and modify the application details through an emotionally responsive interface.

[0739] For example, when a user enters "a certain company, a certain location" as a planned visit, if the emotion engine detects a positive emotion from the user's facial expression, the system will notify the user of the registration in a cheerful tone. Conversely, if a negative emotion is detected, the notification will be presented more cautiously than usual, providing a confirmation process that takes the user's emotions into consideration.

[0740] In this way, the system can recognize the user's emotions and use that information to optimally adjust each step of the travel expense reimbursement process, thereby improving the user experience.

[0741] The following describes the processing flow.

[0742] Step 1:

[0743] The server retrieves visit schedules from the schedule management device. This includes the name and address of the company to be visited, as well as the date and time of the visit. It also periodically retrieves data via API and prepares it for use within the system.

[0744] Step 2:

[0745] The server activates the emotion engine and analyzes the user's current emotions. During this process, the user's camera and microphone are used to transmit facial expressions and voice tone to the emotion engine, thereby acquiring emotion data in real time.

[0746] Step 3:

[0747] The emotion engine analyzes the user's emotional state and sends the results back to the server. It determines whether the emotion is positive, neutral, or negative, and based on that, decides on the appropriate response for subsequent processes.

[0748] Step 4:

[0749] The server performs route searching based on information obtained from the origin and sentiment analysis. Using external transportation APIs, it selects a route that minimizes user stress and detects the travel time and cost.

[0750] Step 5:

[0751] The server automatically registers the calculated route and cost into the expense reimbursement system. During registration, it generates a detailed message tailored to the user's emotional state and sends it to the expense reimbursement system.

[0752] Step 6:

[0753] The device displays a notification to the user. If the user is relaxed, the notification is presented in a normal format; if the user is tense or stressed, the notification is presented in a softer tone and interface, providing a careful review process.

[0754] Step 7:

[0755] The user receives a notification on their device, reviews the travel expense request, and makes corrections if necessary. The UI guides the user according to their emotional state, and the user confirms the request by selecting "Submit," at which point the approval flow automatically begins.

[0756] (Example 2)

[0757] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0758] In modern information management, there is a demand for both efficient expense reimbursement and responses that are sensitive to the user's feelings. While conventional systems have the functionality to automate expense reimbursement, they lack consideration for the user's emotional state, limiting their ability to improve the user experience. As a result, the burden on users, especially in busy schedules or stressful environments, is not reduced.

[0759] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0760] In this invention, the server includes means for acquiring schedule information from a schedule management device, means for analyzing user emotion data related to the schedule information and adjusting the system's response according to the emotion, and means for generating schedule adjustment proposals based on the user's emotional state using a generative AI model. This enables a travel expense settlement process and schedule adjustment that takes the user's emotional state into consideration.

[0761] An "information processing device" is an electronic device used for collecting, analyzing, and processing data.

[0762] A "schedule management device" is a device or software that provides an interface for users to input and manage their visit schedules and appointments.

[0763] "Place information" refers to data about places that users record as planned visits, specifically including the name of the place, address, and date and time of visit.

[0764] "Departure point" refers to the point where the user begins their journey.

[0765] "Destination" refers to the point that the user is expected to reach through their travels.

[0766] "Travel route" refers to route information that shows the means of transportation and the route taken from the departure point to the destination point.

[0767] "Travel expenses" refers to the amount of money required for transportation and related expenses for travel between the departure point and the destination.

[0768] A "expense reimbursement system" is a system for recording travel expenses and automatically processing reimbursements.

[0769] "Emotional data" refers to the results of an analysis that indicates the user's emotional state, and is obtained through methods such as voice tone and facial expression analysis.

[0770] A "generative AI model" is an artificial intelligence model trained for data analysis and inference, and is used for analyzing sentiment data.

[0771] A "schedule adjustment proposal" is information that shows a new schedule suggestion that has been adjusted according to the user's schedule and emotional state.

[0772] This invention is a system that combines travel expense settlement and user sentiment analysis using an information processing device. The system includes a server, a terminal, and user behavior-based interactions.

[0773] The server retrieves schedule information entered by the user from the schedule management device and analyzes destination information based on this information. Destination information includes the destination name and detailed address information registered by the user as a scheduled visit. Furthermore, the server drives an emotion engine to analyze the user's voice tone and facial expressions in real time, collecting user emotion data.

[0774] Generative AI models are used to analyze emotional data. These models identify the user's emotional state and generate optimal schedule adjustments and system responses based on that state. To improve the accuracy of the emotional analysis, a large dataset is used as a pre-trained model.

[0775] As a concrete example, consider a scenario where a user enters "Company X, City Y, Date and Time Z" as destination information. If the emotion engine determines that the user's facial expression is positive, the server sets the system's notification tone to a brighter one and presents the schedule. Conversely, if the user shows signs of stress, the notification will be delivered in a cautious and gentle tone. This ensures that the user does not miss important information and can use the system comfortably.

[0776] An example of a prompt message is: "The user has entered a scheduled visit: Company A, Tokyo Office, December 25, 2023, 10:00 AM. Please consider the user's current emotional state." Based on this, the system provides the user with an appropriate response and adjustment suggestions.

[0777] In this way, the present invention aims to improve the user experience by realizing travel expense settlement and schedule adjustment that take user emotions into consideration.

[0778] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0779] Step 1:

[0780] The user enters their visit schedule into the schedule management device. The user enters the name of the destination, address, and date and time of the visit, and this information is collected by the schedule management device. The entered information is sent from the terminal to the server.

[0781] Step 2:

[0782] The server receives the schedule information sent from the terminal and stores it in the database. Based on this information, the server automatically identifies the departure and arrival points. To identify these points, the server performs a cross-referencing process with a map database and calculates the travel route and travel costs.

[0783] Step 3:

[0784] The server drives the emotion engine, analyzing the user's voice tone and facial expression data. Real-time audio and video data acquired from the user's device is input, and the generating AI model analyzes the user's emotional state. As a result of the analysis, the user's emotional data is generated.

[0785] Step 4:

[0786] The server uses an AI model to generate appropriate system responses and schedule adjustments based on the user's emotional data. For example, if the user is feeling stressed, the urgency of the schedule is re-evaluated, and adjustments to reduce the burden are output.

[0787] Step 5:

[0788] The server automatically registers the calculated travel expenses into the travel expense reimbursement system. Based on the departure point, arrival point, and travel route information, it registers expense items and prepares the data to be sent to the terminal.

[0789] Step 6:

[0790] The server generates a schedule adjustment plan and travel expense reimbursement information, which is then sent to the terminal and notified to the user. The notification displays the information using an interface tailored to the user's emotional state. Through this emotionally responsive UI, the user can review and, if necessary, modify the information provided by the system.

[0791] (Application Example 2)

[0792] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0793] Modern travel expense reimbursement and scheduling systems lack features that consider user emotions, leading to situations where users are likely to experience stress. In particular, as there is a need to improve the user experience in managing the prioritization of scheduled visits and information exchange, these systems are unable to respond flexibly in accordance with user emotions, which poses a significant challenge.

[0794] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0795] In this invention, the server includes means for acquiring time information from a time information manager, means for automatically determining the departure and arrival points based on destination information, and emotion analysis means for analyzing the user's emotions and adjusting the response of the information processing device according to the user's real-time emotional state. This enables efficient management of the user's visit schedule and expense settlement in a manner that takes the user's emotions into consideration, thereby reducing their mental burden.

[0796] An "information processing device" is a device that has the functions of acquiring, analyzing, storing, and communicating data, and that provides a response according to the user's request.

[0797] A "time information manager" is a device or software that manages a user's schedule and plans, and makes necessary data available in conjunction with other systems.

[0798] "Destination information" refers to information about the place the user plans to visit, and specifically includes data such as the name, address, and estimated arrival time of the destination.

[0799] "Departure point and arrival point" refer to the place where a journey begins and the place where it ends, respectively, and serve as the time and spatial reference points in a travel plan.

[0800] "Emotion analysis methods" are technologies that analyze emotions from data such as the tone of a user's voice and facial expressions, and understand the user's emotional state in real time.

[0801] A "cost settlement system" is a mechanism that calculates and records travel expenses and manages and communicates that information to users and approvers.

[0802] To implement this invention, it is necessary to build a system in which a server, acting as an information processing device, is central and operates in conjunction with the user's terminal. The server obtains the user's schedule information from a time information manager and analyzes destination information based on that data. Based on the destination information, the server automatically identifies the departure and arrival points. In this process, the server uses map information services such as the Google Maps API to search for a specific travel route and calculate an estimated travel cost.

[0803] Furthermore, as a means of emotion analysis, the server collaborates with home robots or smart devices equipped with microphones and cameras to analyze the user's voice tone and facial expressions in real time. In this process, generative AI models such as OpenAI's GPT are used to generate appropriate responses that match the user's emotional state. For example, if the server determines that the user is feeling stressed, it instructs the robot to provide content to help the user relax, and the robot then makes suggestions to the user.

[0804] When settling expenses, the server notifies the user's terminal of travel costs, and the user can review the details and make corrections as needed on their terminal. As an example of a prompt, by inputting a sentence such as, "Perform sentiment analysis regarding the next scheduled visit and suggest rescheduling if the user is feeling stressed, include a function that analyzes emotions based on the user's tone of voice and facial expressions and suggests relaxing music," an appropriate response can be obtained.

[0805] For example, if a user schedules a work meeting and the system observes that they are busy just before the meeting, it will provide a gentle voice reminder and recommend relaxing activities after the meeting. This reduces the user's mental burden while enabling smooth schedule management and expense reimbursement.

[0806] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0807] Step 1:

[0808] The server retrieves schedule information from the user's terminal via a time information manager. The retrieved information includes the name of the destination, address, and scheduled date and time of the visit. This aggregates the data that forms the basis for the server's next processing.

[0809] Step 2:

[0810] The server automatically identifies the departure and arrival points using the Google Maps API and other tools based on the acquired destination information. The output of this process confirms the specific departure and arrival points, which are then used in the next process.

[0811] Step 3:

[0812] The server searches for a travel route between the specified origin and destination points and calculates the associated travel costs. Map information is used to search for the travel route, and travel costs are calculated based on a set pricing structure. This results in the travel costs being calculated as cost data.

[0813] Step 4:

[0814] The user's device acquires data on the user's voice tone and facial expressions through robots and smart devices. This data is sent to a server to analyze the user's real-time emotional state.

[0815] Step 5:

[0816] The server uses a generative AI model to analyze the user's emotions based on the transmitted data and generates a response based on the results. This response is appropriate to the user's emotional state. For example, if stress is detected, the server instructs the robot to provide content to help the user relax.

[0817] Step 6:

[0818] The server automatically registers the calculated travel expenses with the expense settlement system and notifies the user's terminal of the result. The user can then review this information on their terminal and make corrections as needed. This ensures the accuracy and convenience of the settlement data.

[0819] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0820] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0821] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0822] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0823] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0824] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0825] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0826] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0827] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0828] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0829] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0830] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0831] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0832] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0833] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0834] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0835] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0836] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0837] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0838] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0839] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0840] The following is further disclosed regarding the embodiments described above.

[0841] (Claim 1)

[0842] The information processing device includes means for acquiring schedule information from the schedule management device,

[0843] A means for automatically identifying the departure point and arrival point based on the destination information included in the aforementioned schedule information,

[0844] A means for searching the travel route between the departure point and the arrival point and calculating the travel cost,

[0845] A means for automatically registering the calculated travel expenses in the expense settlement system,

[0846] A system that includes this.

[0847] (Claim 2)

[0848] The system according to claim 1, wherein the expense settlement system includes means for automatically setting approvers based on the visit destination information.

[0849] (Claim 3)

[0850] The system according to claim 1, wherein the information processing device includes means for providing a function that notifies a user terminal of the contents to be registered in the expense settlement system and allows the user to confirm and correct them.

[0851] "Example 1"

[0852] (Claim 1)

[0853] The information processing device includes means for acquiring schedule data from the schedule management device,

[0854] A means for automatically identifying the departure point and destination point based on the destination data included in the aforementioned planned data,

[0855] A means of obtaining accurate coordinate data using an external location information service,

[0856] A means for searching for a route between a departure point and a destination point and calculating the fare,

[0857] A means for automatically registering the calculated fare in the expense settlement system,

[0858] A system that includes this.

[0859] (Claim 2)

[0860] The system according to claim 1, wherein the expense reimbursement system includes means for automatically setting an approver based on the destination data.

[0861] (Claim 3)

[0862] The system according to claim 1, further comprising means for an information processing device to notify a user terminal of the contents registered in the expense settlement system, and for providing a function for the user to confirm and correct the contents.

[0863] "Application Example 1"

[0864] (Claim 1)

[0865] The information processing system provides a means for obtaining schedule information from the schedule management system,

[0866] A means for automatically identifying the departure point and destination point based on the information of places to visit included in the aforementioned itinerary information,

[0867] A means for searching for a route between the departure point and the destination point and calculating the travel cost,

[0868] A means for automatically registering the calculated travel expenses in the expense settlement system,

[0869] A means of automatically calculating transportation costs using planned route data, by searching for the optimal travel distance between the departure point and destination point using a route search service,

[0870] A system that includes this.

[0871] (Claim 2)

[0872] The system according to claim 1, wherein the expenditure settlement system includes means for automatically setting an approval officer based on the visit location information.

[0873] (Claim 3)

[0874] The system according to claim 1, further comprising means for providing a function that notifies a user terminal of the information processing system of the contents to be registered in the expenditure settlement system, and allows the user to confirm and correct the contents.

[0875] "Example 2 of combining an emotion engine"

[0876] (Claim 1)

[0877] The information processing device includes means for acquiring schedule information from the schedule management device,

[0878] A means for automatically identifying the departure point and arrival point based on the destination information included in the aforementioned schedule information,

[0879] A means for searching the travel route between the departure point and the arrival point and calculating the travel cost,

[0880] A means for automatically registering the calculated travel expenses in the expense settlement system,

[0881] A means for analyzing user sentiment data related to schedule information and adjusting the system's response according to the sentiment,

[0882] A means of generating schedule adjustment proposals based on the user's emotional state using a generative AI model,

[0883] A system that includes this.

[0884] (Claim 2)

[0885] The system according to claim 1, wherein the expense settlement system includes means for automatically setting approvers based on the visit destination information.

[0886] (Claim 3)

[0887] The system according to claim 1, wherein the information processing device includes means for notifying a user terminal of the contents registered in the expense settlement system, and for providing a function for the user to confirm and correct the contents via an interface corresponding to their emotional state.

[0888] "Application example 2 when combining with an emotional engine"

[0889] (Claim 1)

[0890] An information processing device provides a means for acquiring time information from a time information manager,

[0891] A means for automatically determining the departure point and arrival point based on the destination information included in the aforementioned time information,

[0892] A means for searching for a travel route between the aforementioned starting point and destination point and calculating the travel cost,

[0893] A means for automatically registering the calculated travel expenses with the expense settlement mechanism,

[0894] An emotion analysis means that analyzes the user's emotions and adjusts the response of the information processing device according to the real-time emotional state,

[0895] A means of optimizing the priority of scheduled visits based on the aforementioned sentiment analysis and proposing ways to reduce the mental burden on users,

[0896] A system that includes this.

[0897] (Claim 2)

[0898] The system according to claim 1, wherein the expense settlement system includes means for automatically setting an approver based on the destination information.

[0899] (Claim 3)

[0900] The system according to claim 1, further comprising means for the information processing device to notify a user terminal of the contents registered with the expense settlement mechanism, and for the user to provide a function to confirm and correct the contents. [Explanation of Symbols]

[0901] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. The information processing system provides a means for obtaining schedule information from the schedule management system, A means for automatically identifying the departure point and destination point based on the information of places to visit included in the aforementioned itinerary information, A means for searching for a route between the departure point and the destination point and calculating the travel cost, A means for automatically registering the calculated travel expenses in the expense settlement system, A means of automatically calculating transportation costs using planned route data, by searching for the optimal travel distance between the departure point and destination point using a route search service, A system that includes this.

2. The system according to claim 1, wherein the expenditure settlement system includes means for automatically setting an approval officer based on the visit location information.

3. The system according to claim 1, further comprising means for providing a function that notifies a user terminal of the contents registered in the expenditure settlement system of the information processing system, and allows the user to confirm and correct them.

Images