system

Abstract

We provide the system. [Solution] A means of obtaining and analyzing schedule information, A means for automatically selecting the optimal mode of transport and route based on current location information and destination information, A means of updating the schedule based on the mode of transport and route, and setting an alarm appropriate for the departure time, A means of acquiring and updating traffic information in real time, Means of providing visual and audio route guidance while traveling to a destination, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, users need to manage a busy schedule. Especially regarding travel plans, there are problems such as considering means of transportation and adjusting appropriate departure times, which involve a lot of trouble. Also, it is difficult to quickly respond to traffic delays and schedule changes, and as a result, there is a risk of not being able to act as planned. Against this background, a system for more efficiently and accurately managing users' schedules is required.

Means for Solving the Problems

[0005] The present invention relates to a system equipped with means for acquiring and analyzing schedule information. The system includes means for automatically selecting the optimal mode of transport and route based on current location information and destination information. Furthermore, it includes means for updating the schedule based on the mode of transport and route, and setting alarms appropriate for the departure time. By acquiring and updating traffic information in real time, users can flexibly respond to their schedules. In addition, during travel, route guidance is provided visually and audibly to help users travel with peace of mind. This makes it possible to significantly streamline the user's daily schedule management and increase the probability of being able to act according to plan.

[0006] "Schedule information" refers to information such as the date, time, location, and activity details that users use when managing their schedules.

[0007] "Current location information" refers to information that indicates the geographical location where the user is at that moment.

[0008] "Destination information" refers to geographical information about the places the user plans to visit.

[0009] "Means of transportation" refers to the means or methods of transport (e.g., car, train, walking, etc.) that a user uses to reach their destination.

[0010] A "route" refers to the path taken for travel, connecting the current location to the destination.

[0011] An "alarm" refers to a notification or warning function that is set to inform the user of a specific time or event.

[0012] "Real-time" refers to the moment information or data is generated or acquired, meaning it is processed immediately without delay.

[0013] "Traffic information" refers to data on the condition of public transportation and roads, including information on service status, delays, and congestion.

[0014] "Visual guidance" refers to a method of providing information that visually shows a route or instructions to a user.

[0015] "Voice" guidance refers to a method of providing information that conveys a route or instructions to a user by voice.

Brief Explanation of Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [[ID=3X]] [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention is a system that comprehensively supports users' schedule management and is implemented by an AI-based program equipped with various functions. The specific operation of the program involved is described below.

[0038] First, the server runs in a cloud environment and, with the user's permission, retrieves the user's schedule information using the API of the scheduling information service. This allows the system to analyze important information such as the date, time, location, participants, and purpose of the appointment using natural language processing technology, and extract the necessary data.

[0039] Next, the server obtains the user's current location information through location services and searches for the optimal mode of transport and route by comparing it with the planned destination. In this process, it uses public transport operation data and map services to take into account traffic congestion and delays, and creates a real-time travel plan.

[0040] The server then automatically updates the user's schedule based on the planned mode of transport and travel time. It determines a specific departure time and sets an alarm in advance to ensure the user departs on time. The alarm settings provide a more personalized experience by taking into account the user's past travel history and preparation time.

[0041] While on the move, the device provides users with real-time route guidance through visual and audio means. This is done through an application on the mobile device, helping users reach their destination without getting lost. For example, when using public transport, it clearly instructs users on which train or bus to take and where to transfer.

[0042] By using this system, users can automate cumbersome scheduling and travel planning, enabling them to manage their schedules more efficiently. In particular, the ability to obtain real-time traffic information allows for flexible responses to unforeseen circumstances. This reduces daily stress, allowing users to focus on important appointments.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The server, with the user's permission, retrieves appointments from the user's calendar via the schedule information service's API. It then analyzes the date, time, location, and event details using natural language processing techniques to identify the schedule information.

[0046] Step 2:

[0047] The server uses location services to determine the user's current location. Next, based on the destination and acquired schedule information, it uses a transportation API to search for the optimal mode of transport and route. In this process, real-time traffic information is taken into consideration when selecting the route.

[0048] Step 3:

[0049] The calendar schedule is updated based on the mode of transport and route selected by the server. The departure time is calculated taking into account the travel time, and an alarm is set that takes into account the time needed for departure preparation, based on the user's past activity history.

[0050] Step 4:

[0051] When the departure time approaches, the device will notify the user with a set alarm. The alarm will alert the user with both sound and vibration to encourage them to prepare for departure on time.

[0052] Step 5:

[0053] When the device detects that it has started moving, it launches a map app or a dedicated app and provides real-time route guidance. The guidance is provided visually and audibly, offering the user the optimal route and transfer information to their destination.

[0054] Step 6:

[0055] If traffic delays or route changes occur during travel, the server updates information in real time and notifies the user of new instructions via their device. The user can then follow these instructions and reach their destination smoothly.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] In modern society, individuals lead busy daily lives, requiring efficient schedule management. However, manually setting appointments, selecting transportation methods, and dealing with traffic delays are time-consuming and cumbersome. Furthermore, changing schedules and obtaining real-time traffic information are also difficult. Therefore, automated schedule management systems are needed.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for acquiring schedule information via a communication device and analyzing the schedule using a generating AI model; means for automatically selecting the optimal means of transportation and route based on current location information and destination information; and means for updating the schedule based on the means of transportation and route and setting an alarm at an appropriate time for departure. This enables efficient schedule management and the automation of rapid travel planning.

[0061] A "communication device" is a device used to send and receive data, and is a device that has the function of acquiring or transmitting various information via a network.

[0062] "Schedule information" refers to detailed data about an individual's schedule, and is a general term for information including date, time, place, purpose, etc.

[0063] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and derive solutions based on user needs.

[0064] "Location information" refers to information that indicates the current geographical location of a device or individual, and is often expressed as coordinate data.

[0065] "Target destination information" refers to information about the place the user intends to travel to, and includes detailed data indicating the geographical location.

[0066] "Means of transportation" refers to the means or method of transport used to get to a destination, such as walking, public transport, or automobile.

[0067] A "route" is the path taken from the starting point to the destination, and is a route selected based on the shortest or most optimal method of travel.

[0068] An "alarm" is a function that provides a notification to the user at a specific time, and is a warning system set before an event begins.

[0069] "Traffic conditions" refers to information indicating the current state of road and transportation operations, including situations such as congestion and delays.

[0070] This invention provides a system that efficiently automates user schedule management. First, a server operates in a cloud environment and acquires schedule information via a communication device. This process utilizes common communication standards for schedule information services to collect schedule information. For example, it is designed so that users do not need to input detailed information such as the date, time, and location of meetings they are scheduled to attend.

[0071] Next, the server uses a generative AI model to analyze the acquired schedule information. At this stage, natural language processing techniques are used to interpret the purpose and priority of the schedule, supporting efficient schedule management. For example, a prompt sentence such as "Attend a meeting in the city tomorrow at 10am" is entered, and the server performs the necessary processing based on this.

[0072] The server also uses location services to collect current location information and compares it with destination information to select the optimal mode of transport and route. In this process, a wide range of options can be considered as means of transport, from public transport to automobiles.

[0073] Furthermore, the system updates the schedule based on the mode of transport and route, and sets alarms at appropriate times to ensure users can start their journey on time. The alarms are personalized based on the user's behavioral characteristics, allowing for efficient preparation for departure.

[0074] While on the move, the device provides users with real-time visual and audio route guidance. This is done through a smartphone application and helps users reach their destination without getting lost. This process of assisting with travel to the destination can clearly convey instructions for using public transportation and making transfers.

[0075] By using this system, users can gain automated schedule management and more efficient travel, reducing everyday stress and allowing them to focus on important matters.

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] The server retrieves user schedule information via the schedule information service API. It uses the user's account information as input and outputs data such as the date, time, location, and purpose of the appointment. Since this data is provided in several formats, it is converted to a standardized format for easier handling in subsequent processing.

[0079] Step 2:

[0080] The server analyzes the acquired schedule information using a generating AI model. The input is the schedule information standardized in step 1, and the output is an evaluation of the schedule priority and the importance of related tasks. Since natural language processing is performed in this process, it generates prompts that clarify the user's objective and performs specific actions to process them with the AI ​​model.

[0081] Step 3:

[0082] The server obtains the user's current location information from a location services. In this step, the server takes location data provided by the user's mobile device as input and outputs the user's real-time location coordinates. Based on this information, a specific step is taken to search for available public transportation options in real time.

[0083] Step 4:

[0084] The server selects the optimal mode of transport and route based on location and planned destination information. Input is current location and planned destination data, and output is the optimized mode of transport (e.g., train, car) and route. Real-time traffic information, such as congestion and service delays, is also considered in this process.

[0085] Step 5:

[0086] The server automatically updates the schedule and sets an alarm suitable for departure based on the selected mode of transport and route. Inputs include the optimal route information obtained in step 4 and the user's existing schedule. Outputs are the updated schedule information and alarm times, with the alarm time adjusted based on the user's past activity history.

[0087] Step 6:

[0088] The terminal provides users with real-time route guidance through both visual and audio. It uses optimal route information provided by a server as input, and outputs navigation instructions to the user. Specifically, it displays route information visually to the user via a map application and provides voice guidance for the next action.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] In modern society, managing daily schedules and the associated travel plans is complex, and there is a need for efficient systems to handle this. In particular, inappropriate traffic information and transportation choices can lead to wasted time and stress. The challenge is to improve the overall quality of life by enabling systems to manage users' schedules and travel more efficiently.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes means for acquiring and analyzing schedule information, means for automatically selecting the optimal means of transportation and route based on current location information and destination information, and means for updating the schedule based on the means of transportation and route and setting time notifications appropriate for the departure time. This enables users to manage their travel and schedules more efficiently and enjoy a smarter urban lifestyle.

[0094] "Schedule information" refers to information that shows the user's daily schedule and activity plans.

[0095] "Means of analysis" refers to techniques or devices for interpreting schedule information and extracting necessary data.

[0096] "Current location information" refers to geographical data used to identify the user's current location.

[0097] "Destination information" refers to geographical data about the place the user is heading to.

[0098] The "optimal mode of transport" means choosing the most efficient way to travel based on current traffic conditions and schedules.

[0099] A "route" is the path taken to travel from a starting point to a destination.

[0100] "Means for setting time notifications" refers to technologies or devices that notify users at the appropriate time so that they can act according to their schedule.

[0101] "Location data" refers to information about a specific geographical location.

[0102] "Means of providing information visually and aurally" refers to technologies or devices for presenting information to users in a visual form and conveying it audibly.

[0103] This invention is a system designed to streamline schedule management and improve users' daily lives. This system operates primarily on a server running in a cloud environment and utilizes multiple hardware and software components.

[0104] The server utilizes the planning information service's API to acquire and analyze schedule information. Through this API, it collects user schedule information and performs analysis using natural language processing. The software used incorporates an AI model to provide natural language processing capabilities.

[0105] The server also receives location information from the user's terminal and automatically selects the optimal mode of transport and route. This selection utilizes external APIs that provide location services and map data. In particular, it uses public data streams to update traffic information in real time.

[0106] To support users' travel, the server predicts travel time and sends notifications to the device that are appropriate for the departure time. This process incorporates machine learning algorithms that utilize past travel history and usage patterns.

[0107] While on the move, the device provides navigation information through sight and sound. This assumes the user is using a wearable device such as smart glasses. By utilizing the display and speaker of the wearable device, the user can receive guidance through sight and sound.

[0108] By actually using this system, users can enjoy comfortable and efficient urban living through optimized transportation options. For example, when attending a major event, they can be presented with the optimal mode of transportation and route before departure, and receive notifications of departure times.

[0109] An example of a prompt to a generating AI model is: "Based on the user's current location, please suggest the best mode of transportation and route for their planned station visit at 12:00. Please also consider traffic information."

[0110] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0111] Step 1:

[0112] The server retrieves user schedule information using the API of the planning information service. It receives user authentication information as input and retrieves scheduled information as output. The retrieved data is analyzed using natural language processing techniques to extract important keywords and date / time information.

[0113] Step 2:

[0114] The server receives current location information transmitted from the terminal. Based on this information, and combined with destination information, it uses external location services and map data to calculate the optimal mode of transport and route. The input is current location data from the terminal, and the output is information on the mode of transport and route. Traffic conditions and public transport operation data are also taken into consideration.

[0115] Step 3:

[0116] The server updates the user's schedule based on the acquired optimal mode of transport and route information. The inputs here are the travel information obtained in step 2 and the user's schedule effect. The output is a departure time notification adjusted to ensure the user travels on time. The notification timing is optimized using known patterns based on past travel history.

[0117] Step 4:

[0118] The terminal provides the user with real-time visual and auditory guidance. Input consists of route information and current location information sent from the server. Output includes specific actions such as displaying route guidance on the user's smart glasses or other wearable devices. During this process, the guidance information is updated as needed to ensure smooth travel to the destination.

[0119] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0120] This invention is a system that incorporates an emotion recognition engine to further personalize user schedule management and route guidance. In addition to its basic function of automatically analyzing the user's schedule and suggesting the optimal means of transportation and route, this system is capable of dynamic adjustments according to the user's emotional state.

[0121] First, the server uses conventional scheduling information acquisition methods to aggregate users' schedules on the cloud. This allows scheduling information related to date, time, and location to be properly analyzed.

[0122] Next, the emotion engine analyzes the voice and facial expression data acquired by the device from the user. Here, the user's emotional state (e.g., stress, joy, anger) is determined, and further adjustments to the interaction are made based on the results. For example, if it is determined that the user is feeling stressed, the server will change the suggested mode of transportation to a more comfortable option or adjust the voice guidance for the travel route to be gentler.

[0123] Furthermore, the server can update traffic information in real time and dynamically adjust schedules and alarms according to the user's mood. If the user is relaxed, a schedule with ample time will be displayed, while if they are in a hurry, the system will prioritize faster travel plans.

[0124] For example, if a user is feeling stressed during their morning commute, the device can use its emotional engine to reduce that stress by playing relaxing music or suggesting a more comfortable route.

[0125] As a result, the present invention can provide a more flexible and personalized experience that takes into account the individual emotional needs of each user, compared to conventional schedule management systems. By implementing this system, users can significantly reduce stress related to scheduling and travel in their daily lives.

[0126] The following describes the processing flow.

[0127] Step 1:

[0128] Users input speech and facial expression data into the system via smartphones or wearable devices. By granting permission to use the emotion engine during initial setup, the device is prepared to acquire emotion data.

[0129] Step 2:

[0130] The device runs an emotion recognition algorithm that analyzes the user's voice and facial expressions to determine the user's emotional state. This algorithm analyzes the user's speech patterns and facial features in real time to detect emotions such as stress, relief, anger, and joy.

[0131] Step 3:

[0132] The server uses the schedule information service's API to retrieve the user's schedule information from the cloud. The retrieved data is analyzed, and the date, time, location, purpose, and other details of the appointment are extracted.

[0133] Step 4:

[0134] The server acquires location information and selects the optimal mode of transport and route based on destination information. During this process, it incorporates the results of an emotion engine, adjusting the mode of transport and route according to the user's emotional state. For example, if the user is feeling stressed, less congested routes and places to avoid will be prioritized.

[0135] Step 5:

[0136] The server collects traffic information in real time and creates a travel schedule that takes emotions into account. Next, it calculates departure times and automatically sets alarms. During this process, the alarm sound can also be adjusted according to emotions.

[0137] Step 6:

[0138] The device notifies the user at the time the alarm it has set will sound. At this point, it prompts the user to begin preparations and guides them through the process to ensure a comfortable journey.

[0139] Step 7:

[0140] While on the move, the device provides the user with visual and audio route guidance. An emotion engine continuously monitors the user's emotions and adjusts the tempo and tone of the guidance accordingly. For example, if the user is relaxed, the guidance can be made more cheerful.

[0141] Step 8:

[0142] Servers and terminals regularly update information while in transit, allowing for immediate responses to unexpected traffic disruptions or schedule changes. Route and schedule readjustments based on emotional changes are also made during this step.

[0143] (Example 2)

[0144] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0145] Conventional schedule management and route guidance systems suggest modes of transport and routes based on predetermined criteria without considering the individual emotional state of the user. As a result, they fail to provide appropriate suggestions when the user is stressed or in a hurry, leading to decreased user satisfaction. The present invention aims to dynamically adjust modes of transport and routes according to the user's emotional state, providing a comfortable and personalized experience for the user.

[0146] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0147] In this invention, the server includes means for acquiring and analyzing schedule information, means for automatically selecting the optimal means of transportation and route based on current location information and destination information, and means for analyzing the user's emotional state using emotion recognition technology and dynamically adjusting the means of transportation and route based on the analysis results. This makes it possible to suggest the optimal transportation according to the user's emotional state.

[0148] "Schedule information" refers to detailed data such as the date, time, location, and content of appointments and events set by the user.

[0149] "Means of analysis" refers to devices or software that analyze acquired data and perform computational processing or judgments to give it meaning.

[0150] "Current location information" refers to digital data that indicates the user's real-time location, and includes latitude, longitude, and altitude.

[0151] "Destination information" refers to location data of the place the user is heading to, including addresses and place names.

[0152] "Optimal mode of transportation" refers to the method of travel that is deemed most efficient and convenient based on the user's conditions and environment.

[0153] "Means for automatically selecting a route" refers to a function that automatically determines the route to reach the user's destination using algorithms or software.

[0154] "Emotion recognition technology" refers to technology that analyzes a person's voice, facial expressions, and behavior to identify their emotional state at that time.

[0155] "Traffic information" refers to data that affects travel, such as the operating status of public transportation and road congestion.

[0156] "Means of providing route guidance visually and audibly" refers to technologies that provide users with a route to their destination through visual map displays and voice instructions.

[0157] This invention is a system that incorporates emotion recognition technology to highly personalize user schedule management and navigation guidance. Specific embodiments are described below.

[0158] The server uses external connections to manage and analyze schedule information in the cloud. Specifically, it retrieves information including date, time, location, and event details via APIs of publicly available schedule information services. This allows for a proper understanding and management of users' planned activities.

[0159] The device uses the smartphone's built-in microphone and camera to acquire the user's voice and facial expression data. This device is equipped with an emotion recognition engine and utilizes Microsoft® Azure® Cognitive Services and similar services to analyze the user's emotional state. For example, the device can determine whether the user is stressed or relaxed.

[0160] The server dynamically adjusts the optimal mode of transport and route based on analyzed emotion data and schedule information. Specifically, it uses a map service API to analyze traffic flow in order to reflect real-time traffic information. It also flexibly selects the mode of transport to present according to the user's emotional state. When the user is relaxed, it recommends a leisurely mode of transport, while when they are in a hurry, it suggests the fastest route.

[0161] Users can view and select suggested schedules and modes of transportation through their device, and the system provides services based on those selections. For example, if stress levels are measured during the morning commute, the device will play relaxation music, and the server will suggest a comfortable and efficient commute route. Another example of a prompt using a generative AI model is, "Please suggest a suitable restaurant for lunch when the user is in a relaxed state." This prompt allows the AI ​​to generate suggestions appropriate to the situation.

[0162] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0163] Step 1:

[0164] The server obtains schedule information via an external connection. It uses user registration information as input. The retrieved data includes date, time, location, and event type. This data is analyzed to create the user's schedule. The output is the analyzed schedule information.

[0165] Step 2:

[0166] The device collects the user's voice and facial expression data using a microphone and camera. The voice and facial expressions are recorded in real time and input into an emotion recognition engine. The engine analyzes this data to identify the user's emotional state (e.g., stress, relaxation). The output is data on the identified emotional state.

[0167] Step 3:

[0168] The server receives emotional state data and schedule information, and dynamically adjusts the mode of transport and route according to the user's state. Real-time traffic information is also included as input. As data processing, the optimal route is calculated using a map service API. The output is the adjusted mode of transport and route plan.

[0169] Step 4:

[0170] The terminal notifies the user of the travel plan sent from the server. It provides visual map displays and voice guidance, and plays relaxation music as needed. It uses user emotions and travel plan data as input. The output is user interaction and guidance.

[0171] Step 5:

[0172] The user selects a suggested mode of transport and route based on information provided from their device and sends feedback to the server. This feedback is used for future suggestions. The provided travel plan is used as input. The output is the user's selection and feedback data.

[0173] (Application Example 2)

[0174] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0175] Traditional scheduling management systems often fail to adequately address individual user needs because they suggest schedules and routes without considering the user's emotional state. In particular, certain emotional states during travel can cause stress and discomfort, potentially impacting the execution of the schedule. In such situations, a more personalized system is needed.

[0176] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0177] In this invention, the server includes means for acquiring and analyzing schedule information, means for automatically selecting the optimal means of transportation and route based on current location information and destination information, and means for analyzing the user's emotional state and personalizing the means of transportation and route. This makes it possible to suggest schedules and routes based on the user's emotional state.

[0178] "Schedule information" refers to data about a user's plans and schedule, including information related to the date, time, and location.

[0179] "Current location information" refers to data about the geographical location where the user is currently situated.

[0180] "Destination information" refers to data about the geographical location that the user wants to reach.

[0181] "Means of transportation" refers to the means used by a user to move from one place to another, and includes public transport, walking, cycling, and driving.

[0182] A "route" is the path taken from a point of origin to a destination, and is a set of paths based on a specific mode of transportation.

[0183] "Emotional state" refers to the emotional state a user is experiencing, and includes states such as stress, joy, and anger.

[0184] "Personalizing" means adjusting or customizing something to suit the individual user's characteristics and circumstances.

[0185] This invention provides a system using an emotion recognition engine to personalize user schedule management and route guidance. In this system, a server aggregates schedule information on the cloud and analyzes the user's schedule. A terminal acquires the user's voice and facial expression data and uses the emotion recognition engine to analyze their emotional state. For example, machine learning models using TENSORFLOW® or facial recognition technology using OpenCV may be applied.

[0186] The server dynamically adjusts the user's travel route and mode of transport based on acquired emotional information. Specifically, it uses the Google® Maps API to update traffic information in real time and suggests the optimal route and mode of transport according to the user's emotional state. The terminal provides the user with the suggested information and promotes a calm travel experience by playing relaxing music when the user is stressed.

[0187] This system uses the smartphone's camera and microphone to collect user emotion data and communicates it with a server via the internet. Cloud-based data processing allows users to receive seamless and immediate support.

[0188] For example, for users who feel stressed during the morning rush hour commute, the device can suggest routes with pleasant scenery and play calming music, making the start of their day more comfortable.

[0189] An example of a prompt for a generative AI model is, "How can I suggest the best route for a user who is feeling stressed during their morning commute?"

[0190] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0191] Step 1:

[0192] The device sends the user's schedule information to a server in the cloud. It uses data obtained from the user's calendar or schedule management app's API as input, and outputs it to the server as schedule information.

[0193] Step 2:

[0194] The server analyzes the received schedule information and organizes the user's appointments in chronological order. Here, it analyzes the schedule data to create a list of appointments and returns the analyzed schedule as output to the terminal.

[0195] Step 3:

[0196] The device acquires voice and facial expression data from the user and analyzes their emotional state using an emotion recognition engine. It uses data from the camera and microphone as input and sends the identified emotional state (e.g., stress, joy, neutral) to the server as output.

[0197] Step 4:

[0198] The server adjusts the optimal mode of transportation and route based on emotional state data. It incorporates real-time traffic information obtained using the Google Maps API, generates travel suggestions tailored to the user's emotions, and sends them to the device as output.

[0199] Step 5:

[0200] The terminal visually and audibly notifies the user of suggested route information and means of transportation received from the server. The input is route guidance information, and the output is route guidance to the user. If stress is detected, the terminal also plays relaxing music.

[0201] Step 6:

[0202] The user receives route guidance from the device and follows the instructions to travel. The journey is completed based on optimized information provided by the device. The result is a more comfortable travel experience.

[0203] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0204] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0205] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0206] [Second Embodiment]

[0207] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0208] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0209] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0210] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0211] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0212] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0213] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0214] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0215] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0216] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0217] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0218] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0219] This invention is a system that comprehensively supports users' schedule management and is implemented by an AI-based program equipped with various functions. The specific operation of the program involved is described below.

[0220] First, the server runs in a cloud environment and, with the user's permission, retrieves the user's schedule information using the API of the scheduling information service. This allows the system to analyze important information such as the date, time, location, participants, and purpose of the appointment using natural language processing technology, and extract the necessary data.

[0221] Next, the server obtains the user's current location information through location services and searches for the optimal mode of transport and route by comparing it with the planned destination. In this process, it uses public transport operation data and map services to take into account traffic congestion and delays, and creates a real-time travel plan.

[0222] The server then automatically updates the user's schedule based on the planned mode of transport and travel time. It determines a specific departure time and sets an alarm in advance to ensure the user departs on time. The alarm settings provide a more personalized experience by taking into account the user's past travel history and preparation time.

[0223] While on the move, the device provides users with real-time route guidance through visual and audio means. This is done through an application on the mobile device, helping users reach their destination without getting lost. For example, when using public transport, it clearly instructs users on which train or bus to take and where to transfer.

[0224] By using this system, users can automate cumbersome scheduling and travel planning, enabling them to manage their schedules more efficiently. In particular, the ability to obtain real-time traffic information allows for flexible responses to unforeseen circumstances. This reduces daily stress, allowing users to focus on important appointments.

[0225] The following describes the processing flow.

[0226] Step 1:

[0227] The server, with the user's permission, retrieves appointments from the user's calendar via the schedule information service's API. It then analyzes the date, time, location, and event details using natural language processing techniques to identify the schedule information.

[0228] Step 2:

[0229] The server uses location services to determine the user's current location. Next, based on the destination and acquired schedule information, it uses a transportation API to search for the optimal mode of transport and route. In this process, real-time traffic information is taken into consideration when selecting the route.

[0230] Step 3:

[0231] The calendar schedule is updated based on the mode of transport and route selected by the server. The departure time is calculated taking into account the travel time, and an alarm is set that takes into account the time needed for departure preparation, based on the user's past activity history.

[0232] Step 4:

[0233] When the departure time approaches, the device will notify the user with a set alarm. The alarm will alert the user with both sound and vibration to encourage them to prepare for departure on time.

[0234] Step 5:

[0235] When the device detects that it has started moving, it launches a map app or a dedicated app and provides real-time route guidance. The guidance is provided visually and audibly, offering the user the optimal route and transfer information to their destination.

[0236] Step 6:

[0237] If traffic delays or route changes occur during travel, the server updates information in real time and notifies the user of new instructions via their device. The user can then follow these instructions and reach their destination smoothly.

[0238] (Example 1)

[0239] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0240] In modern society, individuals lead busy daily lives, requiring efficient schedule management. However, manually setting appointments, selecting transportation methods, and dealing with traffic delays are time-consuming and cumbersome. Furthermore, changing schedules and obtaining real-time traffic information are also difficult. Therefore, automated schedule management systems are needed.

[0241] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0242] In this invention, the server includes means for acquiring schedule information via a communication device and analyzing the schedule using a generating AI model; means for automatically selecting the optimal means of transportation and route based on current location information and destination information; and means for updating the schedule based on the means of transportation and route and setting an alarm at an appropriate time for departure. This enables efficient schedule management and the automation of rapid travel planning.

[0243] A "communication device" is a device used to send and receive data, and is a device that has the function of acquiring or transmitting various information via a network.

[0244] "Schedule information" refers to detailed data about an individual's schedule, and is a general term for information including date, time, place, purpose, etc.

[0245] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and derive solutions based on user needs.

[0246] "Location information" refers to information that indicates the current geographical location of a device or individual, and is often expressed as coordinate data.

[0247] "Target destination information" refers to information about the place the user intends to travel to, and includes detailed data indicating the geographical location.

[0248] "Means of transportation" refers to the means or method of transport used to get to a destination, such as walking, public transport, or automobile.

[0249] A "route" is the path taken from the starting point to the destination, and is a route selected based on the shortest or most optimal method of travel.

[0250] An "alarm" is a function that provides a notification to the user at a specific time, and is a warning system set before an event begins.

[0251] "Traffic conditions" refers to information indicating the current state of road and transportation operations, including situations such as congestion and delays.

[0252] This invention provides a system that efficiently automates user schedule management. First, a server operates in a cloud environment and acquires schedule information via a communication device. This process utilizes common communication standards for schedule information services to collect schedule information. For example, it is designed so that users do not need to input detailed information such as the date, time, and location of meetings they are scheduled to attend.

[0253] Next, the server uses a generative AI model to analyze the acquired schedule information. At this stage, natural language processing techniques are used to interpret the purpose and priority of the schedule, supporting efficient schedule management. For example, a prompt sentence such as "Attend a meeting in the city tomorrow at 10am" is entered, and the server performs the necessary processing based on this.

[0254] The server also uses location services to collect current location information and compares it with destination information to select the optimal mode of transport and route. In this process, a wide range of options can be considered as means of transport, from public transport to automobiles.

[0255] Furthermore, the system updates the schedule based on the mode of transport and route, and sets alarms at appropriate times to ensure users can start their journey on time. The alarms are personalized based on the user's behavioral characteristics, allowing for efficient preparation for departure.

[0256] While on the move, the device provides users with real-time visual and audio route guidance. This is done through a smartphone application and helps users reach their destination without getting lost. This process of assisting with travel to the destination can clearly convey instructions for using public transportation and making transfers.

[0257] By using this system, users can gain automated schedule management and more efficient travel, reducing everyday stress and allowing them to focus on important matters.

[0258] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0259] Step 1:

[0260] The server retrieves user schedule information via the schedule information service API. It uses the user's account information as input and outputs data such as the date, time, location, and purpose of the appointment. Since this data is provided in several formats, it is converted to a standardized format for easier handling in subsequent processing.

[0261] Step 2:

[0262] The server analyzes the acquired schedule information using a generating AI model. The input is the schedule information standardized in step 1, and the output is an evaluation of the schedule priority and the importance of related tasks. Since natural language processing is performed in this process, it generates prompts that clarify the user's objective and performs specific actions to process them with the AI ​​model.

[0263] Step 3:

[0264] The server obtains the user's current location information from a location services. In this step, the server takes location data provided by the user's mobile device as input and outputs the user's real-time location coordinates. Based on this information, a specific step is taken to search for available public transportation options in real time.

[0265] Step 4:

[0266] The server selects the optimal mode of transport and route based on location and planned destination information. Input is current location and planned destination data, and output is the optimized mode of transport (e.g., train, car) and route. Real-time traffic information, such as congestion and service delays, is also considered in this process.

[0267] Step 5:

[0268] The server automatically updates the schedule and sets an alarm suitable for departure based on the selected mode of transport and route. Inputs include the optimal route information obtained in step 4 and the user's existing schedule. Outputs are the updated schedule information and alarm times, with the alarm time adjusted based on the user's past activity history.

[0269] Step 6:

[0270] The terminal provides users with real-time route guidance through both visual and audio. It uses optimal route information provided by a server as input, and outputs navigation instructions to the user. Specifically, it displays route information visually to the user via a map application and provides voice guidance for the next action.

[0271] (Application Example 1)

[0272] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0273] In modern society, managing daily schedules and the associated travel plans is complex, and there is a need for efficient systems to handle this. In particular, inappropriate traffic information and transportation choices can lead to wasted time and stress. The challenge is to improve the overall quality of life by enabling systems to manage users' schedules and travel more efficiently.

[0274] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0275] In this invention, the server includes means for acquiring and analyzing schedule information, means for automatically selecting the optimal means of transportation and route based on current location information and destination information, and means for updating the schedule based on the means of transportation and route and setting time notifications appropriate for the departure time. This enables users to manage their travel and schedules more efficiently and enjoy a smarter urban lifestyle.

[0276] "Schedule information" refers to information that shows the user's daily schedule and activity plans.

[0277] "Means of analysis" refers to techniques or devices for interpreting schedule information and extracting necessary data.

[0278] "Current location information" refers to geographical data used to identify the user's current location.

[0279] "Destination information" refers to geographical data about the place the user is heading to.

[0280] The "optimal mode of transport" means choosing the most efficient way to travel based on current traffic conditions and schedules.

[0281] "Route" refers to the path for moving from the starting point to the destination point.

[0282] "Means for setting time notification" refers to the technology or device for notifying at an appropriate timing so that the user can act as planned.

[0283] "Location data" refers to information regarding a specific geographical location.

[0284] "Means for providing visually and aurally" refers to the technology or device for presenting information in a visible form to the user and conveying it by voice.

[0285] This invention is a system for improving schedule management and enhancing the user's daily life. This system operates centering around a server running on a cloud environment and uses multiple hardware and software.

[0286] The server utilizes the API of the planning information service to acquire and analyze schedule information. Through this API, the user's schedule information is collected and analyzed using natural language processing. The software used here incorporates an AI model for providing natural language processing technology.

[0287] The server also receives the current location information from the user terminal and automatically selects the optimal means of transportation and route. For this selection, external APIs providing location information services and map data are utilized. In particular, a public data stream is used to update traffic information in real time.

[0288] To support the user's movement, the server predicts the travel time and sends a notification suitable for the departure time to the terminal. This process incorporates a machine learning algorithm that utilizes past movement history and usage patterns.

[0289] While on the move, the device provides navigation information through sight and sound. This assumes the user is using a wearable device such as smart glasses. By utilizing the display and speaker of the wearable device, the user can receive guidance through sight and sound.

[0290] By actually using this system, users can enjoy comfortable and efficient urban living through optimized transportation options. For example, when attending a major event, they can be presented with the optimal mode of transportation and route before departure, and receive notifications of departure times.

[0291] An example of a prompt to a generating AI model is: "Based on the user's current location, please suggest the best mode of transportation and route for their planned station visit at 12:00. Please also consider traffic information."

[0292] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0293] Step 1:

[0294] The server retrieves user schedule information using the API of the planning information service. It receives user authentication information as input and retrieves scheduled information as output. The retrieved data is analyzed using natural language processing techniques to extract important keywords and date / time information.

[0295] Step 2:

[0296] The server receives current location information transmitted from the terminal. Based on this information, and combined with destination information, it uses external location services and map data to calculate the optimal mode of transport and route. The input is current location data from the terminal, and the output is information on the mode of transport and route. Traffic conditions and public transport operation data are also taken into consideration.

[0297] Step 3:

[0298] The server updates the user's schedule based on the acquired optimal mode of transport and route information. The inputs here are the travel information obtained in step 2 and the user's schedule effect. The output is a departure time notification adjusted to ensure the user travels on time. The notification timing is optimized using known patterns based on past travel history.

[0299] Step 4:

[0300] The terminal provides the user with real-time visual and auditory guidance. Input consists of route information and current location information sent from the server. Output includes specific actions such as displaying route guidance on the user's smart glasses or other wearable devices. During this process, the guidance information is updated as needed to ensure smooth travel to the destination.

[0301] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0302] This invention is a system that incorporates an emotion recognition engine to further personalize user schedule management and route guidance. In addition to its basic function of automatically analyzing the user's schedule and suggesting the optimal means of transportation and route, this system is capable of dynamic adjustments according to the user's emotional state.

[0303] First, the server uses conventional scheduling information acquisition methods to aggregate users' schedules on the cloud. This allows scheduling information related to date, time, and location to be properly analyzed.

[0304] Next, through the operation of the emotion engine, the terminal analyzes the voice and facial expression data obtained from the user. Here, the emotional state of the user (e.g., stress, joy, anger, etc.) is discriminated, and based on the result, further adjustment of the interaction is carried out. For example, if it is determined that the user is feeling stressed, the server changes the proposed means of transportation to a more comfortable option or makes adjustments to gently guide the user through voice during the travel route.

[0305] Also, while the server updates traffic information in real time, it can dynamically adjust schedules and alarms according to the user's emotions. If the user is in a relaxed state, a schedule with some leeway is displayed, while if the user is in a hurry, measures such as prioritizing a faster travel plan are made.

[0306] As a specific example, when the user feels stressed during the morning commute, the terminal can utilize the emotion engine to play music with a relaxation effect or propose a more comfortable route to reduce the stress.

[0307] Thereby, compared with the conventional schedule management system, the present invention can provide a more flexible and personalized experience considering the individual emotional needs of the user. By implementing this system, the user can significantly reduce the stress related to schedules and travel in daily life.

[0308] The following describes the processing flow.

[0309] Step 1:

[0310] The user inputs speech and facial expression data into the system through a smartphone or wearable device. By giving permission to use the emotion engine in the initial settings, the device is prepared to acquire emotion data.

[0311] Step 2:

[0312] The device runs an emotion recognition algorithm that analyzes the user's voice and facial expressions to determine the user's emotional state. This algorithm analyzes the user's speech patterns and facial features in real time to detect emotions such as stress, relief, anger, and joy.

[0313] Step 3:

[0314] The server uses the schedule information service's API to retrieve the user's schedule information from the cloud. The retrieved data is analyzed, and the date, time, location, purpose, and other details of the appointment are extracted.

[0315] Step 4:

[0316] The server acquires location information and selects the optimal mode of transport and route based on destination information. During this process, it incorporates the results of an emotion engine, adjusting the mode of transport and route according to the user's emotional state. For example, if the user is feeling stressed, less congested routes and places to avoid will be prioritized.

[0317] Step 5:

[0318] The server collects traffic information in real time and creates a travel schedule that takes emotions into account. Next, it calculates departure times and automatically sets alarms. During this process, the alarm sound can also be adjusted according to emotions.

[0319] Step 6:

[0320] The device notifies the user at the time the alarm it has set will sound. At this point, it prompts the user to begin preparations and guides them through the process to ensure a comfortable journey.

[0321] Step 7:

[0322] While on the move, the device provides the user with visual and audio route guidance. An emotion engine continuously monitors the user's emotions and adjusts the tempo and tone of the guidance accordingly. For example, if the user is relaxed, the guidance can be made more cheerful.

[0323] Step 8:

[0324] Servers and terminals regularly update information while in transit, allowing for immediate responses to unexpected traffic disruptions or schedule changes. Route and schedule readjustments based on emotional changes are also made during this step.

[0325] (Example 2)

[0326] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0327] Conventional schedule management and route guidance systems suggest modes of transport and routes based on predetermined criteria without considering the individual emotional state of the user. As a result, they fail to provide appropriate suggestions when the user is stressed or in a hurry, leading to decreased user satisfaction. The present invention aims to dynamically adjust modes of transport and routes according to the user's emotional state, providing a comfortable and personalized experience for the user.

[0328] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0329] In this invention, the server includes means for acquiring and analyzing schedule information, means for automatically selecting the optimal means of transportation and route based on current location information and destination information, and means for analyzing the user's emotional state using emotion recognition technology and dynamically adjusting the means of transportation and route based on the analysis results. This makes it possible to suggest the optimal transportation according to the user's emotional state.

[0330] "Schedule information" refers to detailed data such as the date, time, location, and content of appointments and events set by the user.

[0331] "Means of analysis" refers to devices or software that analyze acquired data and perform computational processing or judgments to give it meaning.

[0332] "Current location information" refers to digital data that indicates the user's real-time location, and includes latitude, longitude, and altitude.

[0333] "Destination information" refers to location data of the place the user is heading to, including addresses and place names.

[0334] "Optimal mode of transportation" refers to the method of travel that is deemed most efficient and convenient based on the user's conditions and environment.

[0335] "Means for automatically selecting a route" refers to a function that automatically determines the route to reach the user's destination using algorithms or software.

[0336] "Emotion recognition technology" refers to technology that analyzes a person's voice, facial expressions, and behavior to identify their emotional state at that time.

[0337] "Traffic information" refers to data that affects travel, such as the operating status of public transportation and road congestion.

[0338] "Means of providing route guidance visually and audibly" refers to technologies that provide users with a route to their destination through visual map displays and voice instructions.

[0339] This invention is a system that incorporates emotion recognition technology to highly personalize user schedule management and navigation guidance. Specific embodiments are described below.

[0340] The server uses external connections to manage and analyze schedule information in the cloud. Specifically, it retrieves information including date, time, location, and event details via APIs of publicly available schedule information services. This allows for a proper understanding and management of users' planned activities.

[0341] The device uses the smartphone's built-in microphone and camera to acquire the user's voice and facial expression data. This device is equipped with an emotion recognition engine and utilizes Microsoft Azure Cognitive Services or similar services to analyze the user's emotional state. For example, the device can determine whether the user is stressed or relaxed.

[0342] The server dynamically adjusts the optimal mode of transport and route based on analyzed emotion data and schedule information. Specifically, it uses a map service API to analyze traffic flow in order to reflect real-time traffic information. It also flexibly selects the mode of transport to present according to the user's emotional state. When the user is relaxed, it recommends a leisurely mode of transport, while when they are in a hurry, it suggests the fastest route.

[0343] Users can view and select suggested schedules and modes of transportation through their device, and the system provides services based on those selections. For example, if stress levels are measured during the morning commute, the device will play relaxation music, and the server will suggest a comfortable and efficient commute route. Another example of a prompt using a generative AI model is, "Please suggest a suitable restaurant for lunch when the user is in a relaxed state." This prompt allows the AI ​​to generate suggestions appropriate to the situation.

[0344] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0345] Step 1:

[0346] The server obtains schedule information via an external connection. It uses user registration information as input. The retrieved data includes date, time, location, and event type. This data is analyzed to create the user's schedule. The output is the analyzed schedule information.

[0347] Step 2:

[0348] The device collects the user's voice and facial expression data using a microphone and camera. The voice and facial expressions are recorded in real time and input into an emotion recognition engine. The engine analyzes this data to identify the user's emotional state (e.g., stress, relaxation). The output is data on the identified emotional state.

[0349] Step 3:

[0350] The server receives emotional state data and schedule information, and dynamically adjusts the mode of transport and route according to the user's state. Real-time traffic information is also included as input. As data processing, the optimal route is calculated using a map service API. The output is the adjusted mode of transport and route plan.

[0351] Step 4:

[0352] The terminal notifies the user of the travel plan sent from the server. It provides visual map displays and voice guidance, and plays relaxation music as needed. It uses user emotions and travel plan data as input. The output is user interaction and guidance.

[0353] Step 5:

[0354] The user selects a suggested mode of transport and route based on information provided from their device and sends feedback to the server. This feedback is used for future suggestions. The provided travel plan is used as input. The output is the user's selection and feedback data.

[0355] (Application Example 2)

[0356] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0357] Traditional scheduling management systems often fail to adequately address individual user needs because they suggest schedules and routes without considering the user's emotional state. In particular, certain emotional states during travel can cause stress and discomfort, potentially impacting the execution of the schedule. In such situations, a more personalized system is needed.

[0358] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0359] In this invention, the server includes means for acquiring and analyzing schedule information, means for automatically selecting the optimal means of transportation and route based on current location information and destination information, and means for analyzing the user's emotional state and personalizing the means of transportation and route. This makes it possible to suggest schedules and routes based on the user's emotional state.

[0360] "Schedule information" refers to data about a user's plans and schedule, including information related to the date, time, and location.

[0361] "Current location information" refers to data about the geographical location where the user is currently situated.

[0362] "Destination information" refers to data about the geographical location that the user wants to reach.

[0363] "Means of transportation" refers to the means used by a user to move from one place to another, and includes public transport, walking, cycling, and driving.

[0364] A "route" is the path taken from a point of origin to a destination, and is a set of paths based on a specific mode of transportation.

[0365] "Emotional state" refers to the emotional state a user is experiencing, and includes states such as stress, joy, and anger.

[0366] "Personalizing" means adjusting or customizing something to suit the individual user's characteristics and circumstances.

[0367] This invention provides a system using an emotion recognition engine to personalize user schedule management and route guidance. In this system, a server aggregates schedule information on the cloud and analyzes the user's schedule. A terminal acquires the user's voice and facial expression data and uses the emotion recognition engine to analyze their emotional state. For example, machine learning models using TensorFlow or facial recognition technology using OpenCV may be applied.

[0368] The server dynamically adjusts the user's travel route and mode of transport based on acquired emotional information. Specifically, it uses the Google Maps API to update traffic information in real time and suggests the optimal route and mode of transport according to the user's emotional state. The device provides the user with the suggested information and promotes a calm travel experience by playing relaxing music when the user is stressed.

[0369] This system uses the smartphone's camera and microphone to collect user emotion data and communicates it with a server via the internet. Cloud-based data processing allows users to receive seamless and immediate support.

[0370] For example, for users who feel stressed during the morning rush hour commute, the device can suggest routes with pleasant scenery and play calming music, making the start of their day more comfortable.

[0371] An example of a prompt for a generative AI model is, "How can I suggest the best route for a user who is feeling stressed during their morning commute?"

[0372] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0373] Step 1:

[0374] The device sends the user's schedule information to a server in the cloud. It uses data obtained from the user's calendar or schedule management app's API as input, and outputs it to the server as schedule information.

[0375] Step 2:

[0376] The server analyzes the received schedule information and organizes the user's appointments in chronological order. Here, it analyzes the schedule data to create a list of appointments and returns the analyzed schedule as output to the terminal.

[0377] Step 3:

[0378] The device acquires voice and facial expression data from the user and analyzes their emotional state using an emotion recognition engine. It uses data from the camera and microphone as input and sends the identified emotional state (e.g., stress, joy, neutral) to the server as output.

[0379] Step 4:

[0380] The server adjusts the optimal mode of transportation and route based on emotional state data. It incorporates real-time traffic information obtained using the Google Maps API, generates travel suggestions tailored to the user's emotions, and sends them to the device as output.

[0381] Step 5:

[0382] The terminal visually and audibly notifies the user of suggested route information and means of transportation received from the server. The input is route guidance information, and the output is route guidance to the user. If stress is detected, the terminal also plays relaxing music.

[0383] Step 6:

[0384] The user receives route guidance from the device and follows the instructions to travel. The journey is completed based on optimized information provided by the device. The result is a more comfortable travel experience.

[0385] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0386] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0387] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0388] [Third Embodiment]

[0389] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0390] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0391] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0392] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0393] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0394] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0395] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0396] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0397] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0398] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0399] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0400] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0401] This invention is a system that comprehensively supports users' schedule management and is implemented by an AI-based program equipped with various functions. The specific operation of the program involved is described below.

[0402] First, the server runs in a cloud environment and, with the user's permission, retrieves the user's schedule information using the API of the scheduling information service. This allows the system to analyze important information such as the date, time, location, participants, and purpose of the appointment using natural language processing technology, and extract the necessary data.

[0403] Next, the server obtains the user's current location information through location services and searches for the optimal mode of transport and route by comparing it with the planned destination. In this process, it uses public transport operation data and map services to take into account traffic congestion and delays, and creates a real-time travel plan.

[0404] The server then automatically updates the user's schedule based on the planned mode of transport and travel time. It determines a specific departure time and sets an alarm in advance to ensure the user departs on time. The alarm settings provide a more personalized experience by taking into account the user's past travel history and preparation time.

[0405] While on the move, the device provides users with real-time route guidance through visual and audio means. This is done through an application on the mobile device, helping users reach their destination without getting lost. For example, when using public transport, it clearly instructs users on which train or bus to take and where to transfer.

[0406] By using this system, users can automate cumbersome scheduling and travel planning, enabling them to manage their schedules more efficiently. In particular, the ability to obtain real-time traffic information allows for flexible responses to unforeseen circumstances. This reduces daily stress, allowing users to focus on important appointments.

[0407] The following describes the processing flow.

[0408] Step 1:

[0409] The server, with the user's permission, retrieves appointments from the user's calendar via the schedule information service's API. It then analyzes the date, time, location, and event details using natural language processing techniques to identify the schedule information.

[0410] Step 2:

[0411] The server uses location services to determine the user's current location. Next, based on the destination and acquired schedule information, it uses a transportation API to search for the optimal mode of transport and route. In this process, real-time traffic information is taken into consideration when selecting the route.

[0412] Step 3:

[0413] The calendar schedule is updated based on the mode of transport and route selected by the server. The departure time is calculated taking into account the travel time, and an alarm is set that takes into account the time needed for departure preparation, based on the user's past activity history.

[0414] Step 4:

[0415] When the departure time approaches, the device will notify the user with a set alarm. The alarm will alert the user with both sound and vibration to encourage them to prepare for departure on time.

[0416] Step 5:

[0417] When the device detects that it has started moving, it launches a map app or a dedicated app and provides real-time route guidance. The guidance is provided visually and audibly, offering the user the optimal route and transfer information to their destination.

[0418] Step 6:

[0419] If traffic delays or route changes occur during travel, the server updates information in real time and notifies the user of new instructions via their device. The user can then follow these instructions and reach their destination smoothly.

[0420] (Example 1)

[0421] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0422] In modern society, individuals lead busy daily lives, requiring efficient schedule management. However, manually setting appointments, selecting transportation methods, and dealing with traffic delays are time-consuming and cumbersome. Furthermore, changing schedules and obtaining real-time traffic information are also difficult. Therefore, automated schedule management systems are needed.

[0423] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0424] In this invention, the server includes means for acquiring schedule information via a communication device and analyzing the schedule using a generating AI model; means for automatically selecting the optimal means of transportation and route based on current location information and destination information; and means for updating the schedule based on the means of transportation and route and setting an alarm at an appropriate time for departure. This enables efficient schedule management and the automation of rapid travel planning.

[0425] A "communication device" is a device used to send and receive data, and is a device that has the function of acquiring or transmitting various information via a network.

[0426] "Schedule information" refers to detailed data about an individual's schedule, and is a general term for information including date, time, place, purpose, etc.

[0427] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and derive solutions based on user needs.

[0428] "Location information" refers to information that indicates the current geographical location of a device or individual, and is often expressed as coordinate data.

[0429] "Target destination information" refers to information about the place the user intends to travel to, and includes detailed data indicating the geographical location.

[0430] "Means of transportation" refers to the means or method of transport used to get to a destination, such as walking, public transport, or automobile.

[0431] A "route" is the path taken from the starting point to the destination, and is a route selected based on the shortest or most optimal method of travel.

[0432] An "alarm" is a function that provides a notification to the user at a specific time, and is a warning system set before an event begins.

[0433] "Traffic conditions" refers to information indicating the current state of road and transportation operations, including situations such as congestion and delays.

[0434] This invention provides a system that efficiently automates user schedule management. First, a server operates in a cloud environment and acquires schedule information via a communication device. This process utilizes common communication standards for schedule information services to collect schedule information. For example, it is designed so that users do not need to input detailed information such as the date, time, and location of meetings they are scheduled to attend.

[0435] Next, the server uses a generative AI model to analyze the acquired schedule information. At this stage, natural language processing techniques are used to interpret the purpose and priority of the schedule, supporting efficient schedule management. For example, a prompt sentence such as "Attend a meeting in the city tomorrow at 10am" is entered, and the server performs the necessary processing based on this.

[0436] The server also uses location services to collect current location information and compares it with destination information to select the optimal mode of transport and route. In this process, a wide range of options can be considered as means of transport, from public transport to automobiles.

[0437] Furthermore, the system updates the schedule based on the mode of transport and route, and sets alarms at appropriate times to ensure users can start their journey on time. The alarms are personalized based on the user's behavioral characteristics, allowing for efficient preparation for departure.

[0438] While on the move, the device provides users with real-time visual and audio route guidance. This is done through a smartphone application and helps users reach their destination without getting lost. This process of assisting with travel to the destination can clearly convey instructions for using public transportation and making transfers.

[0439] By using this system, users can gain automated schedule management and more efficient travel, reducing everyday stress and allowing them to focus on important matters.

[0440] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0441] Step 1:

[0442] The server retrieves user schedule information via the schedule information service API. It uses the user's account information as input and outputs data such as the date, time, location, and purpose of the appointment. Since this data is provided in several formats, it is converted to a standardized format for easier handling in subsequent processing.

[0443] Step 2:

[0444] The server analyzes the acquired schedule information using a generating AI model. The input is the schedule information standardized in step 1, and the output is an evaluation of the schedule priority and the importance of related tasks. Since natural language processing is performed in this process, it generates prompts that clarify the user's objective and performs specific actions to process them with the AI ​​model.

[0445] Step 3:

[0446] The server obtains the user's current location information from a location services. In this step, the server takes location data provided by the user's mobile device as input and outputs the user's real-time location coordinates. Based on this information, a specific step is taken to search for available public transportation options in real time.

[0447] Step 4:

[0448] The server selects the optimal mode of transport and route based on location and planned destination information. Input is current location and planned destination data, and output is the optimized mode of transport (e.g., train, car) and route. Real-time traffic information, such as congestion and service delays, is also considered in this process.

[0449] Step 5:

[0450] The server automatically updates the schedule and sets an alarm suitable for departure based on the selected mode of transport and route. Inputs include the optimal route information obtained in step 4 and the user's existing schedule. Outputs are the updated schedule information and alarm times, with the alarm time adjusted based on the user's past activity history.

[0451] Step 6:

[0452] The terminal provides users with real-time route guidance through both visual and audio. It uses optimal route information provided by a server as input, and outputs navigation instructions to the user. Specifically, it displays route information visually to the user via a map application and provides voice guidance for the next action.

[0453] (Application Example 1)

[0454] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0455] In modern society, managing daily schedules and the associated travel plans is complex, and there is a need for efficient systems to handle this. In particular, inappropriate traffic information and transportation choices can lead to wasted time and stress. The challenge is to improve the overall quality of life by enabling systems to manage users' schedules and travel more efficiently.

[0456] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0457] In this invention, the server includes means for acquiring and analyzing schedule information, means for automatically selecting the optimal means of transportation and route based on current location information and destination information, and means for updating the schedule based on the means of transportation and route and setting time notifications appropriate for the departure time. This enables users to manage their travel and schedules more efficiently and enjoy a smarter urban lifestyle.

[0458] "Schedule information" refers to information that shows the user's daily schedule and activity plans.

[0459] "Means of analysis" refers to techniques or devices for interpreting schedule information and extracting necessary data.

[0460] "Current location information" refers to geographical data used to identify the user's current location.

[0461] "Destination information" refers to geographical data about the place the user is heading to.

[0462] The "optimal mode of transport" means choosing the most efficient way to travel based on current traffic conditions and schedules.

[0463] A "route" is the path taken to travel from a starting point to a destination.

[0464] "Means for setting time notifications" refers to technologies or devices that notify users at the appropriate time so that they can act according to their schedule.

[0465] "Location data" refers to information about a specific geographical location.

[0466] "Means of providing information visually and aurally" refers to technologies or devices for presenting information to users in a visual form and conveying it audibly.

[0467] This invention is a system designed to streamline schedule management and improve users' daily lives. This system operates primarily on a server running in a cloud environment and utilizes multiple hardware and software components.

[0468] The server utilizes the planning information service's API to acquire and analyze schedule information. Through this API, it collects user schedule information and performs analysis using natural language processing. The software used incorporates an AI model to provide natural language processing capabilities.

[0469] The server also receives location information from the user's terminal and automatically selects the optimal mode of transport and route. This selection utilizes external APIs that provide location services and map data. In particular, it uses public data streams to update traffic information in real time.

[0470] To support users' travel, the server predicts travel time and sends notifications to the device that are appropriate for the departure time. This process incorporates machine learning algorithms that utilize past travel history and usage patterns.

[0471] While on the move, the device provides navigation information through sight and sound. This assumes the user is using a wearable device such as smart glasses. By utilizing the display and speaker of the wearable device, the user can receive guidance through sight and sound.

[0472] By actually using this system, users can enjoy comfortable and efficient urban living through optimized transportation options. For example, when attending a major event, they can be presented with the optimal mode of transportation and route before departure, and receive notifications of departure times.

[0473] An example of a prompt to a generating AI model is: "Based on the user's current location, please suggest the best mode of transportation and route for their planned station visit at 12:00. Please also consider traffic information."

[0474] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0475] Step 1:

[0476] The server retrieves user schedule information using the API of the planning information service. It receives user authentication information as input and retrieves scheduled information as output. The retrieved data is analyzed using natural language processing techniques to extract important keywords and date / time information.

[0477] Step 2:

[0478] The server receives current location information transmitted from the terminal. Based on this information, and combined with destination information, it uses external location services and map data to calculate the optimal mode of transport and route. The input is current location data from the terminal, and the output is information on the mode of transport and route. Traffic conditions and public transport operation data are also taken into consideration.

[0479] Step 3:

[0480] The server updates the user's schedule based on the acquired optimal mode of transport and route information. The inputs here are the travel information obtained in step 2 and the user's schedule effect. The output is a departure time notification adjusted to ensure the user travels on time. The notification timing is optimized using known patterns based on past travel history.

[0481] Step 4:

[0482] The terminal provides the user with real-time visual and auditory guidance. Input consists of route information and current location information sent from the server. Output includes specific actions such as displaying route guidance on the user's smart glasses or other wearable devices. During this process, the guidance information is updated as needed to ensure smooth travel to the destination.

[0483] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0484] This invention is a system that incorporates an emotion recognition engine to further personalize user schedule management and route guidance. In addition to its basic function of automatically analyzing the user's schedule and suggesting the optimal means of transportation and route, this system is capable of dynamic adjustments according to the user's emotional state.

[0485] First, the server uses conventional scheduling information acquisition methods to aggregate users' schedules on the cloud. This allows scheduling information related to date, time, and location to be properly analyzed.

[0486] Next, the emotion engine analyzes the voice and facial expression data acquired by the device from the user. Here, the user's emotional state (e.g., stress, joy, anger) is determined, and further adjustments to the interaction are made based on the results. For example, if it is determined that the user is feeling stressed, the server will change the suggested mode of transportation to a more comfortable option or adjust the voice guidance for the travel route to be gentler.

[0487] Furthermore, the server can update traffic information in real time and dynamically adjust schedules and alarms according to the user's mood. If the user is relaxed, a schedule with ample time will be displayed, while if they are in a hurry, the system will prioritize faster travel plans.

[0488] For example, if a user is feeling stressed during their morning commute, the device can use its emotional engine to reduce that stress by playing relaxing music or suggesting a more comfortable route.

[0489] As a result, the present invention can provide a more flexible and personalized experience that takes into account the individual emotional needs of each user, compared to conventional schedule management systems. By implementing this system, users can significantly reduce stress related to scheduling and travel in their daily lives.

[0490] The following describes the processing flow.

[0491] Step 1:

[0492] Users input speech and facial expression data into the system via smartphones or wearable devices. By granting permission to use the emotion engine during initial setup, the device is prepared to acquire emotion data.

[0493] Step 2:

[0494] The device runs an emotion recognition algorithm that analyzes the user's voice and facial expressions to determine the user's emotional state. This algorithm analyzes the user's speech patterns and facial features in real time to detect emotions such as stress, relief, anger, and joy.

[0495] Step 3:

[0496] The server uses the schedule information service's API to retrieve the user's schedule information from the cloud. The retrieved data is analyzed, and the date, time, location, purpose, and other details of the appointment are extracted.

[0497] Step 4:

[0498] The server acquires location information and selects the optimal mode of transport and route based on destination information. During this process, it incorporates the results of an emotion engine, adjusting the mode of transport and route according to the user's emotional state. For example, if the user is feeling stressed, less congested routes and places to avoid will be prioritized.

[0499] Step 5:

[0500] The server collects traffic information in real time and creates a travel schedule that takes emotions into account. Next, it calculates departure times and automatically sets alarms. During this process, the alarm sound can also be adjusted according to emotions.

[0501] Step 6:

[0502] The device notifies the user at the time the alarm it has set will sound. At this point, it prompts the user to begin preparations and guides them through the process to ensure a comfortable journey.

[0503] Step 7:

[0504] While on the move, the device provides the user with visual and audio route guidance. An emotion engine continuously monitors the user's emotions and adjusts the tempo and tone of the guidance accordingly. For example, if the user is relaxed, the guidance can be made more cheerful.

[0505] Step 8:

[0506] Servers and terminals regularly update information while in transit, allowing for immediate responses to unexpected traffic disruptions or schedule changes. Route and schedule readjustments based on emotional changes are also made during this step.

[0507] (Example 2)

[0508] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0509] Conventional schedule management and route guidance systems suggest modes of transport and routes based on predetermined criteria without considering the individual emotional state of the user. As a result, they fail to provide appropriate suggestions when the user is stressed or in a hurry, leading to decreased user satisfaction. The present invention aims to dynamically adjust modes of transport and routes according to the user's emotional state, providing a comfortable and personalized experience for the user.

[0510] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0511] In this invention, the server includes means for acquiring and analyzing schedule information, means for automatically selecting the optimal means of transportation and route based on current location information and destination information, and means for analyzing the user's emotional state using emotion recognition technology and dynamically adjusting the means of transportation and route based on the analysis results. This makes it possible to suggest the optimal transportation according to the user's emotional state.

[0512] "Schedule information" refers to detailed data such as the date, time, location, and content of appointments and events set by the user.

[0513] "Means of analysis" refers to devices or software that analyze acquired data and perform computational processing or judgments to give it meaning.

[0514] "Current location information" refers to digital data that indicates the user's real-time location, and includes latitude, longitude, and altitude.

[0515] "Destination information" refers to location data of the place the user is heading to, including addresses and place names.

[0516] "Optimal mode of transportation" refers to the method of travel that is deemed most efficient and convenient based on the user's conditions and environment.

[0517] "Means for automatically selecting a route" refers to a function that automatically determines the route to reach the user's destination using algorithms or software.

[0518] "Emotion recognition technology" refers to technology that analyzes a person's voice, facial expressions, and behavior to identify their emotional state at that time.

[0519] "Traffic information" refers to data that affects travel, such as the operating status of public transportation and road congestion.

[0520] "Means of providing route guidance visually and audibly" refers to technologies that provide users with a route to their destination through visual map displays and voice instructions.

[0521] This invention is a system that incorporates emotion recognition technology to highly personalize user schedule management and navigation guidance. Specific embodiments are described below.

[0522] The server uses external connections to manage and analyze schedule information in the cloud. Specifically, it retrieves information including date, time, location, and event details via APIs of publicly available schedule information services. This allows for a proper understanding and management of users' planned activities.

[0523] The device uses the smartphone's built-in microphone and camera to acquire the user's voice and facial expression data. This device is equipped with an emotion recognition engine and utilizes Microsoft Azure Cognitive Services or similar services to analyze the user's emotional state. For example, the device can determine whether the user is stressed or relaxed.

[0524] The server dynamically adjusts the optimal mode of transport and route based on analyzed emotion data and schedule information. Specifically, it uses a map service API to analyze traffic flow in order to reflect real-time traffic information. It also flexibly selects the mode of transport to present according to the user's emotional state. When the user is relaxed, it recommends a leisurely mode of transport, while when they are in a hurry, it suggests the fastest route.

[0525] Users can view and select suggested schedules and modes of transportation through their device, and the system provides services based on those selections. For example, if stress levels are measured during the morning commute, the device will play relaxation music, and the server will suggest a comfortable and efficient commute route. Another example of a prompt using a generative AI model is, "Please suggest a suitable restaurant for lunch when the user is in a relaxed state." This prompt allows the AI ​​to generate suggestions appropriate to the situation.

[0526] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0527] Step 1:

[0528] The server obtains schedule information via an external connection. It uses user registration information as input. The retrieved data includes date, time, location, and event type. This data is analyzed to create the user's schedule. The output is the analyzed schedule information.

[0529] Step 2:

[0530] The device collects the user's voice and facial expression data using a microphone and camera. The voice and facial expressions are recorded in real time and input into an emotion recognition engine. The engine analyzes this data to identify the user's emotional state (e.g., stress, relaxation). The output is data on the identified emotional state.

[0531] Step 3:

[0532] The server receives emotional state data and schedule information, and dynamically adjusts the mode of transport and route according to the user's state. Real-time traffic information is also included as input. As data processing, the optimal route is calculated using a map service API. The output is the adjusted mode of transport and route plan.

[0533] Step 4:

[0534] The terminal notifies the user of the travel plan sent from the server. It provides visual map displays and voice guidance, and plays relaxation music as needed. It uses user emotions and travel plan data as input. The output is user interaction and guidance.

[0535] Step 5:

[0536] The user selects a suggested mode of transport and route based on information provided from their device and sends feedback to the server. This feedback is used for future suggestions. The provided travel plan is used as input. The output is the user's selection and feedback data.

[0537] (Application Example 2)

[0538] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0539] Traditional scheduling management systems often fail to adequately address individual user needs because they suggest schedules and routes without considering the user's emotional state. In particular, certain emotional states during travel can cause stress and discomfort, potentially impacting the execution of the schedule. In such situations, a more personalized system is needed.

[0540] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0541] In this invention, the server includes means for acquiring and analyzing schedule information, means for automatically selecting the optimal means of transportation and route based on current location information and destination information, and means for analyzing the user's emotional state and personalizing the means of transportation and route. This makes it possible to suggest schedules and routes based on the user's emotional state.

[0542] "Schedule information" refers to data about a user's plans and schedule, including information related to the date, time, and location.

[0543] "Current location information" refers to data about the geographical location where the user is currently situated.

[0544] "Destination information" refers to data about the geographical location that the user wants to reach.

[0545] "Means of transportation" refers to the means used by a user to move from one place to another, and includes public transport, walking, cycling, and driving.

[0546] A "route" is the path taken from a point of origin to a destination, and is a set of paths based on a specific mode of transportation.

[0547] "Emotional state" refers to the emotional state a user is experiencing, and includes states such as stress, joy, and anger.

[0548] "Personalizing" means adjusting or customizing something to suit the individual user's characteristics and circumstances.

[0549] This invention provides a system using an emotion recognition engine to personalize user schedule management and route guidance. In this system, a server aggregates schedule information on the cloud and analyzes the user's schedule. A terminal acquires the user's voice and facial expression data and uses the emotion recognition engine to analyze their emotional state. For example, machine learning models using TensorFlow or facial recognition technology using OpenCV may be applied.

[0550] The server dynamically adjusts the user's travel route and mode of transport based on acquired emotional information. Specifically, it uses the Google Maps API to update traffic information in real time and suggests the optimal route and mode of transport according to the user's emotional state. The device provides the user with the suggested information and promotes a calm travel experience by playing relaxing music when the user is stressed.

[0551] This system uses the smartphone's camera and microphone to collect user emotion data and communicates it with a server via the internet. Cloud-based data processing allows users to receive seamless and immediate support.

[0552] For example, for users who feel stressed during the morning rush hour commute, the device can suggest routes with pleasant scenery and play calming music, making the start of their day more comfortable.

[0553] An example of a prompt for a generative AI model is, "How can I suggest the best route for a user who is feeling stressed during their morning commute?"

[0554] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0555] Step 1:

[0556] The device sends the user's schedule information to a server in the cloud. It uses data obtained from the user's calendar or schedule management app's API as input, and outputs it to the server as schedule information.

[0557] Step 2:

[0558] The server analyzes the received schedule information and organizes the user's appointments in chronological order. Here, it analyzes the schedule data to create a list of appointments and returns the analyzed schedule as output to the terminal.

[0559] Step 3:

[0560] The device acquires voice and facial expression data from the user and analyzes their emotional state using an emotion recognition engine. It uses data from the camera and microphone as input and sends the identified emotional state (e.g., stress, joy, neutral) to the server as output.

[0561] Step 4:

[0562] The server adjusts the optimal mode of transportation and route based on emotional state data. It incorporates real-time traffic information obtained using the Google Maps API, generates travel suggestions tailored to the user's emotions, and sends them to the device as output.

[0563] Step 5:

[0564] The terminal visually and audibly notifies the user of suggested route information and means of transportation received from the server. The input is route guidance information, and the output is route guidance to the user. If stress is detected, the terminal also plays relaxing music.

[0565] Step 6:

[0566] The user receives route guidance from the device and follows the instructions to travel. The journey is completed based on optimized information provided by the device. The result is a more comfortable travel experience.

[0567] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0568] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0569] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0570] [Fourth Embodiment]

[0571] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0572] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0573] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0574] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0575] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0576] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0577] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0578] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0579] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0580] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0581] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0582] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0583] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0584] This invention is a system that comprehensively supports users' schedule management and is implemented by an AI-based program equipped with various functions. The specific operation of the program involved is described below.

[0585] First, the server runs in a cloud environment and, with the user's permission, retrieves the user's schedule information using the API of the scheduling information service. This allows the system to analyze important information such as the date, time, location, participants, and purpose of the appointment using natural language processing technology, and extract the necessary data.

[0586] Next, the server obtains the user's current location information through location services and searches for the optimal mode of transport and route by comparing it with the planned destination. In this process, it uses public transport operation data and map services to take into account traffic congestion and delays, and creates a real-time travel plan.

[0587] The server then automatically updates the user's schedule based on the planned mode of transport and travel time. It determines a specific departure time and sets an alarm in advance to ensure the user departs on time. The alarm settings provide a more personalized experience by taking into account the user's past travel history and preparation time.

[0588] While on the move, the device provides users with real-time route guidance through visual and audio means. This is done through an application on the mobile device, helping users reach their destination without getting lost. For example, when using public transport, it clearly instructs users on which train or bus to take and where to transfer.

[0589] By using this system, users can automate cumbersome scheduling and travel planning, enabling them to manage their schedules more efficiently. In particular, the ability to obtain real-time traffic information allows for flexible responses to unforeseen circumstances. This reduces daily stress, allowing users to focus on important appointments.

[0590] The following describes the processing flow.

[0591] Step 1:

[0592] The server, with the user's permission, retrieves appointments from the user's calendar via the schedule information service's API. It then analyzes the date, time, location, and event details using natural language processing techniques to identify the schedule information.

[0593] Step 2:

[0594] The server uses location services to determine the user's current location. Next, based on the destination and acquired schedule information, it uses a transportation API to search for the optimal mode of transport and route. In this process, real-time traffic information is taken into consideration when selecting the route.

[0595] Step 3:

[0596] The calendar schedule is updated based on the mode of transport and route selected by the server. The departure time is calculated taking into account the travel time, and an alarm is set that takes into account the time needed for departure preparation, based on the user's past activity history.

[0597] Step 4:

[0598] When the departure time approaches, the device will notify the user with a set alarm. The alarm will alert the user with both sound and vibration to encourage them to prepare for departure on time.

[0599] Step 5:

[0600] When the device detects that it has started moving, it launches a map app or a dedicated app and provides real-time route guidance. The guidance is provided visually and audibly, offering the user the optimal route and transfer information to their destination.

[0601] Step 6:

[0602] If traffic delays or route changes occur during travel, the server updates information in real time and notifies the user of new instructions via their device. The user can then follow these instructions and reach their destination smoothly.

[0603] (Example 1)

[0604] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0605] In modern society, individuals lead busy daily lives, requiring efficient schedule management. However, manually setting appointments, selecting transportation methods, and dealing with traffic delays are time-consuming and cumbersome. Furthermore, changing schedules and obtaining real-time traffic information are also difficult. Therefore, automated schedule management systems are needed.

[0606] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0607] In this invention, the server includes means for acquiring schedule information via a communication device and analyzing the schedule using a generating AI model; means for automatically selecting the optimal means of transportation and route based on current location information and destination information; and means for updating the schedule based on the means of transportation and route and setting an alarm at an appropriate time for departure. This enables efficient schedule management and the automation of rapid travel planning.

[0608] A "communication device" is a device used to send and receive data, and is a device that has the function of acquiring or transmitting various information via a network.

[0609] "Schedule information" refers to detailed data about an individual's schedule, and is a general term for information including date, time, place, purpose, etc.

[0610] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and derive solutions based on user needs.

[0611] "Location information" refers to information that indicates the current geographical location of a device or individual, and is often expressed as coordinate data.

[0612] "Target destination information" refers to information about the place the user intends to travel to, and includes detailed data indicating the geographical location.

[0613] "Means of transportation" refers to the means or method of transport used to get to a destination, such as walking, public transport, or automobile.

[0614] A "route" is the path taken from the starting point to the destination, and is a route selected based on the shortest or most optimal method of travel.

[0615] An "alarm" is a function that provides a notification to the user at a specific time, and is a warning system set before an event begins.

[0616] "Traffic conditions" refers to information indicating the current state of road and transportation operations, including situations such as congestion and delays.

[0617] This invention provides a system that efficiently automates user schedule management. First, a server operates in a cloud environment and acquires schedule information via a communication device. This process utilizes common communication standards for schedule information services to collect schedule information. For example, it is designed so that users do not need to input detailed information such as the date, time, and location of meetings they are scheduled to attend.

[0618] Next, the server uses a generative AI model to analyze the acquired schedule information. At this stage, natural language processing techniques are used to interpret the purpose and priority of the schedule, supporting efficient schedule management. For example, a prompt sentence such as "Attend a meeting in the city tomorrow at 10am" is entered, and the server performs the necessary processing based on this.

[0619] The server also uses location services to collect current location information and compares it with destination information to select the optimal mode of transport and route. In this process, a wide range of options can be considered as means of transport, from public transport to automobiles.

[0620] Furthermore, the system updates the schedule based on the mode of transport and route, and sets alarms at appropriate times to ensure users can start their journey on time. The alarms are personalized based on the user's behavioral characteristics, allowing for efficient preparation for departure.

[0621] While on the move, the device provides users with real-time visual and audio route guidance. This is done through a smartphone application and helps users reach their destination without getting lost. This process of assisting with travel to the destination can clearly convey instructions for using public transportation and making transfers.

[0622] By using this system, users can gain automated schedule management and more efficient travel, reducing everyday stress and allowing them to focus on important matters.

[0623] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0624] Step 1:

[0625] The server retrieves user schedule information via the schedule information service API. It uses the user's account information as input and outputs data such as the date, time, location, and purpose of the appointment. Since this data is provided in several formats, it is converted to a standardized format for easier handling in subsequent processing.

[0626] Step 2:

[0627] The server analyzes the acquired schedule information using a generating AI model. The input is the schedule information standardized in step 1, and the output is an evaluation of the schedule priority and the importance of related tasks. Since natural language processing is performed in this process, it generates prompts that clarify the user's objective and performs specific actions to process them with the AI ​​model.

[0628] Step 3:

[0629] The server obtains the user's current location information from a location services. In this step, the server takes location data provided by the user's mobile device as input and outputs the user's real-time location coordinates. Based on this information, a specific step is taken to search for available public transportation options in real time.

[0630] Step 4:

[0631] The server selects the optimal mode of transport and route based on location and planned destination information. Input is current location and planned destination data, and output is the optimized mode of transport (e.g., train, car) and route. Real-time traffic information, such as congestion and service delays, is also considered in this process.

[0632] Step 5:

[0633] The server automatically updates the schedule and sets an alarm suitable for departure based on the selected mode of transport and route. Inputs include the optimal route information obtained in step 4 and the user's existing schedule. Outputs are the updated schedule information and alarm times, with the alarm time adjusted based on the user's past activity history.

[0634] Step 6:

[0635] The terminal provides users with real-time route guidance through both visual and audio. It uses optimal route information provided by a server as input, and outputs navigation instructions to the user. Specifically, it displays route information visually to the user via a map application and provides voice guidance for the next action.

[0636] (Application Example 1)

[0637] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0638] In modern society, managing daily schedules and the associated travel plans is complex, and there is a need for efficient systems to handle this. In particular, inappropriate traffic information and transportation choices can lead to wasted time and stress. The challenge is to improve the overall quality of life by enabling systems to manage users' schedules and travel more efficiently.

[0639] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0640] In this invention, the server includes means for acquiring and analyzing schedule information, means for automatically selecting the optimal means of transportation and route based on current location information and destination information, and means for updating the schedule based on the means of transportation and route and setting time notifications appropriate for the departure time. This enables users to manage their travel and schedules more efficiently and enjoy a smarter urban lifestyle.

[0641] "Schedule information" refers to information that shows the user's daily schedule and activity plans.

[0642] "Means of analysis" refers to techniques or devices for interpreting schedule information and extracting necessary data.

[0643] "Current location information" refers to geographical data used to identify the user's current location.

[0644] "Destination information" refers to geographical data about the place the user is heading to.

[0645] The "optimal mode of transport" means choosing the most efficient way to travel based on current traffic conditions and schedules.

[0646] A "route" is the path taken to travel from a starting point to a destination.

[0647] "Means for setting time notifications" refers to technologies or devices that notify users at the appropriate time so that they can act according to their schedule.

[0648] "Location data" refers to information about a specific geographical location.

[0649] "Means of providing information visually and aurally" refers to technologies or devices for presenting information to users in a visual form and conveying it audibly.

[0650] This invention is a system designed to streamline schedule management and improve users' daily lives. This system operates primarily on a server running in a cloud environment and utilizes multiple hardware and software components.

[0651] The server utilizes the planning information service's API to acquire and analyze schedule information. Through this API, it collects user schedule information and performs analysis using natural language processing. The software used incorporates an AI model to provide natural language processing capabilities.

[0652] The server also receives location information from the user's terminal and automatically selects the optimal mode of transport and route. This selection utilizes external APIs that provide location services and map data. In particular, it uses public data streams to update traffic information in real time.

[0653] To support users' travel, the server predicts travel time and sends notifications to the device that are appropriate for the departure time. This process incorporates machine learning algorithms that utilize past travel history and usage patterns.

[0654] While on the move, the device provides navigation information through sight and sound. This assumes the user is using a wearable device such as smart glasses. By utilizing the display and speaker of the wearable device, the user can receive guidance through sight and sound.

[0655] By actually using this system, users can enjoy comfortable and efficient urban living through optimized transportation options. For example, when attending a major event, they can be presented with the optimal mode of transportation and route before departure, and receive notifications of departure times.

[0656] An example of a prompt to a generating AI model is: "Based on the user's current location, please suggest the best mode of transportation and route for their planned station visit at 12:00. Please also consider traffic information."

[0657] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0658] Step 1:

[0659] The server retrieves user schedule information using the API of the planning information service. It receives user authentication information as input and retrieves scheduled information as output. The retrieved data is analyzed using natural language processing techniques to extract important keywords and date / time information.

[0660] Step 2:

[0661] The server receives current location information transmitted from the terminal. Based on this information, and combined with destination information, it uses external location services and map data to calculate the optimal mode of transport and route. The input is current location data from the terminal, and the output is information on the mode of transport and route. Traffic conditions and public transport operation data are also taken into consideration.

[0662] Step 3:

[0663] The server updates the user's schedule based on the acquired optimal mode of transport and route information. The inputs here are the travel information obtained in step 2 and the user's schedule effect. The output is a departure time notification adjusted to ensure the user travels on time. The notification timing is optimized using known patterns based on past travel history.

[0664] Step 4:

[0665] The terminal provides the user with real-time visual and auditory guidance. Input consists of route information and current location information sent from the server. Output includes specific actions such as displaying route guidance on the user's smart glasses or other wearable devices. During this process, the guidance information is updated as needed to ensure smooth travel to the destination.

[0666] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0667] This invention is a system that incorporates an emotion recognition engine to further personalize user schedule management and route guidance. In addition to its basic function of automatically analyzing the user's schedule and suggesting the optimal means of transportation and route, this system is capable of dynamic adjustments according to the user's emotional state.

[0668] First, the server uses conventional scheduling information acquisition methods to aggregate users' schedules on the cloud. This allows scheduling information related to date, time, and location to be properly analyzed.

[0669] Next, the emotion engine analyzes the voice and facial expression data acquired by the device from the user. Here, the user's emotional state (e.g., stress, joy, anger) is determined, and further adjustments to the interaction are made based on the results. For example, if it is determined that the user is feeling stressed, the server will change the suggested mode of transportation to a more comfortable option or adjust the voice guidance for the travel route to be gentler.

[0670] Furthermore, the server can update traffic information in real time and dynamically adjust schedules and alarms according to the user's mood. If the user is relaxed, a schedule with ample time will be displayed, while if they are in a hurry, the system will prioritize faster travel plans.

[0671] For example, if a user is feeling stressed during their morning commute, the device can use its emotional engine to reduce that stress by playing relaxing music or suggesting a more comfortable route.

[0672] As a result, the present invention can provide a more flexible and personalized experience that takes into account the individual emotional needs of each user, compared to conventional schedule management systems. By implementing this system, users can significantly reduce stress related to scheduling and travel in their daily lives.

[0673] The following describes the processing flow.

[0674] Step 1:

[0675] Users input speech and facial expression data into the system via smartphones or wearable devices. By granting permission to use the emotion engine during initial setup, the device is prepared to acquire emotion data.

[0676] Step 2:

[0677] The device runs an emotion recognition algorithm that analyzes the user's voice and facial expressions to determine the user's emotional state. This algorithm analyzes the user's speech patterns and facial features in real time to detect emotions such as stress, relief, anger, and joy.

[0678] Step 3:

[0679] The server uses the schedule information service's API to retrieve the user's schedule information from the cloud. The retrieved data is analyzed, and the date, time, location, purpose, and other details of the appointment are extracted.

[0680] Step 4:

[0681] The server acquires location information and selects the optimal mode of transport and route based on destination information. During this process, it incorporates the results of an emotion engine, adjusting the mode of transport and route according to the user's emotional state. For example, if the user is feeling stressed, less congested routes and places to avoid will be prioritized.

[0682] Step 5:

[0683] The server collects traffic information in real time and creates a travel schedule that takes emotions into account. Next, it calculates departure times and automatically sets alarms. During this process, the alarm sound can also be adjusted according to emotions.

[0684] Step 6:

[0685] The device notifies the user at the time the alarm it has set will sound. At this point, it prompts the user to begin preparations and guides them through the process to ensure a comfortable journey.

[0686] Step 7:

[0687] While on the move, the device provides the user with visual and audio route guidance. An emotion engine continuously monitors the user's emotions and adjusts the tempo and tone of the guidance accordingly. For example, if the user is relaxed, the guidance can be made more cheerful.

[0688] Step 8:

[0689] Servers and terminals regularly update information while in transit, allowing for immediate responses to unexpected traffic disruptions or schedule changes. Route and schedule readjustments based on emotional changes are also made during this step.

[0690] (Example 2)

[0691] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0692] Conventional schedule management and route guidance systems suggest modes of transport and routes based on predetermined criteria without considering the individual emotional state of the user. As a result, they fail to provide appropriate suggestions when the user is stressed or in a hurry, leading to decreased user satisfaction. The present invention aims to dynamically adjust modes of transport and routes according to the user's emotional state, providing a comfortable and personalized experience for the user.

[0693] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0694] In this invention, the server includes means for acquiring and analyzing schedule information, means for automatically selecting the optimal means of transportation and route based on current location information and destination information, and means for analyzing the user's emotional state using emotion recognition technology and dynamically adjusting the means of transportation and route based on the analysis results. This makes it possible to suggest the optimal transportation according to the user's emotional state.

[0695] "Schedule information" refers to detailed data such as the date, time, location, and content of appointments and events set by the user.

[0696] "Means of analysis" refers to devices or software that analyze acquired data and perform computational processing or judgments to give it meaning.

[0697] "Current location information" refers to digital data that indicates the user's real-time location, and includes latitude, longitude, and altitude.

[0698] "Destination information" refers to location data of the place the user is heading to, including addresses and place names.

[0699] "Optimal mode of transportation" refers to the method of travel that is deemed most efficient and convenient based on the user's conditions and environment.

[0700] "Means for automatically selecting a route" refers to a function that automatically determines the route to reach the user's destination using algorithms or software.

[0701] "Emotion recognition technology" refers to technology that analyzes a person's voice, facial expressions, and behavior to identify their emotional state at that time.

[0702] "Traffic information" refers to data that affects travel, such as the operating status of public transportation and road congestion.

[0703] "Means of providing route guidance visually and audibly" refers to technologies that provide users with a route to their destination through visual map displays and voice instructions.

[0704] This invention is a system that incorporates emotion recognition technology to highly personalize user schedule management and navigation guidance. Specific embodiments are described below.

[0705] The server uses external connections to manage and analyze schedule information in the cloud. Specifically, it retrieves information including date, time, location, and event details via APIs of publicly available schedule information services. This allows for a proper understanding and management of users' planned activities.

[0706] The device uses the smartphone's built-in microphone and camera to acquire the user's voice and facial expression data. This device is equipped with an emotion recognition engine and utilizes Microsoft Azure Cognitive Services or similar services to analyze the user's emotional state. For example, the device can determine whether the user is stressed or relaxed.

[0707] The server dynamically adjusts the optimal mode of transport and route based on analyzed emotion data and schedule information. Specifically, it uses a map service API to analyze traffic flow in order to reflect real-time traffic information. It also flexibly selects the mode of transport to present according to the user's emotional state. When the user is relaxed, it recommends a leisurely mode of transport, while when they are in a hurry, it suggests the fastest route.

[0708] Users can view and select suggested schedules and modes of transportation through their device, and the system provides services based on those selections. For example, if stress levels are measured during the morning commute, the device will play relaxation music, and the server will suggest a comfortable and efficient commute route. Another example of a prompt using a generative AI model is, "Please suggest a suitable restaurant for lunch when the user is in a relaxed state." This prompt allows the AI ​​to generate suggestions appropriate to the situation.

[0709] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0710] Step 1:

[0711] The server obtains schedule information via an external connection. It uses user registration information as input. The retrieved data includes date, time, location, and event type. This data is analyzed to create the user's schedule. The output is the analyzed schedule information.

[0712] Step 2:

[0713] The device collects the user's voice and facial expression data using a microphone and camera. The voice and facial expressions are recorded in real time and input into an emotion recognition engine. The engine analyzes this data to identify the user's emotional state (e.g., stress, relaxation). The output is data on the identified emotional state.

[0714] Step 3:

[0715] The server receives emotional state data and schedule information, and dynamically adjusts the mode of transport and route according to the user's state. Real-time traffic information is also included as input. As data processing, the optimal route is calculated using a map service API. The output is the adjusted mode of transport and route plan.

[0716] Step 4:

[0717] The terminal notifies the user of the travel plan sent from the server. It provides visual map displays and voice guidance, and plays relaxation music as needed. It uses user emotions and travel plan data as input. The output is user interaction and guidance.

[0718] Step 5:

[0719] The user selects a suggested mode of transport and route based on information provided from their device and sends feedback to the server. This feedback is used for future suggestions. The provided travel plan is used as input. The output is the user's selection and feedback data.

[0720] (Application Example 2)

[0721] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0722] Traditional scheduling management systems often fail to adequately address individual user needs because they suggest schedules and routes without considering the user's emotional state. In particular, certain emotional states during travel can cause stress and discomfort, potentially impacting the execution of the schedule. In such situations, a more personalized system is needed.

[0723] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0724] In this invention, the server includes means for acquiring and analyzing schedule information, means for automatically selecting the optimal means of transportation and route based on current location information and destination information, and means for analyzing the user's emotional state and personalizing the means of transportation and route. This makes it possible to suggest schedules and routes based on the user's emotional state.

[0725] "Schedule information" refers to data about a user's plans and schedule, including information related to the date, time, and location.

[0726] "Current location information" refers to data about the geographical location where the user is currently situated.

[0727] "Destination information" refers to data about the geographical location that the user wants to reach.

[0728] "Means of transportation" refers to the means used by a user to move from one place to another, and includes public transport, walking, cycling, and driving.

[0729] A "route" is the path taken from a point of origin to a destination, and is a set of paths based on a specific mode of transportation.

[0730] "Emotional state" refers to the emotional state a user is experiencing, and includes states such as stress, joy, and anger.

[0731] "Personalizing" means adjusting or customizing something to suit the individual user's characteristics and circumstances.

[0732] This invention provides a system using an emotion recognition engine to personalize user schedule management and route guidance. In this system, a server aggregates schedule information on the cloud and analyzes the user's schedule. A terminal acquires the user's voice and facial expression data and uses the emotion recognition engine to analyze their emotional state. For example, machine learning models using TensorFlow or facial recognition technology using OpenCV may be applied.

[0733] The server dynamically adjusts the user's travel route and mode of transport based on acquired emotional information. Specifically, it uses the Google Maps API to update traffic information in real time and suggests the optimal route and mode of transport according to the user's emotional state. The device provides the user with the suggested information and promotes a calm travel experience by playing relaxing music when the user is stressed.

[0734] This system uses the smartphone's camera and microphone to collect user emotion data and communicates it with a server via the internet. Cloud-based data processing allows users to receive seamless and immediate support.

[0735] For example, for users who feel stressed during the morning rush hour commute, the device can suggest routes with pleasant scenery and play calming music, making the start of their day more comfortable.

[0736] An example of a prompt for a generative AI model is, "How can I suggest the best route for a user who is feeling stressed during their morning commute?"

[0737] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0738] Step 1:

[0739] The device sends the user's schedule information to a server in the cloud. It uses data obtained from the user's calendar or schedule management app's API as input, and outputs it to the server as schedule information.

[0740] Step 2:

[0741] The server analyzes the received schedule information and organizes the user's appointments in chronological order. Here, it analyzes the schedule data to create a list of appointments and returns the analyzed schedule as output to the terminal.

[0742] Step 3:

[0743] The device acquires voice and facial expression data from the user and analyzes their emotional state using an emotion recognition engine. It uses data from the camera and microphone as input and sends the identified emotional state (e.g., stress, joy, neutral) to the server as output.

[0744] Step 4:

[0745] The server adjusts the optimal mode of transportation and route based on emotional state data. It incorporates real-time traffic information obtained using the Google Maps API, generates travel suggestions tailored to the user's emotions, and sends them to the device as output.

[0746] Step 5:

[0747] The terminal visually and audibly notifies the user of suggested route information and means of transportation received from the server. The input is route guidance information, and the output is route guidance to the user. If stress is detected, the terminal also plays relaxing music.

[0748] Step 6:

[0749] The user receives route guidance from the device and follows the instructions to travel. The journey is completed based on optimized information provided by the device. The result is a more comfortable travel experience.

[0750] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0751] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0752] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0753] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0754] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0755] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0756] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0757] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0758] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0759] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0760] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0761] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0762] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0763] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0764] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0765] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0766] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0767] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0768] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0769] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0770] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0771] The following is further disclosed regarding the embodiments described above.

[0772] (Claim 1)

[0773] A means of obtaining and analyzing schedule information,

[0774] A means for automatically selecting the optimal mode of transport and route based on current location information and destination information,

[0775] A means of updating the schedule based on the mode of transport and route, and setting an alarm appropriate for the departure time,

[0776] A means of acquiring and updating traffic information in real time,

[0777] Means of providing visual and audio route guidance while traveling to a destination,

[0778] A system that includes this.

[0779] (Claim 2)

[0780] The system according to claim 1, characterized in that the means for obtaining schedule information obtains the information via an API of a schedule information service.

[0781] (Claim 3)

[0782] The system according to claim 1, characterized in that it uses current location information to predict travel time.

[0783] "Example 1"

[0784] (Claim 1)

[0785] A means of acquiring schedule information via a communication device and analyzing the schedule using a generated AI model,

[0786] A means for automatically selecting the optimal mode of transportation and route based on current location information and target location information,

[0787] A means of updating the schedule based on the mode of transport and route, and setting an alarm for the appropriate time to depart,

[0788] A means of acquiring and updating traffic conditions in real time,

[0789] Means of providing route guidance visually and audibly while traveling,

[0790] A system that includes this.

[0791] (Claim 2)

[0792] The system according to claim 1, characterized in that the means for acquiring scheduled information acquires the information via a communication standard for information services.

[0793] (Claim 3)

[0794] The system according to claim 1, characterized in that it calculates the estimated time of travel using the current location information.

[0795] "Application Example 1"

[0796] (Claim 1)

[0797] A means of obtaining and analyzing schedule information,

[0798] A means for automatically selecting the optimal mode of transport and route based on current location information and destination information,

[0799] A means of updating the schedule based on the mode of transport and route, and setting time notifications appropriate for the departure time,

[0800] A means of using location data to improve the overall efficiency within an environmental system and proposing means of transportation,

[0801] Means for providing route guidance visually and aurally,

[0802] A system that includes this.

[0803] (Claim 2)

[0804] The system according to claim 1, characterized in that the means for obtaining schedule information obtains the information via an API of a planning information service.

[0805] (Claim 3)

[0806] The system according to claim 1, characterized in that it uses current location information to predict travel time.

[0807] "Example 2 of combining an emotion engine"

[0808] (Claim 1)

[0809] A means of obtaining and analyzing schedule information,

[0810] A means for automatically selecting the optimal mode of transport and route based on current location information and destination information,

[0811] A means of updating the schedule based on the mode of transport and route, and setting an alarm appropriate for the departure time,

[0812] A means for analyzing the user's emotional state using emotion recognition technology and dynamically adjusting the means of transportation and route based on the analysis results,

[0813] A means of acquiring and updating traffic information in real time,

[0814] Means of providing visual and audio route guidance while traveling to a destination,

[0815] A system that includes this.

[0816] (Claim 2)

[0817] The system according to claim 1, characterized in that the means for acquiring schedule information acquires the information via an external connection to a schedule information service.

[0818] (Claim 3)

[0819] The system according to claim 1, characterized in that it proposes a comfortable means of transportation for the user through emotion recognition technology.

[0820] "Application example 2 when combining with an emotional engine"

[0821] (Claim 1)

[0822] A means of obtaining and analyzing schedule information,

[0823] A means for automatically selecting the optimal mode of transport and route based on current location information and destination information,

[0824] A means of updating the schedule based on the mode of transport and route, and setting an alarm appropriate for the departure time,

[0825] A means of acquiring and updating traffic information in real time,

[0826] Means of providing visual and audio route guidance while traveling to a destination,

[0827] A means of analyzing the user's emotional state and personalizing the means of transportation and route,

[0828] A system that includes this.

[0829] (Claim 2)

[0830] The system according to claim 1, characterized in that the means for obtaining schedule information obtains the information via an API of a schedule information service.

[0831] (Claim 3)

[0832] The system according to claim 1, characterized in that it uses current location information to predict travel time. [Explanation of Symbols]

[0833] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of obtaining and analyzing schedule information, A means for automatically selecting the optimal mode of transport and route based on current location information and destination information, A means of updating the schedule based on the mode of transport and route, and setting an alarm appropriate for the departure time, A means of acquiring and updating traffic information in real time, Means of providing visual and audio route guidance while traveling to a destination, A system that includes this.

2. The system according to claim 1, characterized in that the means for obtaining schedule information obtains the information via an API of a schedule information service.

3. The system according to claim 1, characterized in that it uses current location information to predict travel time.

Images