System

Abstract

A system is provided.SOLUTION: The system includes a means for acquiring schedule information of a user, a means for acquiring current location information, a means for proposing a plurality of moving means and required time to a next scheduled point based on the schedule information and the current location information, a means for adding a moving time to the schedule based on the moving means selected by the user, and a means for setting an alarm based on the set moving time.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional schedule management systems require users to manually calculate travel times, add them to their schedules, and set alarms, which can be time-consuming and prone to errors. Furthermore, they are unable to respond to real-time operational information or battery status, making it difficult to take efficient action. The present invention aims to solve these problems by providing a system that allows users to effortlessly manage their schedules and take efficient action. [Means for solving the problem]

[0005] The present invention provides a system including a means for acquiring a user's schedule information, a means for acquiring current location information, a means for proposing multiple means of transportation and travel times to the next scheduled location based on the schedule information and current location information, a means for adding travel time to the schedule based on the transportation method selected by the user, and a means for setting an alarm based on the set travel time. The system also includes a means for acquiring operation information in real time, updating the schedule and alarm, and a means for monitoring the remaining battery level and sending a notification prompting charging when the battery falls below a predetermined threshold. This eliminates the need for manual operation by the user, avoiding mistakes and reducing effort, enabling efficient schedule management and behavior.

[0006] "User" refers to a person who uses the system, an entity that uses the system's functions to effectively manage their life.

[0007] "Schedule information" is data including the user's plans and events, and is information obtained from a calendar application or a schedule application.

[0008] "Current location information" is data indicating the user's current location, and is location information obtained in real time from a GPS sensor or the like.

[0009] "Transportation" refers to the means of transportation used by the user to travel to the planned location, and includes trains, buses, cars, bicycles, walking, and the like.

[0010] "Travel time" is data indicating the time it takes to travel from the current location to the destination using a specific means of transportation.

[0011] "Means" refers to methods, devices, and various modules within a system used to realize a specific function or process.

[0012] An "alarm" is a function that notifies or warns the user at a specific time, and notifies the user at the set time by sound, vibration, screen display, or the like.

[0013] "Real-time information" refers to the latest information at the current time, including operation information, traffic conditions, weather, etc.

[0014] "Remaining battery capacity" is data indicating the power supply status of the user's terminal, and indicates the current battery charge state.

[0015] A "notification" is a message or alert that the system uses to convey information to the user, and is provided in the form of a screen display, sound, vibration, or the like.

[0016] A "threshold" refers to a specific reference value, which is a set value that, when exceeded, causes a specific action or notification to be taken. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a smartphone schedule management system that uses AI to acquire user schedule information, automatically calculate travel time, add travel time to the schedule, set alarms, and update real-time information. The specific configuration of each part and operation of this system is explained below.

[0039] composition

[0040] 1. Get user schedule

[0041] The device obtains event data from the user's calendar application or schedule application via an API. This system is compatible with Google Calendar and other calendar services, for example.

[0042] 2. Travel time and route calculation

[0043] The device acquires the user's current location using the GPS sensor, then extracts the next scheduled location from the schedule and identifies its location information.

[0044] Based on this, the server accesses the API of the map service (for example, Google Maps API) and calculates the optimal means of transportation (train, bus, car, walking, etc.) from the current location to the next destination and the required time for each. After the server obtains this information, it proposes multiple means of transportation and their required times to the user.

[0045] 3. Add travel time to your schedule

[0046] When the user selects their preferred mode of transportation, the device adds the travel time to the schedule. For example, if the user's current location is Chiyoda Ward, Tokyo, and their next destination is Shinjuku Ward, and they select the train, the calculated travel time of 30 minutes will be reflected in the schedule.

[0047] 4. Set an alarm

[0048] The device will set an alarm 10 minutes before the scheduled departure time, so if the departure is scheduled for 2:30 PM, the alarm will be set for 2:20 PM.

[0049] 5. Real-time information

[0050] The device periodically checks real-time train and bus information to obtain delay information. Based on this information, it automatically updates schedules and alarm times as needed. For example, if a train is delayed by 10 minutes, the device will advance the departure time and reset the alarm.

[0051] 6. Reminder notifications

[0052] The device notifies the user of travel and preparation reminders. For example, at 2:10 p.m., it sends a notification saying, "10 minutes until departure. Please start getting ready." The device also sends a notification urging the user to charge the battery if the battery level is low. 10 minutes before the start of a meeting, it also sends a notification saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0053] Specific examples

[0054] For example, suppose a user has a meeting scheduled for 3:00 PM in Shinjuku Ward. The user's current location is Chiyoda Ward, and the schedule is retrieved from a calendar application. The device then identifies the user's current location using GPS, and the server uses a map service API to calculate the optimal route and travel time from Chiyoda Ward to Shinjuku Ward.

[0055] If the user chooses to travel by train, the device calculates the travel time as 30 minutes and adds a departure time of 2:30 p.m. to the schedule. It also sets an alarm for 2:20 p.m. If the train is delayed, the device checks real-time information, automatically updates the schedule and alarm, and notifies the user.

[0056] In this way, the system of the present invention allows users to efficiently manage their schedules without hassle, helping to avoid mistakes and reducing stress.

[0057] The processing flow will be explained below.

[0058] Step 1:

[0059] The device retrieves event data from the user's calendar or schedule application via an API. For example, it sends a request to the calendar API to retrieve a schedule for a meeting starting at 3:00 PM on October 15, 2023.

[0060] Step 2:

[0061] The device acquires the user's current location information using a GPS sensor. For example, the device identifies that the user's current location is Chiyoda Ward, Tokyo.

[0062] Step 3:

[0063] The device identifies the next scheduled location from the schedule data and extracts its location information. For example, it recognizes that the next scheduled location is a "conference room in Shinjuku Ward, Tokyo."

[0064] Step 4:

[0065] The server accesses the API of a map service (e.g., Google Maps API) based on the current location and the next planned destination, and calculates the optimal means of transportation and the required time for each. For example, it obtains information such as 30 minutes by train, 45 minutes by bus, and 50 minutes by bicycle.

[0066] Step 5:

[0067] The terminal presents the acquired information on the means of transportation and the required time to the user, and the user selects the desired means. For example, the user selects "train."

[0068] Step 6:

[0069] The device will then add the travel time to the schedule based on the selected mode of transportation, updating the schedule to, for example, "Depart at 2:30 PM."

[0070] Step 7:

[0071] The device will set an alarm 10 minutes before departure time, for example, "Set alarm for 2:20 p.m."

[0072] Step 8:

[0073] The device periodically retrieves real-time data, including operational information, and checks train and bus delay information. For example, it retrieves information that a train is 10 minutes late.

[0074] Step 9:

[0075] The server automatically updates schedules and alarms based on the acquired operation information. For example, it may move the departure time forward to "2:15 PM" and reset the alarm to "2:05 PM."

[0076] Step 10:

[0077] The device sends a reminder to the user to prepare for travel. For example, at 2:10 p.m., the device sends a notification saying, "There are 10 minutes until departure. Please start preparing."

[0078] Step 11:

[0079] The device checks the battery status and sends a notification to encourage charging if the remaining charge falls below a certain threshold. For example, "I detected that the battery charge is less than 20% and sent a notification saying 'Please charge.'"

[0080] Step 12:

[0081] The device will remind the user to set the silent mode 10 minutes before an important event (e.g., a meeting) starts. For example, a notification will be sent 10 minutes before the start of the meeting saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0082] Through these steps, users can efficiently and accurately manage their schedules, enabling smooth travel and preparation without the need for manual operations.

[0083] Example 1

[0084] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0085] Conventional schedule management systems require users to manually check current location information and transportation options and reflect them in their schedules. This makes schedule management cumbersome and makes it difficult to efficiently complete plans. Furthermore, they often lack the ability to provide real-time traffic information or appropriate alerts or notifications based on remaining battery power, increasing the risk of users missing important appointments.

[0086] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0087] In this invention, the server includes means for acquiring the user's schedule information, means for acquiring location information, means for proposing multiple means of transportation and travel times to the next scheduled location based on the schedule information and location information, means for adding travel time to the schedule based on the transportation means selected by the user, means for setting an alarm based on the set departure time, means for acquiring traffic operation information in real time and updating the schedule and alarm, and means for monitoring the remaining battery level and sending a notification urging the user to charge the battery when the battery falls below a predetermined threshold. This allows the user to efficiently manage their schedule without having to perform complicated manual operations, significantly reducing the risk of missing important appointments.

[0088] "User schedule information" is data relating to the user's schedule, obtained from a calendar application or a schedule application.

[0089] "Location Information" means data about a user's current latitude and longitude obtained using a GPS sensor or other positioning technology.

[0090] "Transportation" refers to the means used for travel, including trains, buses, cars, and walking.

[0091] "Travel time" is the estimated time it will take to travel from the current location to the next scheduled location.

[0092] "Real-time transportation operation information" is data that obtains the operation status of transportation in real time and reflects information such as delays and cancellations.

[0093] "Battery level monitoring" is a feature that measures the remaining battery level of smartphones and other mobile devices and notifies you when it reaches a certain level.

[0094] "Add to schedule" means to reflect new schedules or information in the user's schedule.

[0095] "Set an alarm" is a function that notifies the user when a specific time or condition is met.

[0096] The "notification to encourage charging" is a notification to encourage the user to charge when the remaining battery power is low.

[0097] This invention is a smartphone schedule management system that uses AI, which acquires the user's schedule information, suggests transportation methods along with location information, adds travel time to the schedule, sets alarms, and automatically updates real-time information. The specific configuration of each part and operation of this system is explained below.

[0098] Get user's appointment information

[0099] The device accesses a calendar application (e.g., a general-purpose calendar app) based on the user's authentication information and obtains the user's event information through the API. Once this operation is complete, the obtained event data is saved in the device's database. Specifically, the device communicates with the calendar API using the user's OAuth token, receives the obtained data in JSON format, and stores it in the local database.

[0100] Get the user's current location and identify the next scheduled location

[0101] The device activates the GPS sensor to obtain the user's current location. Then, the device extracts the location information of the next scheduled event from the most recent scheduled event and determines the latitude and longitude. For example, if the current location is Chiyoda Ward, Tokyo, and the next scheduled event location is Shinjuku Ward, the device will obtain the current location using GPS and the location information of the next scheduled event through a geocoding service.

[0102] Calculate travel route and travel time

[0103] The device sends this location information to the server. The server calls a map service API (for example, a general-purpose map service API) to calculate the optimal means of transportation from the current location to the next scheduled destination and the required travel time. The calculation results are returned to the device in JSON format. For example, the device sends a request to the map service API through the server to calculate the required travel time for various modes of transportation, such as train, bus, car, and walking.

[0104] Choosing transportation methods and reflecting them in your schedule

[0105] The device displays options for transportation and travel times to the user. When the user selects their preferred transportation method, the device reflects that travel time in the schedule. For example, if the user selects traveling by train, the device calculates the travel time as 30 minutes and updates the schedule to depart at 2:30 PM. The device then sets the alarm time to 2:20 PM based on this information.

[0106] Reflecting real-time information

[0107] The device obtains real-time traffic information at regular intervals and updates schedules and alarms as necessary. Specifically, the device uses a traffic information API to check real-time traffic information, and if there is delay information, it resets the departure time or alarm and notifies the user.

[0108] Reminders and notifications

[0109] The device sends users reminders to travel and prepare. It also notifies users when the battery is low, urging them to charge the device. For example, at 2:10 p.m., it sends a notification 10 minutes before departure, and 10 minutes before a meeting starts, it sends a notification saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0110] Specific examples

[0111] For example, consider a case where a user has a meeting scheduled in Shinjuku Ward at 3:00 PM. The user's current location is Chiyoda Ward, and this appointment is retrieved from the configured calendar application. The device then uses GPS to determine the user's current location, and the server uses a general-purpose map service API to calculate the optimal route and travel time from Chiyoda Ward to Shinjuku Ward.

[0112] If the user chooses to travel by train, the device calculates the travel time as 30 minutes, updates the schedule to depart at 2:30 p.m., and sets an alarm for 2:20 p.m. If the train is delayed, the device checks real-time information, automatically updates the schedule and alarm, and notifies the user.

[0113] In this way, the system of the present invention allows users to efficiently manage their schedules without having to perform complicated manual operations, significantly reducing the risk of missing important appointments.

[0114] Example prompt sentence:

[0115] "I have a meeting in Shinjuku Ward at 3 PM. My current location is in Chiyoda Ward. Please calculate and suggest the best transportation method and travel time, and update my schedule accordingly. Also, please set an alarm 10 minutes before departure time, and update my schedule as needed based on real-time traffic information."

[0116] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0117] Step 1:

[0118] The device accesses a calendar API (for example, a general-purpose calendar API) based on the user's authentication information and obtains the user's schedule information. Specifically, it makes an API call using an OAuth token and receives the obtained schedule information in JSON format. User authentication information is required as input, and event data is obtained as output. This data is saved in a database within the device.

[0119] Step 2:

[0120] The device activates the GPS sensor and obtains the user's current location information. Specifically, it uses the GPS module to measure the current latitude and longitude. The input requires location measurement using the GPS module, and the output is the latitude and longitude data of the current location.

[0121] Step 3:

[0122] The device extracts the location information of the next scheduled event from the acquired schedule information. Specifically, it parses the address information from the next scheduled event and converts it to latitude and longitude using a geocoding API. The address of the next scheduled event is required as input, and the latitude and longitude data of the scheduled location is obtained as output.

[0123] Step 4:

[0124] The device sends latitude and longitude information of the current location and the next planned location to the server. The server calls a map service API (for example, a general-purpose map service API) and calculates the multiple means of transportation from the current location to the next planned location and the required travel time. Specifically, it uses the Directions API to obtain the route and time for each means. The latitude and longitude of the current location and the planned location are required as input, and the required travel time and route information for each means of transportation are obtained as output. The calculation results are returned to the device in JSON format.

[0125] Step 5:

[0126] The device presents the user with multiple transportation options and their respective travel times. When the user selects their preferred transportation option, the device reflects the selected travel time in the schedule. Specifically, the device displays a list of options through a user interface and receives the user's selection. The inputs are the travel times for each transportation option and the user's selection, and the output is an updated schedule.

[0127] Step 6:

[0128] The terminal sets an alarm 10 minutes before the scheduled departure time. Specifically, the alarm is set using the system's alarm manager. The departure time data is required as input, and the alarm setting is output.

[0129] Step 7:

[0130] The device periodically retrieves real-time traffic information and updates schedules and alarms. Specifically, it uses a traffic information API to check the latest traffic conditions and reset schedules and alarms as needed. Real-time traffic data is required as input, and updated schedules and alarms are obtained as output.

[0131] Step 8:

[0132] The device sends travel and preparation reminders to the user. Specifically, it uses push notifications to provide pre-departure reminders and low battery notifications. The inputs required are the timing of scheduled events and battery level data, and the output is the reminder notification sent to the user.

[0133] (Application example 1)

[0134] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0135] The purpose of this invention is to efficiently manage delivery staff schedules and optimize their travel in food delivery services. When there are multiple delivery destinations, conventional methods make it cumbersome to determine routes and manage travel times, and it is difficult to quickly respond to changes in traffic conditions. Therefore, there is a need for a system that can improve delivery efficiency and accuracy.

[0136] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0137] In this invention, the server includes means for acquiring schedule information of the user, means for acquiring current location information, means for proposing multiple means of transportation and required travel times based on the schedule information and current location information, means for adding travel time to the schedule based on the means of transportation selected by the user, means for setting an alarm based on the set travel time, and means for identifying the current location of the delivery person, optimizing the route to the next delivery destination, and updating the delivery schedule in real time, thereby enabling real-time updating of the delivery schedule and efficient route proposals.

[0138] The "means for acquiring user schedule information" is a function that enables an electronic device to automatically collect event data from a user's calendar or schedule.

[0139] "Means for obtaining current location information" means a function for determining the current geographic location of a user or device using GPS or other location information systems.

[0140] The "means for proposing multiple means of transportation and required travel times" is a function for calculating options for transportation (e.g., walking, car, train, etc.) and the required travel times for each, and presenting them to the user.

[0141] The "means for adding travel time to the schedule" is a function for reflecting the travel time calculated based on the means of transportation selected by the user in the schedule.

[0142] The "means for setting an alarm" is a function for sending a notification to the user at a specified time, and serves to prompt the user to leave or get ready.

[0143] "Means for determining the delivery person's current location" refers to a function that uses GPS or a location information system to determine the delivery person's current geographic location in real time.

[0144] The "means for optimizing the route to the next delivery destination" is a function for calculating and presenting the optimal and shortest route to the delivery destination.

[0145] "Means for updating delivery schedules in real time" refers to a function for updating delivery schedules at any time based on current traffic conditions and delivery progress.

[0146] "Means for obtaining real-time operation information and updating schedules and alarms" refers to a function for obtaining traffic conditions and operation information in real time and adjusting schedule and alarm settings based on this information.

[0147] "Means for monitoring the remaining battery level and sending a notification urging the user to charge when the battery falls below a specified threshold" is a function for constantly monitoring the battery status of the device and notifying the user to charge when the remaining battery level falls below a certain level.

[0148] The present invention provides a system for efficiently managing delivery staff schedules and optimizing their travel in a food delivery service. The system includes a means for acquiring a user's schedule information, a means for acquiring current location information, a means for proposing multiple means of transportation and required travel times, a means for adding travel time to the schedule, a means for setting an alarm, a means for identifying the delivery staff's current location, a means for optimizing the route to the next delivery destination, and a means for updating the delivery schedule in real time.

[0149] Configuration and operation

[0150] 1. Obtaining user schedule information

[0151] The server receives event data from the delivery person's schedule management application via an API. This system is compatible with general calendar services, for example.

[0152] 2. Obtaining current location information

[0153] The device uses a GPS sensor to acquire the delivery person's current location, and then sends the delivery person's current location information to the server.

[0154] 3. Proposing multiple transportation methods and travel times

[0155] The server calculates multiple means of transportation (e.g., car, bicycle, walking, etc.) to the next delivery destination and the required time for each, based on the acquired schedule information and current location information. The server uses a map service API (e.g., map service API) to suggest multiple means of transportation and their required time to the delivery person.

[0156] 4. Add travel time to your schedule

[0157] When a delivery person selects their preferred mode of transportation, the server adds the travel time to the schedule. For example, if the current location is somewhere and the next delivery destination is somewhere else, and the delivery person selects a car, the calculated travel time is reflected in the schedule.

[0158] 5. Set an alarm

[0159] The server sets an alarm 10 minutes before the scheduled departure time, allowing the delivery person to prepare for the next delivery.

[0160] 6. Real-time delivery schedule updates

[0161] The device periodically checks real-time traffic information to obtain traffic conditions and delay information. The server automatically updates schedules and alarm times as needed based on the information obtained. For example, if there is traffic congestion, the server will adjust departure times and reset alarms.

[0162] Hardware and software used

[0163] This system uses smart glasses equipped with a GPS sensor as the terminal, and the server calculates routes and retrieves schedule information using the Google Maps API and general calendar APIs.

[0164] Specific examples

[0165] For example, if a delivery person has multiple delivery destinations, the system calculates the travel time from the current location to each destination in real time and suggests an efficient route. The schedule is updated based on the transportation method selected by the delivery person, and an alarm is set for the departure time. Traffic conditions and delay information are obtained in real time, and the schedule and alarms are automatically updated as needed.

[0166] Prompt Sentence Examples

[0167] Here is an example prompt:

[0168] Locate your delivery drivers with GPS and calculate the best route and travel time to their next delivery. Update schedules in real time and set alarms based on their next due time. Also, monitor traffic conditions and reschedule if delays occur.

[0169] This allows the system of the present invention to support delivery personnel in efficient delivery and improve the reliability and accuracy of delivery services.

[0170] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0171] Step 1:

[0172] The server retrieves schedule information from the delivery driver's schedule management application via an API. The inputs include the delivery driver's account information and API key. The output is the schedule information in JSON format or other structured data format. The server parses this data to obtain the delivery order and location information.

[0173] Step 2:

[0174] The terminal acquires the delivery person's current location information using a GPS sensor. The input is a signal obtained from a GPS device, and the output is real-time latitude and longitude data. The terminal then transmits the acquired geographic information to a server.

[0175] Step 3:

[0176] The server proposes multiple means of transportation to the next delivery destination and the required travel time based on the acquired schedule information and current location information. This step requires the latitude and longitude of the current location and schedule information as input, and the required travel time for each means of transportation (e.g., car, bicycle, walking) as output. The server uses a map service API to calculate the optimal route and means of transportation.

[0177] Step 4:

[0178] The user selects the desired transportation method from the presented options. The input is the user's selection information, and the output is the data of the selected transportation method. The server recalculates the required travel time based on the selected transportation method and adds the travel time to the next delivery destination to the schedule.

[0179] Step 5:

[0180] The server sets an alarm based on the set travel time. The input is the updated schedule information and travel time, and the output is the alarm setting data. Based on this, the server sets a notification to be sent before the departure time.

[0181] Step 6:

[0182] The terminal periodically checks real-time traffic information and obtains traffic conditions and delay information. As input, it sends a request to the traffic information API, and as output, it obtains the latest traffic information. The terminal then sends this to the server.

[0183] Step 7:

[0184] The server updates the schedule based on the acquired real-time operation information. This step requires the delivery schedule and real-time operation information as input, and obtains the updated schedule data as output. The server also resets the alarm time if necessary and sends a notification to the user.

[0185] This allows delivery personnel to efficiently manage their delivery schedules and respond quickly to changes in traffic conditions.

[0186] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0187] This invention is a smartphone schedule management system that uses AI. It acquires the user's schedule information, calculates travel time, adds travel time to the schedule, sets alarms, and automatically updates real-time information. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to flexibly suggest schedules and provide notifications according to the user's emotional state. The specific components and operations of this system are described below.

[0188] composition

[0189] 1. Get user schedule

[0190] The device retrieves event data from the user's calendar or schedule application via an API. This system is compatible with Google Calendar and other calendar services, for example.

[0191] 2. Travel time and route calculation

[0192] The device acquires the user's current location using the GPS sensor, then extracts the next scheduled location from the schedule and identifies its location information.

[0193] Based on this, the server accesses the API of the map service (for example, Google Maps API) and calculates the optimal means of transportation (train, bus, car, walking, etc.) from the current location to the next destination and the required time for each. After the server obtains this information, it proposes multiple means of transportation and their required times to the user.

[0194] 3. Add travel time to your schedule

[0195] When the user selects their preferred mode of transportation, the device adds the travel time to the schedule. For example, if the user's current location is Chiyoda Ward, Tokyo, and their next destination is Shinjuku Ward, and they select the train, the calculated travel time of 30 minutes will be reflected in the schedule.

[0196] 4. Set an alarm

[0197] The device will set an alarm 10 minutes before the scheduled departure time, so if the departure is scheduled for 2:30 PM, the alarm will be set for 2:20 PM.

[0198] 5. Real-time information

[0199] The device periodically checks real-time train and bus information to obtain delay information. Based on this information, it automatically updates schedules and alarm times as needed. For example, if a train is delayed by 10 minutes, the device will advance the departure time and reset the alarm.

[0200] 6. Reminder notifications

[0201] The device notifies the user of travel and preparation reminders. For example, at 2:10 p.m., it sends a notification saying, "10 minutes until departure. Please start getting ready." The device also sends a notification urging the user to charge the battery if the battery level is low. 10 minutes before the start of a meeting, it also sends a notification saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0202] Combining Emotion Engines

[0203] 1. Emotion Engine Configuration

[0204] The device is equipped with an emotion engine that uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice, recognizing the user's emotional state (e.g., joy, anger, sadness, surprise, etc.) in real time.

[0205] 2. Emotion-based transportation suggestions

[0206] The server analyzes the user's emotional state and suggests transportation options based on the results. For example, if the server detects that the user is feeling stressed, it will prioritize more comfortable transportation options (such as taxis).

[0207] 3. Adjust your schedule based on emotions

[0208] The device will flexibly adjust the schedule based on the user's emotional state. For example, if it detects that the user is tired, it will suggest flexible schedules and break times.

[0209] 4. Tailoring notification content based on emotions

[0210] The device changes the content of the reminder notification depending on the user's emotional state. For example, if the device detects that the user is nervous, it will soften the tone of the reminder.

[0211] Specific examples

[0212] For example, suppose a user has a meeting scheduled in Shinjuku Ward at 3:00 PM. The user's current location is Chiyoda Ward. If the user's emotion engine detects high stress, the server suggests taking a taxi in addition to the train. If the user chooses to take a taxi, the device adds the taxi's travel time to the schedule and sets the departure time to 1:45 PM. The device also suggests relaxing music as a reminder before the user gets in the taxi.

[0213] In this way, the system of the present invention allows users to efficiently manage their schedules without hassle and allows them to respond flexibly to their emotional state. This system reduces users' stress and helps them travel and prepare comfortably.

[0214] The processing flow will be explained below.

[0215] Step 1:

[0216] The device retrieves event data from the user's calendar or schedule application via an API. For example, it sends a request to the calendar API to retrieve a schedule for a meeting starting at 3:00 PM on October 15, 2023.

[0217] Step 2:

[0218] The device acquires the user's current location information using a GPS sensor. For example, the device identifies that the user's current location is Chiyoda Ward, Tokyo.

[0219] Step 3:

[0220] The device identifies the next scheduled location from the schedule data and extracts its location information. For example, it recognizes that the next scheduled location is a "conference room in Shinjuku Ward, Tokyo."

[0221] Step 4:

[0222] The server accesses the API of a map service (e.g., Google Maps API) based on the current location and the next planned destination, and calculates the optimal means of transportation and the required time for each. For example, it obtains information such as 30 minutes by train, 45 minutes by bus, and 50 minutes by bicycle.

[0223] Step 5:

[0224] The terminal presents the acquired information on the means of transportation and the required time to the user, and the user selects the desired means. For example, the user selects "train."

[0225] Step 6:

[0226] The device will then add the travel time to the schedule based on the selected mode of transportation, updating the schedule to, for example, "Depart at 2:30 PM."

[0227] Step 7:

[0228] The device will set an alarm 10 minutes before departure time, for example, "Set alarm for 2:20 p.m."

[0229] Step 8:

[0230] The device periodically checks the traffic information API to get real-time service information, for example, every 10 minutes to check train and bus delay information.

[0231] Step 9:

[0232] The server automatically updates schedules and alarms based on real-time train information. For example, if a train is 10 minutes late, the departure time will be moved forward to 14:10 and the alarm will be reset to 14:00.

[0233] Step 10:

[0234] The device sends a reminder to the user to prepare for travel. For example, at 2:10 p.m., the device sends a notification saying, "There are 10 minutes until departure. Please start preparing."

[0235] Step 11:

[0236] The device checks the battery status and sends a notification to encourage charging if the remaining charge falls below a certain threshold. For example, "I detected that the battery charge is less than 20% and sent a notification saying 'Please charge.'"

[0237] Step 12:

[0238] The device will remind the user to set the silent mode 10 minutes before an important event (e.g., a meeting) starts. For example, a notification will be sent 10 minutes before the start of the meeting saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0239] Adding an Emotion Engine

[0240] Step 13:

[0241] The device activates an emotion engine that uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice. For example, the camera captures the user's face and applies emotion analysis algorithms.

[0242] Step 14:

[0243] The emotion engine analyzes the user's emotional state (e.g., joy, anger, sadness, surprise, etc.) and sends the results to the server. For example, it detects that the user is feeling stressed.

[0244] Step 15:

[0245] The server suggests optimal transportation methods to users based on their emotional state: for example, if a user is feeling stressed, it will prioritize taxis over walking or buses.

[0246] Step 16:

[0247] The user selects the desired transportation method from the suggested transportation methods, for example, "taxi."

[0248] Step 17:

[0249] The device will add travel time to the schedule based on the selected mode of transportation and send notifications that take emotional state into account, such as adding a taxi ride time to the schedule and sending a relaxing music reminder.

[0250] By implementing these steps, users can efficiently manage their schedules without any hassle and can respond flexibly to their emotional state. This system reduces users' stress and supports comfortable travel and preparation.

[0251] Example 2

[0252] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0253] Conventional schedule management systems can suggest daily plans and transportation options to users, but they lack the functionality to respond to real-time transportation information or make flexible suggestions based on the user's emotional state. As a result, they face challenges in smoothly planning trips and adjusting schedules when users feel stressed or when unexpected delays occur. Furthermore, they do not take into account the inconvenience of low battery levels while traveling. There is a need to address these challenges.

[0254] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring schedule information of the user, means for acquiring current location information, means for proposing multiple means of transportation and required times to the next scheduled location based on the schedule information and the current location information, means for adding travel time to the schedule based on the means of transportation selected by the user, means for setting a warning based on the set travel time, means for recognizing the emotional state of the user using emotion analysis means, and means for proposing adjustments to the means of transportation and the schedule based on the emotional state. This allows the user to select the optimal means of transportation based on real-time operation information, and further enables flexible suggestions based on the user's emotional state, thereby reducing stress and enabling smooth schedule management.

[0255] "User schedule information" refers to information about events and schedules that a user has registered in a schedule management system or calendar application.

[0256] "Current Location Information" means real-time geographic location data obtained by a User's Device through GPS or other location services.

[0257] "Multiple means of transportation" refers to the means of transportation available for the user to travel (e.g., train, bus, car, walking, etc.).

[0258] "Travel time" refers to the time required to travel from the current location to the destination, and is calculated based on calculation results that differ for each means of transportation.

[0259] "Travel time" refers to the time required to travel by the means of transportation selected by the user.

[0260] "Warnings" are alarms or notifications that are set to help users stay on schedule, and are issued before departure time or just before an event.

[0261] "Emotion analysis means" is a technology that uses sensors such as cameras and microphones to analyze the user's facial expressions and tone of voice, and recognizes the user's emotional state (e.g., joy, anger, sadness, surprise, etc.).

[0262] "Emotional state" refers to the user's psychological and emotional state as detected using emotion analysis means.

[0263] "Suggestions" are notifications and advice that suggest optimal travel options or schedule adjustments based on the user's emotional state and other real-time information.

[0264] System Overview

[0265] This invention is a schedule management system that uses AI, and provides a means to acquire a user's schedule information, suggest transportation methods and travel times based on current location information, and add them to the schedule. This system is composed of a server, terminals, and users, and by incorporating emotion analysis means, it is also possible to make flexible suggestions based on the user's emotional state.

[0266] System Configuration

[0267] The system is configured using the following hardware and software.

[0268] Device: Mobile devices such as smartphones and tablets

[0269] Server: Cloud Server

[0270] Sentiment analysis tools: camera, microphone, and sentiment analysis algorithms

[0271] Location information acquisition method: GPS sensor

[0272] Data acquisition and proposal method: Data communication using API (e.g., Google Calendar API, map service API)

[0273] Real-time information acquisition method: API that periodically acquires operation information (e.g., map service API)

[0274] System Operation

[0275] The device retrieves event data from the user's calendar or schedule application via an API. This system works with Google Calendar and other calendar services. The retrieved data includes the event start time, end time, location, and event name.

[0276] The device acquires the user's current location using a GPS sensor and stores the acquired information in its internal memory. The device then sends the user's current location to a server. The server extracts the next planned destination from a database and calculates the means of transportation and route based on the location information using a map service API (e.g., Google Maps API). The server then sends the calculation results to the device, which then suggests multiple means of transportation and their estimated travel times to the user.

[0277] When the user selects their preferred mode of transportation on the device, the information is sent to the server, which adds the travel time of the selected mode to the schedule, and the updated schedule is returned to the device, which notifies the user.

[0278] In addition, the device will set a warning 10 minutes before the scheduled departure time and notify the user of the set warning. The device will periodically check the map service API and update the schedule and warning time based on real-time service information.

[0279] Using emotion analysis, the device analyzes the user's facial expressions and voice data through a camera and microphone to recognize the user's emotional state in real time. The analysis results are sent to a cloud server, which then suggests transportation options and adjusts the schedule based on the user's emotional state.

[0280] Specific examples

[0281] For example, consider a case where a user travels to Shinjuku Ward for a meeting scheduled for 3 p.m. The user's current location is Chiyoda Ward, and if the emotion analysis means detects the user's stress, the server will suggest taking a taxi in addition to the train. If the user selects a taxi, the estimated taxi travel time is calculated to be 1:45 p.m., and the device will set an alert accordingly. Before leaving, the device will make suggestions such as "listen to relaxing music."

[0282] Prompt Sentence Examples

[0283] "Create a program that provides users with schedules and transportation options. The system also makes suggestions based on the user's emotional state."

[0284] "Please explain in detail how your schedule management system works, combined with the emotion engine. Please also include the optimal route from the user's current location to their destination."

[0285] In this way, the system of the present invention allows users to efficiently manage their schedules without any hassle. Furthermore, flexible suggestions based on the user's emotional state reduce stress and allow for comfortable travel and preparation.

[0286] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0287] Program processing flow

[0288] Step 1: Get the user's schedule information

[0289] Process flow:

[0290] The device accesses the user's calendar application and obtains schedule information through the API.

[0291] input:

[0292] The user's calendar application access permissions and API key.

[0293] Data processing / calculation:

[0294] The schedule data obtained through the API is analyzed and stored in an internal database.

[0295] output:

[0296] Schedule data captured (e.g., event name, start time, end time, location).

[0297] Specific behavior:

[0298] The device uses the Google Calendar API to retrieve the user's schedule information and store it in an internal database.

[0299] Step 2: Get your current location

[0300] Process flow:

[0301] The device uses the GPS sensor to obtain the user's current location and stores it in its internal memory.

[0302] input:

[0303] GPS sensor data.

[0304] Data processing / calculation:

[0305] The acquired latitude and longitude information is analyzed to determine the current location.

[0306] output:

[0307] Current location information (latitude and longitude).

[0308] Specific behavior:

[0309] The device periodically uses the GPS sensor to obtain the user's current location and stores it in its internal memory.

[0310] Step 3: Identify the next planned location and calculate the optimal route

[0311] Process flow:

[0312] The server identifies the next planned location and uses the map service's API to calculate the optimal route and travel time from the current location to the next planned location.

[0313] input:

[0314] Current location information, schedule information, map service API key.

[0315] Data processing / calculation:

[0316] Use the map service API to calculate multiple means of transportation (e.g., train, bus, car, walking) and the travel time required for each.

[0317] output:

[0318] Travel time and route information for each mode of transportation.

[0319] Specific behavior:

[0320] The server sends a request to the Google Maps API to calculate the optimal route from the current location to the next destination and the travel time for multiple modes of transportation.

[0321] Step 4: Select your mode of transport and add travel time

[0322] Process flow:

[0323] The terminal displays a notification prompting the user to select a means of transportation and adds travel time to the schedule based on the user's selection.

[0324] input:

[0325] Travel time and route information for each mode of transportation.

[0326] Data processing / calculation:

[0327] Add and recalculate travel time to the schedule based on the user's selected mode of travel.

[0328] output:

[0329] Updated schedule information.

[0330] Specific behavior:

[0331] The terminal displays a notification to the user to select a mode of transportation such as "train," "bus," or "taxi," and updates the schedule based on the selected mode of transportation.

[0332] Step 5: Set an alarm

[0333] Process flow:

[0334] The device will set an alarm 10 minutes before the scheduled departure time.

[0335] input:

[0336] Updated schedule information.

[0337] Data processing / calculation:

[0338] Calculate 10 minutes before departure time and set the alarm.

[0339] output:

[0340] Pre-departure warning alarm.

[0341] Specific behavior:

[0342] The device will set a warning alarm 10 minutes before departure time and send a notification to the user saying, "10 minutes left until departure."

[0343] Step 6: Reflecting real-time information

[0344] Process flow:

[0345] The device periodically checks real-time operation information and updates schedules and alarm times.

[0346] input:

[0347] Real-time information, updated schedules.

[0348] Data processing / calculation:

[0349] Recalculate and update schedules and alarms based on delay and operational information.

[0350] output:

[0351] Updated schedule information and reminders.

[0352] Specific behavior:

[0353] The device checks the map service API every 15 minutes to obtain delay and operation information and update schedules and alarms.

[0354] Step 7: Reminders

[0355] Process flow:

[0356] The device will notify the user of travel and preparation reminders.

[0357] input:

[0358] Updated schedule information, alarm 2 hours before departure.

[0359] Data processing / calculation:

[0360] Generate reminders before you leave or before an event.

[0361] output:

[0362] Reminder notifications.

[0363] Specific behavior:

[0364] The device will send reminder notifications such as "Leave in 10 minutes. Start getting ready" or "Meeting is about to start. Set your smartphone to silent mode."

[0365] Step 8: Emotion Recognition with the Emotion Engine

[0366] Process flow:

[0367] The device uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize their emotional state.

[0368] input:

[0369] Sensor data from cameras and microphones.

[0370] Data processing / calculation:

[0371] Emotion analysis algorithms are used to determine the user's emotional state in real time.

[0372] output:

[0373] The user's emotional state (e.g., happy, anger, sadness, surprise).

[0374] Specific behavior:

[0375] The device analyzes the user's facial expressions and tone of voice through a camera and microphone, and recognizes the user's emotional state, such as "feeling stressed."

[0376] Step 9: Emotion-Based Suggestions

[0377] Process flow:

[0378] The server generates optimal travel and schedule adjustment suggestions based on the emotional state.

[0379] input:

[0380] User's emotional state, schedule information, real-time traffic information.

[0381] Data processing / calculation:

[0382] Apply algorithms that recommend transportation and schedule adjustments based on emotional state.

[0383] output:

[0384] Proposal Notice.

[0385] Specific behavior:

[0386] Based on the analysis result that "the user is feeling stressed," the server generates suggestions such as "taking a taxi" or "listening to relaxing music" and notifies the user via the terminal.

[0387] (Application example 2)

[0388] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0389] While conventional schedule management systems can efficiently adjust schedules by taking into account users' travel times and real-time information, they have the problem of being unable to flexibly respond to users' emotional states. In particular, when managing schedules for store staff, if staff stress and fatigue are not properly managed, work efficiency and the quality of service may decline.

[0390] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring the user's schedule information, means for acquiring current location information, means for proposing multiple means of transportation and required times to the next scheduled location based on the schedule information and the current location information, means for adding travel time to the schedule based on the transportation means selected by the user, means for setting an alarm based on the set travel time, means for performing emotion analysis using a camera and microphone to recognize the user's emotional state, and means for adjusting the schedule and changing the notification content based on the user's emotional state. This enables flexible schedule adjustment and notification that takes the user's emotional state into consideration, enabling store staff to perform their work efficiently and provide high-quality services.

[0391] "User" refers to a person who uses the system.

[0392] "Schedule information" refers to data that records a user's schedules and appointments.

[0393] "Current location information" refers to information about the user's current location, and is mainly obtained using a GPS sensor.

[0394] "Transportation" refers to the means of transportation used by the user to travel to the next scheduled location.

[0395] "Travel time" refers to the time it takes for the user to travel from their current location to their next scheduled location.

[0396] "Emotional state" refers to a user's mental and emotional state, including emotions such as joy, anger, sadness, and surprise.

[0397] A "camera" is a device that captures images or videos using an optical sensor.

[0398] A "microphone" is a device for picking up sound, and the user's voice is used for emotion analysis.

[0399] "Emotion analysis" refers to the process of analyzing a user's facial expressions and tone of voice to recognize their emotional state.

[0400] An "alarm" is a function that notifies the user at a set time.

[0401] "Notification content" refers to the content of the message sent to the user as an alarm or reminder.

[0402] "Real-time information" refers to data that changes over time, such as the latest operation and traffic information.

[0403] The term "system" refers to a collection of hardware and software required to realize the various functions of the present invention.

[0404] The present invention is a system for improving the efficiency of a user's schedule management, and is embodied in the following specific form.

[0405] System configuration

[0406] The system includes means for acquiring the user's schedule information, means for acquiring current location information, means for suggesting multiple means of transportation and travel times to the next scheduled location, means for adding travel time to the schedule based on the means of transportation selected by the user, means for setting an alarm based on the set travel time, a camera and microphone and emotion analysis means for recognizing the user's emotional state, means for adjusting the schedule based on the user's emotional state, and means for changing the content of notifications.

[0407] Program processing overview

[0408] The system performs processing using the following hardware and software.

[0409] Hardware

[0410] 1. Smart glasses: display information and track user behavior.

[0411] 2. Camera: Used to recognize the user's facial expressions.

[0412] 3. Microphone: Uses the user's tone of voice for sentiment analysis.

[0413] 4. GPS sensor: Used to obtain current location information.

[0414] software

[0415] 1. Schedule management API: Obtain and manage user schedules (e.g., Google Calendar API).

[0416] 2. Map service API: Calculates the means of transportation and travel time to the next scheduled destination (e.g., Google Maps API).

[0417] 3. Emotion Recognition API: Analyze the user's emotional state (e.g., Emotion Recognition API).

[0418] 4. Notification system: Send reminders and alarms to users.

[0419] Data processing and calculation

[0420] The system processes and calculates the data as follows:

[0421] 1. Get user schedule:

[0422] The server retrieves the user's schedule information by calling the schedule management API, which downloads the user's schedule and appointment details.

[0423] 2. Obtaining current location information:

[0424] The server uses a GPS sensor to obtain the user's current location, which provides real-time location information about where the user is currently located.

[0425] 3. Travel options and travel times:

[0426] The server calls the map service API and calculates the travel time and options for travelling from the current location to the next destination, such as train, bus, car, and walking.

[0427] 4. Emotional state recognition and analysis:

[0428] The server collects data from the camera and microphone and uses an emotion recognition API to analyze the user's emotional state, thereby identifying the emotion the user is currently feeling (e.g., joy, anger, sadness, surprise, etc.).

[0429] 5. Adjust schedule and notification content:

[0430] The server adjusts the schedule based on the user's emotional state, for example by adding breaks to the schedule if it senses that the user is tired, and it also uses a notification system to send emotionally sensitive notifications.

[0431] Specific examples

[0432] As a specific example, consider a situation where store staff member A is scheduled to work the cash register from 2 p.m. and put merchandise on display from 3 p.m. The user's smart glasses calculate the time it takes to move around the store and the store's congestion status in real time, and reflect this in the schedule. Furthermore, if the analysis shows that staff member A is feeling "tired" or "stressed," the glasses will suggest adding more break time or notify the employee of an adjustment to the division of labor.

[0433] Prompt Sentence Examples

[0434] "Based on the new schedule, Staff A's task starting at 3 p.m. is to stock the shelves. However, if Staff A is feeling stressed, how do you adjust their schedule and alarms?"

[0435] This allows users to efficiently manage their schedules without hassle and allows them to respond flexibly to their emotional state. This system reduces stress for users and enables them to carry out their work comfortably.

[0436] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0437] Step 1:

[0438] The device calls the schedule management API to obtain the user's schedule information. This process involves sending a request to the API to obtain data including details of the user's schedule and appointments. Once the data is obtained, the device stores it in its internal database.

[0439] Step 2:

[0440] The device uses the GPS sensor to obtain the user's current location information. In this process, the current latitude and longitude data is obtained from the GPS sensor to obtain the coordinates of the user's current location. The obtained coordinate data becomes the input for proceeding to the next processing step.

[0441] Step 3:

[0442] The server proposes multiple means of transportation and the required travel time to the next scheduled location based on the schedule information and current location information. In this process, it calls the map service API to obtain the route and means of transportation (e.g., train, bus, car, walking, etc.) from the current location to the scheduled location, and calculates the required travel time for each. The server sends the obtained route information to the terminal and proposes multiple means of transportation to the user.

[0443] Step 4:

[0444] The user selects a desired transportation method from the transportation methods proposed through the terminal. In this process, travel time data is acquired based on the transportation method selected by the user. The terminal adds the selected transportation method and travel time to the schedule and updates it.

[0445] Step 5:

[0446] The device sets an alarm based on the set travel time. In this process, the alarm is set a certain time (for example, 10 minutes) before the scheduled departure time based on the travel time data. The set alarm information is saved in the device, and the user is notified when the departure time approaches.

[0447] Step 6:

[0448] The server uses a camera and microphone to capture the user's emotional state. This process involves capturing facial expression data through the camera and recording voice tone through the microphone. It then calls an emotion recognition API to analyze these data and identify the user's emotional state (e.g., joy, anger, sadness, surprise, etc.).

[0449] Step 7:

[0450] The server adjusts the schedule based on the user's emotional state. This process involves making necessary schedule changes depending on the user's emotional state. For example, if the server determines that the user is "tired," it adds a break to the schedule. It also changes work allocation and adjusts tasks as necessary.

[0451] Step 8:

[0452] The device changes the notification content according to the user's emotional state. This process flexibly adjusts the content of reminder and alarm notifications based on the user's emotional state. For example, if the device determines that the user is "feeling stressed," it sends a message to soften the tone of the notification.

[0453] Step 9:

[0454] The device obtains real-time traffic information and updates schedules and alarms. This process involves calling the traffic information API to obtain the latest traffic information and checking whether changes to the mode of travel or travel time are necessary. If necessary, the device automatically updates schedules and alarms and notifies the user.

[0455] Step 10:

[0456] The device monitors the remaining battery level and sends a notification to the user urging charging if it falls below a predetermined threshold. The battery monitoring system periodically checks the remaining battery level and, if it falls below a threshold, sends a notification to the user saying, "The battery is low. Please start charging."

[0457] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0458] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0459] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0460] [Second embodiment]

[0461] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0462] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0463] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0464] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0465] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0466] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0467] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0468] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0469] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0470] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0471] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0472] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0473] This invention is a smartphone schedule management system that uses AI to acquire user schedule information, automatically calculate travel time, add travel time to the schedule, set alarms, and update real-time information. The specific configuration of each part and operation of this system is explained below.

[0474] composition

[0475] 1. Get user schedule

[0476] The device retrieves event data from the user's calendar or schedule application via an API. This system is compatible with Google Calendar and other calendar services, for example.

[0477] 2. Travel time and route calculation

[0478] The device acquires the user's current location using the GPS sensor, then extracts the next scheduled location from the schedule and identifies its location information.

[0479] Based on this, the server accesses the API of the map service (for example, Google Maps API) and calculates the optimal means of transportation (train, bus, car, walking, etc.) from the current location to the next destination and the required time for each. After the server obtains this information, it proposes multiple means of transportation and their required times to the user.

[0480] 3. Add travel time to your schedule

[0481] When the user selects their preferred mode of transportation, the device adds the travel time to the schedule. For example, if the user's current location is Chiyoda Ward, Tokyo, and their next destination is Shinjuku Ward, and they select the train, the calculated travel time of 30 minutes will be reflected in the schedule.

[0482] 4. Set an alarm

[0483] The device will set an alarm 10 minutes before the scheduled departure time, so if the departure is scheduled for 2:30 PM, the alarm will be set for 2:20 PM.

[0484] 5. Real-time information

[0485] The device periodically checks real-time train and bus information to obtain delay information. Based on this information, it automatically updates schedules and alarm times as needed. For example, if a train is delayed by 10 minutes, the device will advance the departure time and reset the alarm.

[0486] 6. Reminder notifications

[0487] The device notifies the user of travel and preparation reminders. For example, at 2:10 p.m., it sends a notification saying, "10 minutes until departure. Please start getting ready." The device also sends a notification urging the user to charge the battery if the battery level is low. 10 minutes before the start of a meeting, it also sends a notification saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0488] Specific examples

[0489] For example, suppose a user has a meeting scheduled for 3:00 PM in Shinjuku Ward. The user's current location is Chiyoda Ward, and the schedule is retrieved from a calendar application. The device then identifies the user's current location using GPS, and the server uses a map service API to calculate the optimal route and travel time from Chiyoda Ward to Shinjuku Ward.

[0490] If the user chooses to travel by train, the device calculates the travel time as 30 minutes and adds a departure time of 2:30 p.m. to the schedule. It also sets an alarm for 2:20 p.m. If the train is delayed, the device checks real-time information, automatically updates the schedule and alarm, and notifies the user.

[0491] In this way, the system of the present invention allows users to efficiently manage their schedules without hassle, helping to avoid mistakes and reducing stress.

[0492] The processing flow will be explained below.

[0493] Step 1:

[0494] The device retrieves event data from the user's calendar or schedule application via an API. For example, it sends a request to the calendar API to retrieve a schedule for a meeting starting at 3:00 PM on October 15, 2023.

[0495] Step 2:

[0496] The device acquires the user's current location information using a GPS sensor. For example, the device identifies that the user's current location is Chiyoda Ward, Tokyo.

[0497] Step 3:

[0498] The device identifies the next scheduled location from the schedule data and extracts its location information. For example, it recognizes that the next scheduled location is a "conference room in Shinjuku Ward, Tokyo."

[0499] Step 4:

[0500] The server accesses the API of a map service (e.g., Google Maps API) based on the current location and the next planned destination, and calculates the optimal means of transportation and the required time for each. For example, it obtains information such as 30 minutes by train, 45 minutes by bus, and 50 minutes by bicycle.

[0501] Step 5:

[0502] The terminal presents the acquired information on the means of transportation and the required time to the user, and the user selects the desired means. For example, the user selects "train."

[0503] Step 6:

[0504] The device will then add the travel time to the schedule based on the selected mode of transportation, updating the schedule to, for example, "Depart at 2:30 PM."

[0505] Step 7:

[0506] The device will set an alarm 10 minutes before departure time, for example, "Set alarm for 2:20 p.m."

[0507] Step 8:

[0508] The device periodically retrieves real-time data, including operational information, and checks train and bus delay information. For example, it retrieves information that a train is 10 minutes late.

[0509] Step 9:

[0510] The server automatically updates schedules and alarms based on the acquired operation information. For example, it may move the departure time forward to "2:15 PM" and reset the alarm to "2:05 PM."

[0511] Step 10:

[0512] The device sends a reminder to the user to prepare for travel. For example, at 2:10 p.m., the device sends a notification saying, "There are 10 minutes until departure. Please start preparing."

[0513] Step 11:

[0514] The device checks the battery status and sends a notification to encourage charging if the remaining charge falls below a certain threshold. For example, "I detected that the battery charge is less than 20% and sent a notification saying 'Please charge.'"

[0515] Step 12:

[0516] The device will remind the user to set the silent mode 10 minutes before an important event (e.g., a meeting) starts. For example, a notification will be sent 10 minutes before the start of the meeting saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0517] Through these steps, users can efficiently and accurately manage their schedules, enabling smooth travel and preparation without the need for manual operations.

[0518] Example 1

[0519] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0520] Conventional schedule management systems require users to manually check current location information and transportation options and reflect them in their schedules. This makes schedule management cumbersome and makes it difficult to efficiently complete plans. Furthermore, they often lack the ability to provide real-time traffic information or appropriate alerts or notifications based on remaining battery power, increasing the risk of users missing important appointments.

[0521] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0522] In this invention, the server includes means for acquiring the user's schedule information, means for acquiring location information, means for proposing multiple means of transportation and travel times to the next scheduled location based on the schedule information and location information, means for adding travel time to the schedule based on the transportation means selected by the user, means for setting an alarm based on the set departure time, means for acquiring traffic operation information in real time and updating the schedule and alarm, and means for monitoring the remaining battery level and sending a notification urging the user to charge the battery when the battery falls below a predetermined threshold. This allows the user to efficiently manage their schedule without having to perform complicated manual operations, significantly reducing the risk of missing important appointments.

[0523] "User schedule information" is data relating to the user's schedule, obtained from a calendar application or a schedule application.

[0524] "Location Information" means data about a user's current latitude and longitude obtained using a GPS sensor or other positioning technology.

[0525] "Transportation" refers to the means used for travel, including trains, buses, cars, and walking.

[0526] "Travel time" is the estimated time it will take to travel from the current location to the next scheduled location.

[0527] "Real-time transportation operation information" is data that obtains the operation status of transportation in real time and reflects information such as delays and cancellations.

[0528] "Battery level monitoring" is a feature that measures the remaining battery level of smartphones and other mobile devices and notifies you when it reaches a certain level.

[0529] "Add to schedule" means to reflect new schedules or information in the user's schedule.

[0530] "Set an alarm" is a function that notifies the user when a specific time or condition is met.

[0531] The "notification to encourage charging" is a notification to encourage the user to charge when the remaining battery power is low.

[0532] This invention is a smartphone schedule management system that uses AI, which acquires the user's schedule information, suggests transportation methods along with location information, adds travel time to the schedule, sets alarms, and automatically updates real-time information. The specific configuration of each part and operation of this system is explained below.

[0533] Get user's appointment information

[0534] The device accesses a calendar application (e.g., a general-purpose calendar app) based on the user's authentication information and obtains the user's event information through the API. Once this operation is complete, the obtained event data is saved in the device's database. Specifically, the device communicates with the calendar API using the user's OAuth token, receives the obtained data in JSON format, and stores it in the local database.

[0535] Get the user's current location and identify the next scheduled location

[0536] The device activates the GPS sensor to obtain the user's current location. Then, the device extracts the location information of the next scheduled event from the most recent scheduled event and determines the latitude and longitude. For example, if the current location is Chiyoda Ward, Tokyo, and the next scheduled event location is Shinjuku Ward, the device will obtain the current location using GPS and the location information of the next scheduled event through a geocoding service.

[0537] Calculate travel route and travel time

[0538] The device sends this location information to the server. The server calls a map service API (for example, a general-purpose map service API) to calculate the optimal means of transportation from the current location to the next scheduled destination and the required travel time. The calculation results are returned to the device in JSON format. For example, the device sends a request to the map service API through the server to calculate the required travel time for various modes of transportation, such as train, bus, car, and walking.

[0539] Choosing transportation methods and reflecting them in your schedule

[0540] The device displays options for transportation and travel times to the user. When the user selects their preferred transportation method, the device reflects that travel time in the schedule. For example, if the user selects traveling by train, the device calculates the travel time as 30 minutes and updates the schedule to depart at 2:30 PM. The device then sets the alarm time to 2:20 PM based on this information.

[0541] Reflecting real-time information

[0542] The device obtains real-time traffic information at regular intervals and updates schedules and alarms as necessary. Specifically, the device uses a traffic information API to check real-time traffic information, and if there is delay information, it resets the departure time or alarm and notifies the user.

[0543] Reminders and notifications

[0544] The device sends users reminders to travel and prepare. It also notifies users when the battery is low, urging them to charge the device. For example, at 2:10 p.m., it sends a notification 10 minutes before departure, and 10 minutes before a meeting starts, it sends a notification saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0545] Specific examples

[0546] For example, consider a case where a user has a meeting scheduled in Shinjuku Ward at 3:00 PM. The user's current location is Chiyoda Ward, and this appointment is retrieved from the configured calendar application. The device then uses GPS to determine the user's current location, and the server uses a general-purpose map service API to calculate the optimal route and travel time from Chiyoda Ward to Shinjuku Ward.

[0547] If the user chooses to travel by train, the device calculates the travel time as 30 minutes, updates the schedule to depart at 2:30 p.m., and sets an alarm for 2:20 p.m. If the train is delayed, the device checks real-time information, automatically updates the schedule and alarm, and notifies the user.

[0548] In this way, the system of the present invention allows users to efficiently manage their schedules without having to perform complicated manual operations, significantly reducing the risk of missing important appointments.

[0549] Example prompt sentence:

[0550] "I have a meeting in Shinjuku Ward at 3 PM. My current location is in Chiyoda Ward. Please calculate and suggest the best transportation method and travel time, and update my schedule accordingly. Also, please set an alarm 10 minutes before departure time, and update my schedule as needed based on real-time traffic information."

[0551] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0552] Step 1:

[0553] The device accesses a calendar API (for example, a general-purpose calendar API) based on the user's authentication information and obtains the user's schedule information. Specifically, it makes an API call using an OAuth token and receives the obtained schedule information in JSON format. User authentication information is required as input, and event data is obtained as output. This data is saved in a database within the device.

[0554] Step 2:

[0555] The device activates the GPS sensor and obtains the user's current location information. Specifically, it uses the GPS module to measure the current latitude and longitude. The input requires location measurement using the GPS module, and the output is the latitude and longitude data of the current location.

[0556] Step 3:

[0557] The device extracts the location information of the next scheduled event from the acquired schedule information. Specifically, it parses the address information from the next scheduled event and converts it to latitude and longitude using a geocoding API. The address of the next scheduled event is required as input, and the latitude and longitude data of the scheduled location is obtained as output.

[0558] Step 4:

[0559] The device sends latitude and longitude information of the current location and the next planned location to the server. The server calls a map service API (for example, a general-purpose map service API) and calculates the multiple means of transportation from the current location to the next planned location and the required travel time. Specifically, it uses the Directions API to obtain the route and time for each means. The latitude and longitude of the current location and the planned location are required as input, and the required travel time and route information for each means of transportation are obtained as output. The calculation results are returned to the device in JSON format.

[0560] Step 5:

[0561] The device presents the user with multiple transportation options and their respective travel times. When the user selects their preferred transportation option, the device reflects the selected travel time in the schedule. Specifically, the device displays a list of options through a user interface and receives the user's selection. The inputs are the travel times for each transportation option and the user's selection, and the output is an updated schedule.

[0562] Step 6:

[0563] The terminal sets an alarm 10 minutes before the scheduled departure time. Specifically, the alarm is set using the system's alarm manager. The departure time data is required as input, and the alarm setting is output.

[0564] Step 7:

[0565] The device periodically retrieves real-time traffic information and updates schedules and alarms. Specifically, it uses a traffic information API to check the latest traffic conditions and reset schedules and alarms as needed. Real-time traffic data is required as input, and updated schedules and alarms are obtained as output.

[0566] Step 8:

[0567] The device sends travel and preparation reminders to the user. Specifically, it uses push notifications to provide pre-departure reminders and low battery notifications. The inputs required are the timing of scheduled events and battery level data, and the output is the reminder notification sent to the user.

[0568] (Application example 1)

[0569] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0570] The purpose of this invention is to efficiently manage delivery staff schedules and optimize their travel in food delivery services. When there are multiple delivery destinations, conventional methods make it cumbersome to determine routes and manage travel times, and it is difficult to quickly respond to changes in traffic conditions. Therefore, there is a need for a system that can improve delivery efficiency and accuracy.

[0571] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0572] In this invention, the server includes means for acquiring schedule information of the user, means for acquiring current location information, means for proposing multiple means of transportation and required travel times based on the schedule information and current location information, means for adding travel time to the schedule based on the means of transportation selected by the user, means for setting an alarm based on the set travel time, and means for identifying the current location of the delivery person, optimizing the route to the next delivery destination, and updating the delivery schedule in real time, thereby enabling real-time updating of the delivery schedule and efficient route proposals.

[0573] The "means for acquiring user schedule information" is a function that enables an electronic device to automatically collect event data from a user's calendar or schedule.

[0574] "Means for obtaining current location information" means a function for determining the current geographic location of a user or device using GPS or other location information systems.

[0575] The "means for proposing multiple means of transportation and required travel times" is a function for calculating options for transportation (e.g., walking, car, train, etc.) and the required travel times for each, and presenting them to the user.

[0576] The "means for adding travel time to the schedule" is a function for reflecting the travel time calculated based on the means of transportation selected by the user in the schedule.

[0577] The "means for setting an alarm" is a function for sending a notification to the user at a specified time, and serves to prompt the user to leave or get ready.

[0578] "Means for determining the delivery person's current location" refers to a function that uses GPS or a location information system to determine the delivery person's current geographic location in real time.

[0579] The "means for optimizing the route to the next delivery destination" is a function for calculating and presenting the optimal and shortest route to the delivery destination.

[0580] "Means for updating delivery schedules in real time" refers to a function for updating delivery schedules at any time based on current traffic conditions and delivery progress.

[0581] "Means for obtaining real-time operation information and updating schedules and alarms" refers to a function for obtaining traffic conditions and operation information in real time and adjusting schedule and alarm settings based on this information.

[0582] "Means for monitoring the remaining battery level and sending a notification urging the user to charge when the battery falls below a specified threshold" is a function for constantly monitoring the battery status of the device and notifying the user to charge when the remaining battery level falls below a certain level.

[0583] The present invention provides a system for efficiently managing delivery staff schedules and optimizing their travel in a food delivery service. The system includes a means for acquiring a user's schedule information, a means for acquiring current location information, a means for proposing multiple means of transportation and required travel times, a means for adding travel time to the schedule, a means for setting an alarm, a means for identifying the delivery staff's current location, a means for optimizing the route to the next delivery destination, and a means for updating the delivery schedule in real time.

[0584] Configuration and operation

[0585] 1. Obtaining user schedule information

[0586] The server receives event data from the delivery person's schedule management application via an API. This system is compatible with general calendar services, for example.

[0587] 2. Obtaining current location information

[0588] The device uses a GPS sensor to acquire the delivery person's current location, and then sends the delivery person's current location information to the server.

[0589] 3. Proposing multiple transportation methods and travel times

[0590] The server calculates multiple means of transportation (e.g., car, bicycle, walking, etc.) to the next delivery destination and the required time for each, based on the acquired schedule information and current location information. The server uses a map service API (e.g., map service API) to suggest multiple means of transportation and their required time to the delivery person.

[0591] 4. Add travel time to your schedule

[0592] When a delivery person selects their preferred mode of transportation, the server adds the travel time to the schedule. For example, if the current location is somewhere and the next delivery destination is somewhere else, and the delivery person selects a car, the calculated travel time is reflected in the schedule.

[0593] 5. Set an alarm

[0594] The server sets an alarm 10 minutes before the scheduled departure time, allowing the delivery person to prepare for the next delivery.

[0595] 6. Real-time delivery schedule updates

[0596] The device periodically checks real-time traffic information to obtain traffic conditions and delay information. The server automatically updates schedules and alarm times as needed based on the information obtained. For example, if there is traffic congestion, the server will adjust departure times and reset alarms.

[0597] Hardware and software used

[0598] This system uses smart glasses equipped with a GPS sensor as the terminal, and the server calculates routes and retrieves schedule information using the Google Maps API and general calendar APIs.

[0599] Specific examples

[0600] For example, if a delivery person has multiple delivery destinations, the system calculates the travel time from the current location to each destination in real time and suggests an efficient route. The schedule is updated based on the transportation method selected by the delivery person, and an alarm is set for the departure time. Traffic conditions and delay information are obtained in real time, and the schedule and alarms are automatically updated as needed.

[0601] Prompt Sentence Examples

[0602] Here is an example prompt:

[0603] Locate your delivery drivers with GPS and calculate the best route and travel time to their next delivery. Update schedules in real time and set alarms based on their next due time. Also, monitor traffic conditions and reschedule if delays occur.

[0604] This allows the system of the present invention to support delivery personnel in efficient delivery and improve the reliability and accuracy of delivery services.

[0605] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0606] Step 1:

[0607] The server retrieves schedule information from the delivery driver's schedule management application via an API. The inputs include the delivery driver's account information and API key. The output is the schedule information in JSON format or other structured data format. The server parses this data to obtain the delivery order and location information.

[0608] Step 2:

[0609] The terminal acquires the delivery person's current location information using a GPS sensor. The input is a signal obtained from a GPS device, and the output is real-time latitude and longitude data. The terminal then transmits the acquired geographic information to a server.

[0610] Step 3:

[0611] The server proposes multiple means of transportation to the next delivery destination and the required travel time based on the acquired schedule information and current location information. This step requires the latitude and longitude of the current location and schedule information as input, and the required travel time for each means of transportation (e.g., car, bicycle, walking) as output. The server uses a map service API to calculate the optimal route and means of transportation.

[0612] Step 4:

[0613] The user selects the desired transportation method from the presented options. The input is the user's selection information, and the output is the data of the selected transportation method. The server recalculates the required travel time based on the selected transportation method and adds the travel time to the next delivery destination to the schedule.

[0614] Step 5:

[0615] The server sets an alarm based on the set travel time. The input is the updated schedule information and travel time, and the output is the alarm setting data. Based on this, the server sets a notification to be sent before the departure time.

[0616] Step 6:

[0617] The terminal periodically checks real-time traffic information and obtains traffic conditions and delay information. As input, it sends a request to the traffic information API, and as output, it obtains the latest traffic information. The terminal then sends this to the server.

[0618] Step 7:

[0619] The server updates the schedule based on the acquired real-time operation information. This step requires the delivery schedule and real-time operation information as input, and obtains the updated schedule data as output. The server also resets the alarm time if necessary and sends a notification to the user.

[0620] This allows delivery personnel to efficiently manage their delivery schedules and respond quickly to changes in traffic conditions.

[0621] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0622] This invention is a smartphone schedule management system that uses AI. It acquires the user's schedule information, calculates travel time, adds travel time to the schedule, sets alarms, and automatically updates real-time information. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to flexibly suggest schedules and provide notifications according to the user's emotional state. The specific components and operations of this system are described below.

[0623] composition

[0624] 1. Get user schedule

[0625] The device retrieves event data from the user's calendar or schedule application via an API. This system is compatible with Google Calendar and other calendar services, for example.

[0626] 2. Travel time and route calculation

[0627] The device acquires the user's current location using the GPS sensor, then extracts the next scheduled location from the schedule and identifies its location information.

[0628] Based on this, the server accesses the API of the map service (for example, Google Maps API) and calculates the optimal means of transportation (train, bus, car, walking, etc.) from the current location to the next destination and the required time for each. After the server obtains this information, it proposes multiple means of transportation and their required times to the user.

[0629] 3. Add travel time to your schedule

[0630] When the user selects their preferred mode of transportation, the device adds the travel time to the schedule. For example, if the user's current location is Chiyoda Ward, Tokyo, and their next destination is Shinjuku Ward, and they select the train, the calculated travel time of 30 minutes will be reflected in the schedule.

[0631] 4. Set an alarm

[0632] The device will set an alarm 10 minutes before the scheduled departure time, so if the departure is scheduled for 2:30 PM, the alarm will be set for 2:20 PM.

[0633] 5. Real-time information

[0634] The device periodically checks real-time train and bus information to obtain delay information. Based on this information, it automatically updates schedules and alarm times as needed. For example, if a train is delayed by 10 minutes, the device will advance the departure time and reset the alarm.

[0635] 6. Reminder notifications

[0636] The device notifies the user of travel and preparation reminders. For example, at 2:10 p.m., it sends a notification saying, "10 minutes until departure. Please start getting ready." The device also sends a notification urging the user to charge the battery if the battery level is low. 10 minutes before the start of a meeting, it also sends a notification saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0637] Combining Emotion Engines

[0638] 1. Emotion Engine Configuration

[0639] The device is equipped with an emotion engine that uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice, recognizing the user's emotional state (e.g., joy, anger, sadness, surprise, etc.) in real time.

[0640] 2. Emotion-based transportation suggestions

[0641] The server analyzes the user's emotional state and suggests transportation options based on the results. For example, if the server detects that the user is feeling stressed, it will prioritize more comfortable transportation options (such as taxis).

[0642] 3. Adjust your schedule based on emotions

[0643] The device will flexibly adjust the schedule based on the user's emotional state. For example, if it detects that the user is tired, it will suggest flexible schedules and break times.

[0644] 4. Tailoring notification content based on emotions

[0645] The device changes the content of the reminder notification depending on the user's emotional state. For example, if the device detects that the user is nervous, it will soften the tone of the reminder.

[0646] Specific examples

[0647] For example, suppose a user has a meeting scheduled in Shinjuku Ward at 3:00 PM. The user's current location is Chiyoda Ward. If the user's emotion engine detects high stress, the server suggests taking a taxi in addition to the train. If the user chooses to take a taxi, the device adds the taxi's travel time to the schedule and sets the departure time to 1:45 PM. The device also suggests relaxing music as a reminder before the user gets in the taxi.

[0648] In this way, the system of the present invention allows users to efficiently manage their schedules without hassle and allows them to respond flexibly to their emotional state. This system reduces users' stress and helps them travel and prepare comfortably.

[0649] The processing flow will be explained below.

[0650] Step 1:

[0651] The device retrieves event data from the user's calendar or schedule application via an API. For example, it sends a request to the calendar API to retrieve a schedule for a meeting starting at 3:00 PM on October 15, 2023.

[0652] Step 2:

[0653] The device acquires the user's current location information using a GPS sensor. For example, the device identifies that the user's current location is Chiyoda Ward, Tokyo.

[0654] Step 3:

[0655] The device identifies the next scheduled location from the schedule data and extracts its location information. For example, it recognizes that the next scheduled location is a "conference room in Shinjuku Ward, Tokyo."

[0656] Step 4:

[0657] The server accesses the API of a map service (e.g., Google Maps API) based on the current location and the next planned destination, and calculates the optimal means of transportation and the required time for each. For example, it obtains information such as 30 minutes by train, 45 minutes by bus, and 50 minutes by bicycle.

[0658] Step 5:

[0659] The terminal presents the acquired information on the means of transportation and the required time to the user, and the user selects the desired means. For example, the user selects "train."

[0660] Step 6:

[0661] The device will then add the travel time to the schedule based on the selected mode of transportation, updating the schedule to, for example, "Depart at 2:30 PM."

[0662] Step 7:

[0663] The device will set an alarm 10 minutes before departure time, for example, "Set alarm for 2:20 p.m."

[0664] Step 8:

[0665] The device periodically checks the traffic information API to get real-time service information, for example, every 10 minutes to check train and bus delay information.

[0666] Step 9:

[0667] The server automatically updates schedules and alarms based on real-time train information. For example, if a train is 10 minutes late, the departure time will be moved forward to 14:10 and the alarm will be reset to 14:00.

[0668] Step 10:

[0669] The device sends a reminder to the user to prepare for travel. For example, at 2:10 p.m., the device sends a notification saying, "There are 10 minutes until departure. Please start preparing."

[0670] Step 11:

[0671] The device checks the battery status and sends a notification to encourage charging if the remaining charge falls below a certain threshold. For example, "I detected that the battery charge is less than 20% and sent a notification saying 'Please charge.'"

[0672] Step 12:

[0673] The device will remind the user to set the silent mode 10 minutes before an important event (e.g., a meeting) starts. For example, a notification will be sent 10 minutes before the start of the meeting saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0674] Adding an Emotion Engine

[0675] Step 13:

[0676] The device activates an emotion engine that uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice. For example, the camera captures the user's face and applies emotion analysis algorithms.

[0677] Step 14:

[0678] The emotion engine analyzes the user's emotional state (e.g., joy, anger, sadness, surprise, etc.) and sends the results to the server. For example, it detects that the user is feeling stressed.

[0679] Step 15:

[0680] The server suggests optimal transportation methods to users based on their emotional state: for example, if a user is feeling stressed, it will prioritize taxis over walking or buses.

[0681] Step 16:

[0682] The user selects the desired transportation method from the suggested transportation methods, for example, "taxi."

[0683] Step 17:

[0684] The device will add travel time to the schedule based on the selected mode of transportation and send notifications that take emotional state into account, such as adding a taxi ride time to the schedule and sending a relaxing music reminder.

[0685] By implementing these steps, users can efficiently manage their schedules without any hassle and can respond flexibly to their emotional state. This system reduces users' stress and supports comfortable travel and preparation.

[0686] Example 2

[0687] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0688] Conventional schedule management systems can suggest daily plans and transportation options to users, but they lack the functionality to respond to real-time transportation information or make flexible suggestions based on the user's emotional state. As a result, they face challenges in smoothly planning trips and adjusting schedules when users feel stressed or when unexpected delays occur. Furthermore, they do not take into account the inconvenience of low battery levels while traveling. There is a need to address these challenges.

[0689] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring schedule information of the user, means for acquiring current location information, means for proposing multiple means of transportation and required times to the next scheduled location based on the schedule information and the current location information, means for adding travel time to the schedule based on the means of transportation selected by the user, means for setting a warning based on the set travel time, means for recognizing the emotional state of the user using emotion analysis means, and means for proposing adjustments to the means of transportation and the schedule based on the emotional state. This allows the user to select the optimal means of transportation based on real-time operation information, and further enables flexible suggestions based on the user's emotional state, thereby reducing stress and enabling smooth schedule management.

[0690] "User schedule information" refers to information about events and schedules that a user has registered in a schedule management system or calendar application.

[0691] "Current Location Information" means real-time geographic location data obtained by a User's Device through GPS or other location services.

[0692] "Multiple means of transportation" refers to the means of transportation available for the user to travel (e.g., train, bus, car, walking, etc.).

[0693] "Travel time" refers to the time required to travel from the current location to the destination, and is calculated based on calculation results that differ for each means of transportation.

[0694] "Travel time" refers to the time required to travel by the means of transportation selected by the user.

[0695] "Warnings" are alarms or notifications that are set to help users stay on schedule, and are issued before departure time or just before an event.

[0696] "Emotion analysis means" is a technology that uses sensors such as cameras and microphones to analyze the user's facial expressions and tone of voice, and recognizes the user's emotional state (e.g., joy, anger, sadness, surprise, etc.).

[0697] "Emotional state" refers to the user's psychological and emotional state as detected using emotion analysis means.

[0698] "Suggestions" are notifications and advice that suggest optimal travel options or schedule adjustments based on the user's emotional state and other real-time information.

[0699] System Overview

[0700] This invention is a schedule management system that uses AI, and provides a means to acquire a user's schedule information, suggest transportation methods and travel times based on current location information, and add them to the schedule. This system is composed of a server, terminals, and users, and by incorporating emotion analysis means, it is also possible to make flexible suggestions based on the user's emotional state.

[0701] System Configuration

[0702] The system is configured using the following hardware and software.

[0703] Device: Mobile devices such as smartphones and tablets

[0704] Server: Cloud Server

[0705] Sentiment analysis tools: camera, microphone, and sentiment analysis algorithms

[0706] Location information acquisition method: GPS sensor

[0707] Data acquisition and proposal method: Data communication using API (e.g., Google Calendar API, map service API)

[0708] Real-time information acquisition method: API that periodically acquires operation information (e.g., map service API)

[0709] System Operation

[0710] The device retrieves event data from the user's calendar or schedule application via an API. This system works with Google Calendar and other calendar services. The retrieved data includes the event start time, end time, location, and event name.

[0711] The device acquires the user's current location using a GPS sensor and stores the acquired information in its internal memory. The device then sends the user's current location to a server. The server extracts the next planned destination from a database and calculates the means of transportation and route based on the location information using a map service API (e.g., Google Maps API). The server then sends the calculation results to the device, which then suggests multiple means of transportation and their estimated travel times to the user.

[0712] When the user selects their preferred mode of transportation on the device, the information is sent to the server, which adds the travel time of the selected mode to the schedule, and the updated schedule is returned to the device, which notifies the user.

[0713] In addition, the device will set a warning 10 minutes before the scheduled departure time and notify the user of the set warning. The device will periodically check the map service API and update the schedule and warning time based on real-time service information.

[0714] Using emotion analysis, the device analyzes the user's facial expressions and voice data through a camera and microphone to recognize the user's emotional state in real time. The analysis results are sent to a cloud server, which then suggests transportation options and adjusts the schedule based on the user's emotional state.

[0715] Specific examples

[0716] For example, consider a case where a user travels to Shinjuku Ward for a meeting scheduled for 3 p.m. The user's current location is Chiyoda Ward, and if the emotion analysis means detects the user's stress, the server will suggest taking a taxi in addition to the train. If the user selects a taxi, the estimated taxi travel time is calculated to be 1:45 p.m., and the device will set an alert accordingly. Before leaving, the device will make suggestions such as "listen to relaxing music."

[0717] Prompt Sentence Examples

[0718] "Create a program that provides users with schedules and transportation options. The system also makes suggestions based on the user's emotional state."

[0719] "Please explain in detail how your schedule management system works, combined with the emotion engine. Please also include the optimal route from the user's current location to their destination."

[0720] In this way, the system of the present invention allows users to efficiently manage their schedules without any hassle. Furthermore, flexible suggestions based on the user's emotional state reduce stress and allow for comfortable travel and preparation.

[0721] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0722] Program processing flow

[0723] Step 1: Get the user's schedule information

[0724] Process flow:

[0725] The device accesses the user's calendar application and obtains schedule information through the API.

[0726] input:

[0727] The user's calendar application access permissions and API key.

[0728] Data processing / calculation:

[0729] The schedule data obtained through the API is analyzed and stored in an internal database.

[0730] output:

[0731] Schedule data captured (e.g., event name, start time, end time, location).

[0732] Specific behavior:

[0733] The device uses the Google Calendar API to retrieve the user's schedule information and store it in an internal database.

[0734] Step 2: Get your current location

[0735] Process flow:

[0736] The device uses the GPS sensor to obtain the user's current location and stores it in its internal memory.

[0737] input:

[0738] GPS sensor data.

[0739] Data processing / calculation:

[0740] The acquired latitude and longitude information is analyzed to determine the current location.

[0741] output:

[0742] Current location information (latitude and longitude).

[0743] Specific behavior:

[0744] The device periodically uses the GPS sensor to obtain the user's current location and stores it in its internal memory.

[0745] Step 3: Identify the next planned location and calculate the optimal route

[0746] Process flow:

[0747] The server identifies the next planned location and uses the map service's API to calculate the optimal route and travel time from the current location to the next planned location.

[0748] input:

[0749] Current location information, schedule information, map service API key.

[0750] Data processing / calculation:

[0751] Use the map service API to calculate multiple means of transportation (e.g., train, bus, car, walking) and the travel time required for each.

[0752] output:

[0753] Travel time and route information for each mode of transportation.

[0754] Specific behavior:

[0755] The server sends a request to the Google Maps API to calculate the optimal route from the current location to the next destination and the travel time for multiple modes of transportation.

[0756] Step 4: Select your mode of transport and add travel time

[0757] Process flow:

[0758] The terminal displays a notification prompting the user to select a means of transportation and adds travel time to the schedule based on the user's selection.

[0759] input:

[0760] Travel time and route information for each mode of transportation.

[0761] Data processing / calculation:

[0762] Add and recalculate travel time to the schedule based on the user's selected mode of travel.

[0763] output:

[0764] Updated schedule information.

[0765] Specific behavior:

[0766] The terminal displays a notification to the user to select a mode of transportation such as "train," "bus," or "taxi," and updates the schedule based on the selected mode of transportation.

[0767] Step 5: Set an alarm

[0768] Process flow:

[0769] The device will set an alarm 10 minutes before the scheduled departure time.

[0770] input:

[0771] Updated schedule information.

[0772] Data processing / calculation:

[0773] Calculate 10 minutes before departure time and set the alarm.

[0774] output:

[0775] Pre-departure warning alarm.

[0776] Specific behavior:

[0777] The device will set a warning alarm 10 minutes before departure time and send a notification to the user saying, "10 minutes left until departure."

[0778] Step 6: Reflecting real-time information

[0779] Process flow:

[0780] The device periodically checks real-time operation information and updates schedules and alarm times.

[0781] input:

[0782] Real-time information, updated schedules.

[0783] Data processing / calculation:

[0784] Recalculate and update schedules and alarms based on delay and operational information.

[0785] output:

[0786] Updated schedule information and reminders.

[0787] Specific behavior:

[0788] The device checks the map service API every 15 minutes to obtain delay and operation information and update schedules and alarms.

[0789] Step 7: Reminders

[0790] Process flow:

[0791] The device will notify the user of travel and preparation reminders.

[0792] input:

[0793] Updated schedule information, alarm 2 hours before departure.

[0794] Data processing / calculation:

[0795] Generate reminders before you leave or before an event.

[0796] output:

[0797] Reminder notifications.

[0798] Specific behavior:

[0799] The device will send reminder notifications such as "Leave in 10 minutes. Start getting ready" or "Meeting is about to start. Set your smartphone to silent mode."

[0800] Step 8: Emotion Recognition with the Emotion Engine

[0801] Process flow:

[0802] The device uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize their emotional state.

[0803] input:

[0804] Sensor data from cameras and microphones.

[0805] Data processing / calculation:

[0806] Emotion analysis algorithms are used to determine the user's emotional state in real time.

[0807] output:

[0808] The user's emotional state (e.g., happy, anger, sadness, surprise).

[0809] Specific behavior:

[0810] The device analyzes the user's facial expressions and tone of voice through a camera and microphone, and recognizes the user's emotional state, such as "feeling stressed."

[0811] Step 9: Emotion-Based Suggestions

[0812] Process flow:

[0813] The server generates optimal travel and schedule adjustment suggestions based on the emotional state.

[0814] input:

[0815] User's emotional state, schedule information, real-time traffic information.

[0816] Data processing / calculation:

[0817] Apply algorithms that recommend transportation and schedule adjustments based on emotional state.

[0818] output:

[0819] Proposal Notice.

[0820] Specific behavior:

[0821] Based on the analysis result that "the user is feeling stressed," the server generates suggestions such as "taking a taxi" or "listening to relaxing music" and notifies the user via the terminal.

[0822] (Application example 2)

[0823] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0824] While conventional schedule management systems can efficiently adjust schedules by taking into account users' travel times and real-time information, they have the problem of being unable to flexibly respond to users' emotional states. In particular, when managing schedules for store staff, if staff stress and fatigue are not properly managed, work efficiency and the quality of service may decline.

[0825] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring the user's schedule information, means for acquiring current location information, means for proposing multiple means of transportation and required times to the next scheduled location based on the schedule information and the current location information, means for adding travel time to the schedule based on the transportation means selected by the user, means for setting an alarm based on the set travel time, means for performing emotion analysis using a camera and microphone to recognize the user's emotional state, and means for adjusting the schedule and changing the notification content based on the user's emotional state. This enables flexible schedule adjustment and notification that takes the user's emotional state into consideration, enabling store staff to perform their work efficiently and provide high-quality services.

[0826] "User" refers to a person who uses the system.

[0827] "Schedule information" refers to data that records a user's schedules and appointments.

[0828] "Current location information" refers to information about the user's current location, and is mainly obtained using a GPS sensor.

[0829] "Transportation" refers to the means of transportation used by the user to travel to the next scheduled location.

[0830] "Travel time" refers to the time it takes for the user to travel from their current location to their next scheduled location.

[0831] "Emotional state" refers to a user's mental and emotional state, including emotions such as joy, anger, sadness, and surprise.

[0832] A "camera" is a device that captures images or videos using an optical sensor.

[0833] A "microphone" is a device for picking up sound, and the user's voice is used for emotion analysis.

[0834] "Emotion analysis" refers to the process of analyzing a user's facial expressions and tone of voice to recognize their emotional state.

[0835] An "alarm" is a function that notifies the user at a set time.

[0836] "Notification content" refers to the content of the message sent to the user as an alarm or reminder.

[0837] "Real-time information" refers to data that changes over time, such as the latest operation and traffic information.

[0838] The term "system" refers to a collection of hardware and software required to realize the various functions of the present invention.

[0839] The present invention is a system for improving the efficiency of a user's schedule management, and is embodied in the following specific form.

[0840] System configuration

[0841] The system includes means for acquiring the user's schedule information, means for acquiring current location information, means for suggesting multiple means of transportation and travel times to the next scheduled location, means for adding travel time to the schedule based on the means of transportation selected by the user, means for setting an alarm based on the set travel time, a camera and microphone and emotion analysis means for recognizing the user's emotional state, means for adjusting the schedule based on the user's emotional state, and means for changing the content of notifications.

[0842] Program processing overview

[0843] The system performs processing using the following hardware and software.

[0844] Hardware

[0845] 1. Smart glasses: display information and track user behavior.

[0846] 2. Camera: Used to recognize the user's facial expressions.

[0847] 3. Microphone: Uses the user's tone of voice for sentiment analysis.

[0848] 4. GPS sensor: Used to obtain current location information.

[0849] software

[0850] 1. Schedule management API: Obtain and manage user schedules (e.g., Google Calendar API).

[0851] 2. Map service API: Calculates the means of transportation and travel time to the next scheduled destination (e.g., Google Maps API).

[0852] 3. Emotion Recognition API: Analyze the user's emotional state (e.g., Emotion Recognition API).

[0853] 4. Notification system: Send reminders and alarms to users.

[0854] Data processing and calculation

[0855] The system processes and calculates the data as follows:

[0856] 1. Get user schedule:

[0857] The server retrieves the user's schedule information by calling the schedule management API, which downloads the user's schedule and appointment details.

[0858] 2. Obtaining current location information:

[0859] The server uses a GPS sensor to obtain the user's current location, which provides real-time location information about where the user is currently located.

[0860] 3. Travel options and travel times:

[0861] The server calls the map service API and calculates the travel time and options for travelling from the current location to the next destination, such as train, bus, car, and walking.

[0862] 4. Emotional state recognition and analysis:

[0863] The server collects data from the camera and microphone and uses an emotion recognition API to analyze the user's emotional state, thereby identifying the emotion the user is currently feeling (e.g., joy, anger, sadness, surprise, etc.).

[0864] 5. Adjust schedule and notification content:

[0865] The server adjusts the schedule based on the user's emotional state, for example by adding breaks to the schedule if it senses that the user is tired, and it also uses a notification system to send emotionally sensitive notifications.

[0866] Specific examples

[0867] As a specific example, consider a situation where store staff member A is scheduled to work the cash register from 2 p.m. and put merchandise on display from 3 p.m. The user's smart glasses calculate the time it takes to move around the store and the store's congestion status in real time, and reflect this in the schedule. Furthermore, if the analysis shows that staff member A is feeling "tired" or "stressed," the glasses will suggest adding more break time or notify the employee of an adjustment to the division of labor.

[0868] Prompt Sentence Examples

[0869] "Based on the new schedule, Staff A's task starting at 3 p.m. is to stock the shelves. However, if Staff A is feeling stressed, how do you adjust their schedule and alarms?"

[0870] This allows users to efficiently manage their schedules without hassle and allows them to respond flexibly to their emotional state. This system reduces stress for users and enables them to carry out their work comfortably.

[0871] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0872] Step 1:

[0873] The device calls the schedule management API to obtain the user's schedule information. This process involves sending a request to the API to obtain data including details of the user's schedule and appointments. Once the data is obtained, the device stores it in its internal database.

[0874] Step 2:

[0875] The device uses the GPS sensor to obtain the user's current location information. In this process, the current latitude and longitude data is obtained from the GPS sensor to obtain the coordinates of the user's current location. The obtained coordinate data becomes the input for proceeding to the next processing step.

[0876] Step 3:

[0877] The server proposes multiple means of transportation and the required travel time to the next scheduled location based on the schedule information and current location information. In this process, it calls the map service API to obtain the route and means of transportation (e.g., train, bus, car, walking, etc.) from the current location to the scheduled location, and calculates the required travel time for each. The server sends the obtained route information to the terminal and proposes multiple means of transportation to the user.

[0878] Step 4:

[0879] The user selects a desired transportation method from the transportation methods proposed through the terminal. In this process, travel time data is acquired based on the transportation method selected by the user. The terminal adds the selected transportation method and travel time to the schedule and updates it.

[0880] Step 5:

[0881] The device sets an alarm based on the set travel time. In this process, the alarm is set a certain time (for example, 10 minutes) before the scheduled departure time based on the travel time data. The set alarm information is saved in the device, and the user is notified when the departure time approaches.

[0882] Step 6:

[0883] The server uses a camera and microphone to capture the user's emotional state. This process involves capturing facial expression data through the camera and recording voice tone through the microphone. It then calls an emotion recognition API to analyze these data and identify the user's emotional state (e.g., joy, anger, sadness, surprise, etc.).

[0884] Step 7:

[0885] The server adjusts the schedule based on the user's emotional state. This process involves making necessary schedule changes depending on the user's emotional state. For example, if the server determines that the user is "tired," it adds a break to the schedule. It also changes work allocation and adjusts tasks as necessary.

[0886] Step 8:

[0887] The device changes the notification content according to the user's emotional state. This process flexibly adjusts the content of reminder and alarm notifications based on the user's emotional state. For example, if the device determines that the user is "feeling stressed," it sends a message to soften the tone of the notification.

[0888] Step 9:

[0889] The device obtains real-time traffic information and updates schedules and alarms. This process involves calling the traffic information API to obtain the latest traffic information and checking whether changes to the mode of travel or travel time are necessary. If necessary, the device automatically updates schedules and alarms and notifies the user.

[0890] Step 10:

[0891] The device monitors the remaining battery level and sends a notification to the user urging charging if it falls below a predetermined threshold. The battery monitoring system periodically checks the remaining battery level and, if it falls below a threshold, sends a notification to the user saying, "The battery is low. Please start charging."

[0892] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0893] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0894] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0895] [Third embodiment]

[0896] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0897] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0898] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0899] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0900] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0901] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0902] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0903] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0904] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0905] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0906] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0907] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0908] This invention is a smartphone schedule management system that uses AI to acquire user schedule information, automatically calculate travel time, add travel time to the schedule, set alarms, and update real-time information. The specific configuration of each part and operation of this system is explained below.

[0909] composition

[0910] 1. Get user schedule

[0911] The device retrieves event data from the user's calendar or schedule application via an API. This system is compatible with Google Calendar and other calendar services, for example.

[0912] 2. Travel time and route calculation

[0913] The device acquires the user's current location using the GPS sensor, then extracts the next scheduled location from the schedule and identifies its location information.

[0914] Based on this, the server accesses the API of the map service (for example, Google Maps API) and calculates the optimal means of transportation (train, bus, car, walking, etc.) from the current location to the next destination and the required time for each. After the server obtains this information, it proposes multiple means of transportation and their required times to the user.

[0915] 3. Add travel time to your schedule

[0916] When the user selects their preferred mode of transportation, the device adds the travel time to the schedule. For example, if the user's current location is Chiyoda Ward, Tokyo, and their next destination is Shinjuku Ward, and they select the train, the calculated travel time of 30 minutes will be reflected in the schedule.

[0917] 4. Set an alarm

[0918] The device will set an alarm 10 minutes before the scheduled departure time, so if the departure is scheduled for 2:30 PM, the alarm will be set for 2:20 PM.

[0919] 5. Real-time information

[0920] The device periodically checks real-time train and bus information to obtain delay information. Based on this information, it automatically updates schedules and alarm times as needed. For example, if a train is delayed by 10 minutes, the device will advance the departure time and reset the alarm.

[0921] 6. Reminder notifications

[0922] The device notifies the user of travel and preparation reminders. For example, at 2:10 p.m., it sends a notification saying, "10 minutes until departure. Please start getting ready." The device also sends a notification urging the user to charge the battery if the battery level is low. 10 minutes before the start of a meeting, it also sends a notification saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0923] Specific examples

[0924] For example, suppose a user has a meeting scheduled for 3:00 PM in Shinjuku Ward. The user's current location is Chiyoda Ward, and the schedule is retrieved from a calendar application. The device then identifies the user's current location using GPS, and the server uses a map service API to calculate the optimal route and travel time from Chiyoda Ward to Shinjuku Ward.

[0925] If the user chooses to travel by train, the device calculates the travel time as 30 minutes and adds a departure time of 2:30 p.m. to the schedule. It also sets an alarm for 2:20 p.m. If the train is delayed, the device checks real-time information, automatically updates the schedule and alarm, and notifies the user.

[0926] In this way, the system of the present invention allows users to efficiently manage their schedules without hassle, helping to avoid mistakes and reducing stress.

[0927] The processing flow will be explained below.

[0928] Step 1:

[0929] The device retrieves event data from the user's calendar or schedule application via an API. For example, it sends a request to the calendar API to retrieve a schedule for a meeting starting at 3:00 PM on October 15, 2023.

[0930] Step 2:

[0931] The device acquires the user's current location information using a GPS sensor. For example, the device identifies that the user's current location is Chiyoda Ward, Tokyo.

[0932] Step 3:

[0933] The device identifies the next scheduled location from the schedule data and extracts its location information. For example, it recognizes that the next scheduled location is a "conference room in Shinjuku Ward, Tokyo."

[0934] Step 4:

[0935] The server accesses the API of a map service (e.g., Google Maps API) based on the current location and the next planned destination, and calculates the optimal means of transportation and the required time for each. For example, it obtains information such as 30 minutes by train, 45 minutes by bus, and 50 minutes by bicycle.

[0936] Step 5:

[0937] The terminal presents the acquired information on the means of transportation and the required time to the user, and the user selects the desired means. For example, the user selects "train."

[0938] Step 6:

[0939] The device will then add the travel time to the schedule based on the selected mode of transportation, updating the schedule to, for example, "Depart at 2:30 PM."

[0940] Step 7:

[0941] The device will set an alarm 10 minutes before departure time, for example, "Set alarm for 2:20 p.m."

[0942] Step 8:

[0943] The device periodically retrieves real-time data, including operational information, and checks train and bus delay information. For example, it retrieves information that a train is 10 minutes late.

[0944] Step 9:

[0945] The server automatically updates schedules and alarms based on the acquired operation information. For example, it may move the departure time forward to "2:15 PM" and reset the alarm to "2:05 PM."

[0946] Step 10:

[0947] The device sends a reminder to the user to prepare for travel. For example, at 2:10 p.m., the device sends a notification saying, "There are 10 minutes until departure. Please start preparing."

[0948] Step 11:

[0949] The device checks the battery status and sends a notification to encourage charging if the remaining charge falls below a certain threshold. For example, "I detected that the battery charge is less than 20% and sent a notification saying 'Please charge.'"

[0950] Step 12:

[0951] The device will remind the user to set the silent mode 10 minutes before an important event (e.g., a meeting) starts. For example, a notification will be sent 10 minutes before the start of the meeting saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0952] Through these steps, users can efficiently and accurately manage their schedules, enabling smooth travel and preparation without the need for manual operations.

[0953] Example 1

[0954] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0955] Conventional schedule management systems require users to manually check current location information and transportation options and reflect them in their schedules. This makes schedule management cumbersome and makes it difficult to efficiently complete plans. Furthermore, they often lack the ability to provide real-time traffic information or appropriate alerts or notifications based on remaining battery power, increasing the risk of users missing important appointments.

[0956] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0957] In this invention, the server includes means for acquiring the user's schedule information, means for acquiring location information, means for proposing multiple means of transportation and travel times to the next scheduled location based on the schedule information and location information, means for adding travel time to the schedule based on the transportation means selected by the user, means for setting an alarm based on the set departure time, means for acquiring traffic operation information in real time and updating the schedule and alarm, and means for monitoring the remaining battery level and sending a notification urging the user to charge the battery when the battery falls below a predetermined threshold. This allows the user to efficiently manage their schedule without having to perform complicated manual operations, significantly reducing the risk of missing important appointments.

[0958] "User schedule information" is data relating to the user's schedule, obtained from a calendar application or a schedule application.

[0959] "Location Information" means data about a user's current latitude and longitude obtained using a GPS sensor or other positioning technology.

[0960] "Transportation" refers to the means used for travel, including trains, buses, cars, and walking.

[0961] "Travel time" is the estimated time it will take to travel from the current location to the next scheduled location.

[0962] "Real-time transportation operation information" is data that obtains the operation status of transportation in real time and reflects information such as delays and cancellations.

[0963] "Battery level monitoring" is a feature that measures the remaining battery level of smartphones and other mobile devices and notifies you when it reaches a certain level.

[0964] "Add to schedule" means to reflect new schedules or information in the user's schedule.

[0965] "Set an alarm" is a function that notifies the user when a specific time or condition is met.

[0966] The "notification to encourage charging" is a notification to encourage the user to charge when the remaining battery power is low.

[0967] This invention is a smartphone schedule management system that uses AI, which acquires the user's schedule information, suggests transportation methods along with location information, adds travel time to the schedule, sets alarms, and automatically updates real-time information. The specific configuration of each part and operation of this system is explained below.

[0968] Get user's appointment information

[0969] The device accesses a calendar application (e.g., a general-purpose calendar app) based on the user's authentication information and obtains the user's event information through the API. Once this operation is complete, the obtained event data is saved in the device's database. Specifically, the device communicates with the calendar API using the user's OAuth token, receives the obtained data in JSON format, and stores it in the local database.

[0970] Get the user's current location and identify the next scheduled location

[0971] The device activates the GPS sensor to obtain the user's current location. Then, the device extracts the location information of the next scheduled event from the most recent scheduled event and determines the latitude and longitude. For example, if the current location is Chiyoda Ward, Tokyo, and the next scheduled event location is Shinjuku Ward, the device will obtain the current location using GPS and the location information of the next scheduled event through a geocoding service.

[0972] Calculate travel route and travel time

[0973] The device sends this location information to the server. The server calls a map service API (for example, a general-purpose map service API) to calculate the optimal means of transportation from the current location to the next scheduled destination and the required travel time. The calculation results are returned to the device in JSON format. For example, the device sends a request to the map service API through the server to calculate the required travel time for various modes of transportation, such as train, bus, car, and walking.

[0974] Choosing transportation methods and reflecting them in your schedule

[0975] The device displays options for transportation and travel times to the user. When the user selects their preferred transportation method, the device reflects that travel time in the schedule. For example, if the user selects traveling by train, the device calculates the travel time as 30 minutes and updates the schedule to depart at 2:30 PM. The device then sets the alarm time to 2:20 PM based on this information.

[0976] Reflecting real-time information

[0977] The device obtains real-time traffic information at regular intervals and updates schedules and alarms as necessary. Specifically, the device uses a traffic information API to check real-time traffic information, and if there is delay information, it resets the departure time or alarm and notifies the user.

[0978] Reminders and notifications

[0979] The device sends users reminders to travel and prepare. It also notifies users when the battery is low, urging them to charge the device. For example, at 2:10 p.m., it sends a notification 10 minutes before departure, and 10 minutes before a meeting starts, it sends a notification saying, "The meeting is about to start. Please set your smartphone to silent mode."

[0980] Specific examples

[0981] For example, consider a case where a user has a meeting scheduled in Shinjuku Ward at 3:00 PM. The user's current location is Chiyoda Ward, and this appointment is retrieved from the configured calendar application. The device then uses GPS to determine the user's current location, and the server uses a general-purpose map service API to calculate the optimal route and travel time from Chiyoda Ward to Shinjuku Ward.

[0982] If the user chooses to travel by train, the device calculates the travel time as 30 minutes, updates the schedule to depart at 2:30 p.m., and sets an alarm for 2:20 p.m. If the train is delayed, the device checks real-time information, automatically updates the schedule and alarm, and notifies the user.

[0983] In this way, the system of the present invention allows users to efficiently manage their schedules without having to perform complicated manual operations, significantly reducing the risk of missing important appointments.

[0984] Example prompt sentence:

[0985] "I have a meeting in Shinjuku Ward at 3 PM. My current location is in Chiyoda Ward. Please calculate and suggest the best transportation method and travel time, and update my schedule accordingly. Also, please set an alarm 10 minutes before departure time, and update my schedule as needed based on real-time traffic information."

[0986] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0987] Step 1:

[0988] The device accesses a calendar API (for example, a general-purpose calendar API) based on the user's authentication information and obtains the user's schedule information. Specifically, it makes an API call using an OAuth token and receives the obtained schedule information in JSON format. User authentication information is required as input, and event data is obtained as output. This data is saved in a database within the device.

[0989] Step 2:

[0990] The device activates the GPS sensor and obtains the user's current location information. Specifically, it uses the GPS module to measure the current latitude and longitude. The input requires location measurement using the GPS module, and the output is the latitude and longitude data of the current location.

[0991] Step 3:

[0992] The device extracts the location information of the next scheduled event from the acquired schedule information. Specifically, it parses the address information from the next scheduled event and converts it to latitude and longitude using a geocoding API. The address of the next scheduled event is required as input, and the latitude and longitude data of the scheduled location is obtained as output.

[0993] Step 4:

[0994] The device sends latitude and longitude information of the current location and the next planned location to the server. The server calls a map service API (for example, a general-purpose map service API) and calculates the multiple means of transportation from the current location to the next planned location and the required travel time. Specifically, it uses the Directions API to obtain the route and time for each means. The latitude and longitude of the current location and the planned location are required as input, and the required travel time and route information for each means of transportation are obtained as output. The calculation results are returned to the device in JSON format.

[0995] Step 5:

[0996] The device presents the user with multiple transportation options and their respective travel times. When the user selects their preferred transportation option, the device reflects the selected travel time in the schedule. Specifically, the device displays a list of options through a user interface and receives the user's selection. The inputs are the travel times for each transportation option and the user's selection, and the output is an updated schedule.

[0997] Step 6:

[0998] The terminal sets an alarm 10 minutes before the scheduled departure time. Specifically, the alarm is set using the system's alarm manager. The departure time data is required as input, and the alarm setting is output.

[0999] Step 7:

[1000] The device periodically retrieves real-time traffic information and updates schedules and alarms. Specifically, it uses a traffic information API to check the latest traffic conditions and reset schedules and alarms as needed. Real-time traffic data is required as input, and updated schedules and alarms are obtained as output.

[1001] Step 8:

[1002] The device sends travel and preparation reminders to the user. Specifically, it uses push notifications to provide pre-departure reminders and low battery notifications. The inputs required are the timing of scheduled events and battery level data, and the output is the reminder notification sent to the user.

[1003] (Application example 1)

[1004] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1005] The purpose of this invention is to efficiently manage delivery staff schedules and optimize their travel in food delivery services. When there are multiple delivery destinations, conventional methods make it cumbersome to determine routes and manage travel times, and it is difficult to quickly respond to changes in traffic conditions. Therefore, there is a need for a system that can improve delivery efficiency and accuracy.

[1006] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1007] In this invention, the server includes means for acquiring schedule information of the user, means for acquiring current location information, means for proposing multiple means of transportation and required travel times based on the schedule information and current location information, means for adding travel time to the schedule based on the means of transportation selected by the user, means for setting an alarm based on the set travel time, and means for identifying the current location of the delivery person, optimizing the route to the next delivery destination, and updating the delivery schedule in real time, thereby enabling real-time updating of the delivery schedule and efficient route proposals.

[1008] The "means for acquiring user schedule information" is a function that enables an electronic device to automatically collect event data from a user's calendar or schedule.

[1009] "Means for obtaining current location information" means a function for determining the current geographic location of a user or device using GPS or other location information systems.

[1010] The "means for proposing multiple means of transportation and required travel times" is a function for calculating options for transportation (e.g., walking, car, train, etc.) and the required travel times for each, and presenting them to the user.

[1011] The "means for adding travel time to the schedule" is a function for reflecting the travel time calculated based on the means of transportation selected by the user in the schedule.

[1012] The "means for setting an alarm" is a function for sending a notification to the user at a specified time, and serves to prompt the user to leave or get ready.

[1013] "Means for determining the delivery person's current location" refers to a function that uses GPS or a location information system to determine the delivery person's current geographic location in real time.

[1014] The "means for optimizing the route to the next delivery destination" is a function for calculating and presenting the optimal and shortest route to the delivery destination.

[1015] "Means for updating delivery schedules in real time" refers to a function for updating delivery schedules at any time based on current traffic conditions and delivery progress.

[1016] "Means for obtaining real-time operation information and updating schedules and alarms" refers to a function for obtaining traffic conditions and operation information in real time and adjusting schedule and alarm settings based on this information.

[1017] "Means for monitoring the remaining battery level and sending a notification urging the user to charge when the battery falls below a specified threshold" is a function for constantly monitoring the battery status of the device and notifying the user to charge when the remaining battery level falls below a certain level.

[1018] The present invention provides a system for efficiently managing delivery staff schedules and optimizing their travel in a food delivery service. The system includes a means for acquiring a user's schedule information, a means for acquiring current location information, a means for proposing multiple means of transportation and required travel times, a means for adding travel time to the schedule, a means for setting an alarm, a means for identifying the delivery staff's current location, a means for optimizing the route to the next delivery destination, and a means for updating the delivery schedule in real time.

[1019] Configuration and operation

[1020] 1. Obtaining user schedule information

[1021] The server receives event data from the delivery person's schedule management application via an API. This system is compatible with general calendar services, for example.

[1022] 2. Obtaining current location information

[1023] The device uses a GPS sensor to acquire the delivery person's current location, and then sends the delivery person's current location information to the server.

[1024] 3. Proposing multiple transportation methods and travel times

[1025] The server calculates multiple means of transportation (e.g., car, bicycle, walking, etc.) to the next delivery destination and the required time for each, based on the acquired schedule information and current location information. The server uses a map service API (e.g., map service API) to suggest multiple means of transportation and their required time to the delivery person.

[1026] 4. Add travel time to your schedule

[1027] When a delivery person selects their preferred mode of transportation, the server adds the travel time to the schedule. For example, if the current location is somewhere and the next delivery destination is somewhere else, and the delivery person selects a car, the calculated travel time is reflected in the schedule.

[1028] 5. Set an alarm

[1029] The server sets an alarm 10 minutes before the scheduled departure time, allowing the delivery person to prepare for the next delivery.

[1030] 6. Real-time delivery schedule updates

[1031] The device periodically checks real-time traffic information to obtain traffic conditions and delay information. The server automatically updates schedules and alarm times as needed based on the information obtained. For example, if there is traffic congestion, the server will adjust departure times and reset alarms.

[1032] Hardware and software used

[1033] This system uses smart glasses equipped with a GPS sensor as the terminal, and the server calculates routes and retrieves schedule information using the Google Maps API and general calendar APIs.

[1034] Specific examples

[1035] For example, if a delivery person has multiple delivery destinations, the system calculates the travel time from the current location to each destination in real time and suggests an efficient route. The schedule is updated based on the transportation method selected by the delivery person, and an alarm is set for the departure time. Traffic conditions and delay information are obtained in real time, and the schedule and alarms are automatically updated as needed.

[1036] Prompt Sentence Examples

[1037] Here is an example prompt:

[1038] Locate your delivery drivers with GPS and calculate the best route and travel time to their next delivery. Update schedules in real time and set alarms based on their next due time. Also, monitor traffic conditions and reschedule if delays occur.

[1039] This allows the system of the present invention to support delivery personnel in efficient delivery and improve the reliability and accuracy of delivery services.

[1040] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1041] Step 1:

[1042] The server retrieves schedule information from the delivery driver's schedule management application via an API. The inputs include the delivery driver's account information and API key. The output is the schedule information in JSON format or other structured data format. The server parses this data to obtain the delivery order and location information.

[1043] Step 2:

[1044] The terminal acquires the delivery person's current location information using a GPS sensor. The input is a signal obtained from a GPS device, and the output is real-time latitude and longitude data. The terminal then transmits the acquired geographic information to a server.

[1045] Step 3:

[1046] The server proposes multiple means of transportation to the next delivery destination and the required travel time based on the acquired schedule information and current location information. This step requires the latitude and longitude of the current location and schedule information as input, and the required travel time for each means of transportation (e.g., car, bicycle, walking) as output. The server uses a map service API to calculate the optimal route and means of transportation.

[1047] Step 4:

[1048] The user selects the desired transportation method from the presented options. The input is the user's selection information, and the output is the data of the selected transportation method. The server recalculates the required travel time based on the selected transportation method and adds the travel time to the next delivery destination to the schedule.

[1049] Step 5:

[1050] The server sets an alarm based on the set travel time. The input is the updated schedule information and travel time, and the output is the alarm setting data. Based on this, the server sets a notification to be sent before the departure time.

[1051] Step 6:

[1052] The terminal periodically checks real-time traffic information and obtains traffic conditions and delay information. As input, it sends a request to the traffic information API, and as output, it obtains the latest traffic information. The terminal then sends this to the server.

[1053] Step 7:

[1054] The server updates the schedule based on the acquired real-time operation information. This step requires the delivery schedule and real-time operation information as input, and obtains the updated schedule data as output. The server also resets the alarm time if necessary and sends a notification to the user.

[1055] This allows delivery personnel to efficiently manage their delivery schedules and respond quickly to changes in traffic conditions.

[1056] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1057] This invention is a smartphone schedule management system that uses AI. It acquires the user's schedule information, calculates travel time, adds travel time to the schedule, sets alarms, and automatically updates real-time information. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to flexibly suggest schedules and provide notifications according to the user's emotional state. The specific components and operations of this system are described below.

[1058] composition

[1059] 1. Get user schedule

[1060] The device retrieves event data from the user's calendar or schedule application via an API. This system is compatible with Google Calendar and other calendar services, for example.

[1061] 2. Travel time and route calculation

[1062] The device acquires the user's current location using the GPS sensor, then extracts the next scheduled location from the schedule and identifies its location information.

[1063] Based on this, the server accesses the API of the map service (for example, Google Maps API) and calculates the optimal means of transportation (train, bus, car, walking, etc.) from the current location to the next destination and the required time for each. After the server obtains this information, it proposes multiple means of transportation and their required times to the user.

[1064] 3. Add travel time to your schedule

[1065] When the user selects their preferred mode of transportation, the device adds the travel time to the schedule. For example, if the user's current location is Chiyoda Ward, Tokyo, and their next destination is Shinjuku Ward, and they select the train, the calculated travel time of 30 minutes will be reflected in the schedule.

[1066] 4. Set an alarm

[1067] The device will set an alarm 10 minutes before the scheduled departure time, so if the departure is scheduled for 2:30 PM, the alarm will be set for 2:20 PM.

[1068] 5. Real-time information

[1069] The device periodically checks real-time train and bus information to obtain delay information. Based on this information, it automatically updates schedules and alarm times as needed. For example, if a train is delayed by 10 minutes, the device will advance the departure time and reset the alarm.

[1070] 6. Reminder notifications

[1071] The device notifies the user of travel and preparation reminders. For example, at 2:10 p.m., it sends a notification saying, "10 minutes until departure. Please start getting ready." The device also sends a notification urging the user to charge the battery if the battery level is low. 10 minutes before the start of a meeting, it also sends a notification saying, "The meeting is about to start. Please set your smartphone to silent mode."

[1072] Combining Emotion Engines

[1073] 1. Emotion Engine Configuration

[1074] The device is equipped with an emotion engine that uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice, recognizing the user's emotional state (e.g., joy, anger, sadness, surprise, etc.) in real time.

[1075] 2. Emotion-based transportation suggestions

[1076] The server analyzes the user's emotional state and suggests transportation options based on the results. For example, if the server detects that the user is feeling stressed, it will prioritize more comfortable transportation options (such as taxis).

[1077] 3. Adjust your schedule based on emotions

[1078] The device will flexibly adjust the schedule based on the user's emotional state. For example, if it detects that the user is tired, it will suggest flexible schedules and break times.

[1079] 4. Tailoring notification content based on emotions

[1080] The device changes the content of the reminder notification depending on the user's emotional state. For example, if the device detects that the user is nervous, it will soften the tone of the reminder.

[1081] Specific examples

[1082] For example, suppose a user has a meeting scheduled in Shinjuku Ward at 3:00 PM. The user's current location is Chiyoda Ward. If the user's emotion engine detects high stress, the server suggests taking a taxi in addition to the train. If the user chooses to take a taxi, the device adds the taxi's travel time to the schedule and sets the departure time to 1:45 PM. The device also suggests relaxing music as a reminder before the user gets in the taxi.

[1083] In this way, the system of the present invention allows users to efficiently manage their schedules without hassle and allows them to respond flexibly to their emotional state. This system reduces users' stress and helps them travel and prepare comfortably.

[1084] The processing flow will be explained below.

[1085] Step 1:

[1086] The device retrieves event data from the user's calendar or schedule application via an API. For example, it sends a request to the calendar API to retrieve a schedule for a meeting starting at 3:00 PM on October 15, 2023.

[1087] Step 2:

[1088] The device acquires the user's current location information using a GPS sensor. For example, the device identifies that the user's current location is Chiyoda Ward, Tokyo.

[1089] Step 3:

[1090] The device identifies the next scheduled location from the schedule data and extracts its location information. For example, it recognizes that the next scheduled location is a "conference room in Shinjuku Ward, Tokyo."

[1091] Step 4:

[1092] The server accesses the API of a map service (e.g., Google Maps API) based on the current location and the next planned destination, and calculates the optimal means of transportation and the required time for each. For example, it obtains information such as 30 minutes by train, 45 minutes by bus, and 50 minutes by bicycle.

[1093] Step 5:

[1094] The terminal presents the acquired information on the means of transportation and the required time to the user, and the user selects the desired means. For example, the user selects "train."

[1095] Step 6:

[1096] The device will then add the travel time to the schedule based on the selected mode of transportation, updating the schedule to, for example, "Depart at 2:30 PM."

[1097] Step 7:

[1098] The device will set an alarm 10 minutes before departure time, for example, "Set alarm for 2:20 p.m."

[1099] Step 8:

[1100] The device periodically checks the traffic information API to get real-time service information, for example, every 10 minutes to check train and bus delay information.

[1101] Step 9:

[1102] The server automatically updates schedules and alarms based on real-time train information. For example, if a train is 10 minutes late, the departure time will be moved forward to 14:10 and the alarm will be reset to 14:00.

[1103] Step 10:

[1104] The device sends a reminder to the user to prepare for travel. For example, at 2:10 p.m., the device sends a notification saying, "There are 10 minutes until departure. Please start preparing."

[1105] Step 11:

[1106] The device checks the battery status and sends a notification to encourage charging if the remaining charge falls below a certain threshold. For example, "I detected that the battery charge is less than 20% and sent a notification saying 'Please charge.'"

[1107] Step 12:

[1108] The device will remind the user to set the silent mode 10 minutes before an important event (e.g., a meeting) starts. For example, a notification will be sent 10 minutes before the start of the meeting saying, "The meeting is about to start. Please set your smartphone to silent mode."

[1109] Adding an Emotion Engine

[1110] Step 13:

[1111] The device activates an emotion engine that uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice. For example, the camera captures the user's face and applies emotion analysis algorithms.

[1112] Step 14:

[1113] The emotion engine analyzes the user's emotional state (e.g., joy, anger, sadness, surprise, etc.) and sends the results to the server. For example, it detects that the user is feeling stressed.

[1114] Step 15:

[1115] The server suggests optimal transportation methods to users based on their emotional state: for example, if a user is feeling stressed, it will prioritize taxis over walking or buses.

[1116] Step 16:

[1117] The user selects the desired transportation method from the suggested transportation methods, for example, "taxi."

[1118] Step 17:

[1119] The device will add travel time to the schedule based on the selected mode of transportation and send notifications that take emotional state into account, such as adding a taxi ride time to the schedule and sending a relaxing music reminder.

[1120] By implementing these steps, users can efficiently manage their schedules without any hassle and can respond flexibly to their emotional state. This system reduces users' stress and supports comfortable travel and preparation.

[1121] Example 2

[1122] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1123] Conventional schedule management systems can suggest daily plans and transportation options to users, but they lack the functionality to respond to real-time transportation information or make flexible suggestions based on the user's emotional state. As a result, they face challenges in smoothly planning trips and adjusting schedules when users feel stressed or when unexpected delays occur. Furthermore, they do not take into account the inconvenience of low battery levels while traveling. There is a need to address these challenges.

[1124] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring schedule information of the user, means for acquiring current location information, means for proposing multiple means of transportation and required times to the next scheduled location based on the schedule information and the current location information, means for adding travel time to the schedule based on the means of transportation selected by the user, means for setting a warning based on the set travel time, means for recognizing the emotional state of the user using emotion analysis means, and means for proposing adjustments to the means of transportation and the schedule based on the emotional state. This allows the user to select the optimal means of transportation based on real-time operation information, and further enables flexible suggestions based on the user's emotional state, thereby reducing stress and enabling smooth schedule management.

[1125] "User schedule information" refers to information about events and schedules that a user has registered in a schedule management system or calendar application.

[1126] "Current Location Information" means real-time geographic location data obtained by a User's Device through GPS or other location services.

[1127] "Multiple means of transportation" refers to the means of transportation available for the user to travel (e.g., train, bus, car, walking, etc.).

[1128] "Travel time" refers to the time required to travel from the current location to the destination, and is calculated based on calculation results that differ for each means of transportation.

[1129] "Travel time" refers to the time required to travel by the means of transportation selected by the user.

[1130] "Warnings" are alarms or notifications that are set to help users stay on schedule, and are issued before departure time or just before an event.

[1131] "Emotion analysis means" is a technology that uses sensors such as cameras and microphones to analyze the user's facial expressions and tone of voice, and recognizes the user's emotional state (e.g., joy, anger, sadness, surprise, etc.).

[1132] "Emotional state" refers to the user's psychological and emotional state as detected using emotion analysis means.

[1133] "Suggestions" are notifications and advice that suggest optimal travel options or schedule adjustments based on the user's emotional state and other real-time information.

[1134] System Overview

[1135] This invention is a schedule management system that uses AI, and provides a means to acquire a user's schedule information, suggest transportation methods and travel times based on current location information, and add them to the schedule. This system is composed of a server, terminals, and users, and by incorporating emotion analysis means, it is also possible to make flexible suggestions based on the user's emotional state.

[1136] System Configuration

[1137] The system is configured using the following hardware and software.

[1138] Device: Mobile devices such as smartphones and tablets

[1139] Server: Cloud Server

[1140] Sentiment analysis tools: camera, microphone, and sentiment analysis algorithms

[1141] Location information acquisition method: GPS sensor

[1142] Data acquisition and proposal method: Data communication using API (e.g., Google Calendar API, map service API)

[1143] Real-time information acquisition method: API that periodically acquires operation information (e.g., map service API)

[1144] System Operation

[1145] The device retrieves event data from the user's calendar or schedule application via an API. This system works with Google Calendar and other calendar services. The retrieved data includes the event start time, end time, location, and event name.

[1146] The device acquires the user's current location using a GPS sensor and stores the acquired information in its internal memory. The device then sends the user's current location to a server. The server extracts the next planned destination from a database and calculates the means of transportation and route based on the location information using a map service API (e.g., Google Maps API). The server then sends the calculation results to the device, which then suggests multiple means of transportation and their estimated travel times to the user.

[1147] When the user selects their preferred mode of transportation on the device, the information is sent to the server, which adds the travel time of the selected mode to the schedule, and the updated schedule is returned to the device, which notifies the user.

[1148] In addition, the device will set a warning 10 minutes before the scheduled departure time and notify the user of the set warning. The device will periodically check the map service API and update the schedule and warning time based on real-time service information.

[1149] Using emotion analysis, the device analyzes the user's facial expressions and voice data through a camera and microphone to recognize the user's emotional state in real time. The analysis results are sent to a cloud server, which then suggests transportation options and adjusts the schedule based on the user's emotional state.

[1150] Specific examples

[1151] For example, consider a case where a user travels to Shinjuku Ward for a meeting scheduled for 3 p.m. The user's current location is Chiyoda Ward, and if the emotion analysis means detects the user's stress, the server will suggest taking a taxi in addition to the train. If the user selects a taxi, the estimated taxi travel time is calculated to be 1:45 p.m., and the device will set an alert accordingly. Before leaving, the device will make suggestions such as "listen to relaxing music."

[1152] Prompt Sentence Examples

[1153] "Create a program that provides users with schedules and transportation options. The system also makes suggestions based on the user's emotional state."

[1154] "Please explain in detail how your schedule management system works, combined with the emotion engine. Please also include the optimal route from the user's current location to their destination."

[1155] In this way, the system of the present invention allows users to efficiently manage their schedules without any hassle. Furthermore, flexible suggestions based on the user's emotional state reduce stress and allow for comfortable travel and preparation.

[1156] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1157] Program processing flow

[1158] Step 1: Get the user's schedule information

[1159] Process flow:

[1160] The device accesses the user's calendar application and obtains schedule information through the API.

[1161] input:

[1162] The user's calendar application access permissions and API key.

[1163] Data processing / calculation:

[1164] The schedule data obtained through the API is analyzed and stored in an internal database.

[1165] output:

[1166] Schedule data captured (e.g., event name, start time, end time, location).

[1167] Specific behavior:

[1168] The device uses the Google Calendar API to retrieve the user's schedule information and store it in an internal database.

[1169] Step 2: Get your current location

[1170] Process flow:

[1171] The device uses the GPS sensor to obtain the user's current location and stores it in its internal memory.

[1172] input:

[1173] GPS sensor data.

[1174] Data processing / calculation:

[1175] The acquired latitude and longitude information is analyzed to determine the current location.

[1176] output:

[1177] Current location information (latitude and longitude).

[1178] Specific behavior:

[1179] The device periodically uses the GPS sensor to obtain the user's current location and stores it in its internal memory.

[1180] Step 3: Identify the next planned location and calculate the optimal route

[1181] Process flow:

[1182] The server identifies the next planned location and uses the map service's API to calculate the optimal route and travel time from the current location to the next planned location.

[1183] input:

[1184] Current location information, schedule information, map service API key.

[1185] Data processing / calculation:

[1186] Use the map service API to calculate multiple means of transportation (e.g., train, bus, car, walking) and the travel time required for each.

[1187] output:

[1188] Travel time and route information for each mode of transportation.

[1189] Specific behavior:

[1190] The server sends a request to the Google Maps API to calculate the optimal route from the current location to the next destination and the travel time for multiple modes of transportation.

[1191] Step 4: Select your mode of transport and add travel time

[1192] Process flow:

[1193] The terminal displays a notification prompting the user to select a means of transportation and adds travel time to the schedule based on the user's selection.

[1194] input:

[1195] Travel time and route information for each mode of transportation.

[1196] Data processing / calculation:

[1197] Add and recalculate travel time to the schedule based on the user's selected mode of travel.

[1198] output:

[1199] Updated schedule information.

[1200] Specific behavior:

[1201] The terminal displays a notification to the user to select a mode of transportation such as "train," "bus," or "taxi," and updates the schedule based on the selected mode of transportation.

[1202] Step 5: Set an alarm

[1203] Process flow:

[1204] The device will set an alarm 10 minutes before the scheduled departure time.

[1205] input:

[1206] Updated schedule information.

[1207] Data processing / calculation:

[1208] Calculate 10 minutes before departure time and set the alarm.

[1209] output:

[1210] Pre-departure warning alarm.

[1211] Specific behavior:

[1212] The device will set a warning alarm 10 minutes before departure time and send a notification to the user saying, "10 minutes left until departure."

[1213] Step 6: Reflecting real-time information

[1214] Process flow:

[1215] The device periodically checks real-time operation information and updates schedules and alarm times.

[1216] input:

[1217] Real-time information, updated schedules.

[1218] Data processing / calculation:

[1219] Recalculate and update schedules and alarms based on delay and operational information.

[1220] output:

[1221] Updated schedule information and reminders.

[1222] Specific behavior:

[1223] The device checks the map service API every 15 minutes to obtain delay and operation information and update schedules and alarms.

[1224] Step 7: Reminders

[1225] Process flow:

[1226] The device will notify the user of travel and preparation reminders.

[1227] input:

[1228] Updated schedule information, alarm 2 hours before departure.

[1229] Data processing / calculation:

[1230] Generate reminders before you leave or before an event.

[1231] output:

[1232] Reminder notifications.

[1233] Specific behavior:

[1234] The device will send reminder notifications such as "Leave in 10 minutes. Start getting ready" or "Meeting is about to start. Set your smartphone to silent mode."

[1235] Step 8: Emotion Recognition with the Emotion Engine

[1236] Process flow:

[1237] The device uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize their emotional state.

[1238] input:

[1239] Sensor data from cameras and microphones.

[1240] Data processing / calculation:

[1241] Emotion analysis algorithms are used to determine the user's emotional state in real time.

[1242] output:

[1243] The user's emotional state (e.g., happy, anger, sadness, surprise).

[1244] Specific behavior:

[1245] The device analyzes the user's facial expressions and tone of voice through a camera and microphone, and recognizes the user's emotional state, such as "feeling stressed."

[1246] Step 9: Emotion-Based Suggestions

[1247] Process flow:

[1248] The server generates optimal travel and schedule adjustment suggestions based on the emotional state.

[1249] input:

[1250] User's emotional state, schedule information, real-time traffic information.

[1251] Data processing / calculation:

[1252] Apply algorithms that recommend transportation and schedule adjustments based on emotional state.

[1253] output:

[1254] Proposal Notice.

[1255] Specific behavior:

[1256] Based on the analysis result that "the user is feeling stressed," the server generates suggestions such as "taking a taxi" or "listening to relaxing music" and notifies the user via the terminal.

[1257] (Application example 2)

[1258] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1259] While conventional schedule management systems can efficiently adjust schedules by taking into account users' travel times and real-time information, they have the problem of being unable to flexibly respond to users' emotional states. In particular, when managing schedules for store staff, if staff stress and fatigue are not properly managed, work efficiency and the quality of service may decline.

[1260] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring the user's schedule information, means for acquiring current location information, means for proposing multiple means of transportation and required times to the next scheduled location based on the schedule information and the current location information, means for adding travel time to the schedule based on the transportation means selected by the user, means for setting an alarm based on the set travel time, means for performing emotion analysis using a camera and microphone to recognize the user's emotional state, and means for adjusting the schedule and changing the notification content based on the user's emotional state. This enables flexible schedule adjustment and notification that takes the user's emotional state into consideration, enabling store staff to perform their work efficiently and provide high-quality services.

[1261] "User" refers to a person who uses the system.

[1262] "Schedule information" refers to data that records a user's schedules and appointments.

[1263] "Current location information" refers to information about the user's current location, and is mainly obtained using a GPS sensor.

[1264] "Transportation" refers to the means of transportation used by the user to travel to the next scheduled location.

[1265] "Travel time" refers to the time it takes for the user to travel from their current location to their next scheduled location.

[1266] "Emotional state" refers to a user's mental and emotional state, including emotions such as joy, anger, sadness, and surprise.

[1267] A "camera" is a device that captures images or videos using an optical sensor.

[1268] A "microphone" is a device for picking up sound, and the user's voice is used for emotion analysis.

[1269] "Emotion analysis" refers to the process of analyzing a user's facial expressions and tone of voice to recognize their emotional state.

[1270] An "alarm" is a function that notifies the user at a set time.

[1271] "Notification content" refers to the content of the message sent to the user as an alarm or reminder.

[1272] "Real-time information" refers to data that changes over time, such as the latest operation and traffic information.

[1273] The term "system" refers to a collection of hardware and software required to realize the various functions of the present invention.

[1274] The present invention is a system for improving the efficiency of a user's schedule management, and is embodied in the following specific form.

[1275] System configuration

[1276] The system includes means for acquiring the user's schedule information, means for acquiring current location information, means for suggesting multiple means of transportation and travel times to the next scheduled location, means for adding travel time to the schedule based on the means of transportation selected by the user, means for setting an alarm based on the set travel time, a camera and microphone and emotion analysis means for recognizing the user's emotional state, means for adjusting the schedule based on the user's emotional state, and means for changing the content of notifications.

[1277] Program processing overview

[1278] The system performs processing using the following hardware and software.

[1279] Hardware

[1280] 1. Smart glasses: display information and track user behavior.

[1281] 2. Camera: Used to recognize the user's facial expressions.

[1282] 3. Microphone: Uses the user's tone of voice for sentiment analysis.

[1283] 4. GPS sensor: Used to obtain current location information.

[1284] software

[1285] 1. Schedule management API: Obtain and manage user schedules (e.g., Google Calendar API).

[1286] 2. Map service API: Calculates the means of transportation and travel time to the next scheduled destination (e.g., Google Maps API).

[1287] 3. Emotion Recognition API: Analyze the user's emotional state (e.g., Emotion Recognition API).

[1288] 4. Notification system: Send reminders and alarms to users.

[1289] Data processing and calculation

[1290] The system processes and calculates the data as follows:

[1291] 1. Get user schedule:

[1292] The server retrieves the user's schedule information by calling the schedule management API, which downloads the user's schedule and appointment details.

[1293] 2. Obtaining current location information:

[1294] The server uses a GPS sensor to obtain the user's current location, which provides real-time location information about where the user is currently located.

[1295] 3. Travel options and travel times:

[1296] The server calls the map service API and calculates the travel time and options for travelling from the current location to the next destination, such as train, bus, car, and walking.

[1297] 4. Emotional state recognition and analysis:

[1298] The server collects data from the camera and microphone and uses an emotion recognition API to analyze the user's emotional state, thereby identifying the emotion the user is currently feeling (e.g., joy, anger, sadness, surprise, etc.).

[1299] 5. Adjust schedule and notification content:

[1300] The server adjusts the schedule based on the user's emotional state, for example by adding breaks to the schedule if it senses that the user is tired, and it also uses a notification system to send emotionally sensitive notifications.

[1301] Specific examples

[1302] As a specific example, consider a situation where store staff member A is scheduled to work the cash register from 2 p.m. and put merchandise on display from 3 p.m. The user's smart glasses calculate the time it takes to move around the store and the store's congestion status in real time, and reflect this in the schedule. Furthermore, if the analysis shows that staff member A is feeling "tired" or "stressed," the glasses will suggest adding more break time or notify the employee of an adjustment to the division of labor.

[1303] Prompt Sentence Examples

[1304] "Based on the new schedule, Staff A's task starting at 3 p.m. is to stock the shelves. However, if Staff A is feeling stressed, how do you adjust their schedule and alarms?"

[1305] This allows users to efficiently manage their schedules without hassle and allows them to respond flexibly to their emotional state. This system reduces stress for users and enables them to carry out their work comfortably.

[1306] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1307] Step 1:

[1308] The device calls the schedule management API to obtain the user's schedule information. This process involves sending a request to the API to obtain data including details of the user's schedule and appointments. Once the data is obtained, the device stores it in its internal database.

[1309] Step 2:

[1310] The device uses the GPS sensor to obtain the user's current location information. In this process, the current latitude and longitude data is obtained from the GPS sensor to obtain the coordinates of the user's current location. The obtained coordinate data becomes the input for proceeding to the next processing step.

[1311] Step 3:

[1312] The server proposes multiple means of transportation and the required travel time to the next scheduled location based on the schedule information and current location information. In this process, it calls the map service API to obtain the route and means of transportation (e.g., train, bus, car, walking, etc.) from the current location to the scheduled location, and calculates the required travel time for each. The server sends the obtained route information to the terminal and proposes multiple means of transportation to the user.

[1313] Step 4:

[1314] The user selects a desired transportation method from the transportation methods proposed through the terminal. In this process, travel time data is acquired based on the transportation method selected by the user. The terminal adds the selected transportation method and travel time to the schedule and updates it.

[1315] Step 5:

[1316] The device sets an alarm based on the set travel time. In this process, the alarm is set a certain time (for example, 10 minutes) before the scheduled departure time based on the travel time data. The set alarm information is saved in the device, and the user is notified when the departure time approaches.

[1317] Step 6:

[1318] The server uses a camera and microphone to capture the user's emotional state. This process involves capturing facial expression data through the camera and recording voice tone through the microphone. It then calls an emotion recognition API to analyze these data and identify the user's emotional state (e.g., joy, anger, sadness, surprise, etc.).

[1319] Step 7:

[1320] The server adjusts the schedule based on the user's emotional state. This process involves making necessary schedule changes depending on the user's emotional state. For example, if the server determines that the user is "tired," it adds a break to the schedule. It also changes work allocation and adjusts tasks as necessary.

[1321] Step 8:

[1322] The device changes the notification content according to the user's emotional state. This process flexibly adjusts the content of reminder and alarm notifications based on the user's emotional state. For example, if the device determines that the user is "feeling stressed," it sends a message to soften the tone of the notification.

[1323] Step 9:

[1324] The device obtains real-time traffic information and updates schedules and alarms. This process involves calling the traffic information API to obtain the latest traffic information and checking whether changes to the mode of travel or travel time are necessary. If necessary, the device automatically updates schedules and alarms and notifies the user.

[1325] Step 10:

[1326] The device monitors the remaining battery level and sends a notification to the user urging charging if it falls below a predetermined threshold. The battery monitoring system periodically checks the remaining battery level and, if it falls below a threshold, sends a notification to the user saying, "The battery is low. Please start charging."

[1327] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1328] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1329] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1330] [Fourth embodiment]

[1331] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1332] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1333] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1334] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1335] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1336] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1337] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1338] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1339] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1340] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1341] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1342] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1343] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1344] This invention is a smartphone schedule management system that uses AI to acquire user schedule information, automatically calculate travel time, add travel time to the schedule, set alarms, and update real-time information. The specific configuration of each part and operation of this system is explained below.

[1345] composition

[1346] 1. Get user schedule

[1347] The device retrieves event data from the user's calendar or schedule application via an API. This system is compatible with Google Calendar and other calendar services, for example.

[1348] 2. Travel time and route calculation

[1349] The device acquires the user's current location using the GPS sensor, then extracts the next scheduled location from the schedule and identifies its location information.

[1350] Based on this, the server accesses the API of the map service (for example, Google Maps API) and calculates the optimal means of transportation (train, bus, car, walking, etc.) from the current location to the next destination and the required time for each. After the server obtains this information, it proposes multiple means of transportation and their required times to the user.

[1351] 3. Add travel time to your schedule

[1352] When the user selects their preferred mode of transportation, the device adds the travel time to the schedule. For example, if the user's current location is Chiyoda Ward, Tokyo, and their next destination is Shinjuku Ward, and they select the train, the calculated travel time of 30 minutes will be reflected in the schedule.

[1353] 4. Set an alarm

[1354] The device will set an alarm 10 minutes before the scheduled departure time, so if the departure is scheduled for 2:30 PM, the alarm will be set for 2:20 PM.

[1355] 5. Real-time information

[1356] The device periodically checks real-time train and bus information to obtain delay information. Based on this information, it automatically updates schedules and alarm times as needed. For example, if a train is delayed by 10 minutes, the device will advance the departure time and reset the alarm.

[1357] 6. Reminder notifications

[1358] The device notifies the user of travel and preparation reminders. For example, at 2:10 p.m., it sends a notification saying, "10 minutes until departure. Please start getting ready." The device also sends a notification urging the user to charge the battery if the battery level is low. 10 minutes before the start of a meeting, it also sends a notification saying, "The meeting is about to start. Please set your smartphone to silent mode."

[1359] Specific examples

[1360] For example, suppose a user has a meeting scheduled for 3:00 PM in Shinjuku Ward. The user's current location is Chiyoda Ward, and the schedule is retrieved from a calendar application. The device then identifies the user's current location using GPS, and the server uses a map service API to calculate the optimal route and travel time from Chiyoda Ward to Shinjuku Ward.

[1361] If the user chooses to travel by train, the device calculates the travel time as 30 minutes and adds a departure time of 2:30 p.m. to the schedule. It also sets an alarm for 2:20 p.m. If the train is delayed, the device checks real-time information, automatically updates the schedule and alarm, and notifies the user.

[1362] In this way, the system of the present invention allows users to efficiently manage their schedules without hassle, helping to avoid mistakes and reducing stress.

[1363] The processing flow will be explained below.

[1364] Step 1:

[1365] The device retrieves event data from the user's calendar or schedule application via an API. For example, it sends a request to the calendar API to retrieve a schedule for a meeting starting at 3:00 PM on October 15, 2023.

[1366] Step 2:

[1367] The device acquires the user's current location information using a GPS sensor. For example, the device identifies that the user's current location is Chiyoda Ward, Tokyo.

[1368] Step 3:

[1369] The device identifies the next scheduled location from the schedule data and extracts its location information. For example, it recognizes that the next scheduled location is a "conference room in Shinjuku Ward, Tokyo."

[1370] Step 4:

[1371] The server accesses the API of a map service (e.g., Google Maps API) based on the current location and the next planned destination, and calculates the optimal means of transportation and the required time for each. For example, it obtains information such as 30 minutes by train, 45 minutes by bus, and 50 minutes by bicycle.

[1372] Step 5:

[1373] The terminal presents the acquired information on the means of transportation and the required time to the user, and the user selects the desired means. For example, the user selects "train."

[1374] Step 6:

[1375] The device will then add the travel time to the schedule based on the selected mode of transportation, updating the schedule to, for example, "Depart at 2:30 PM."

[1376] Step 7:

[1377] The device will set an alarm 10 minutes before departure time, for example, "Set alarm for 2:20 p.m."

[1378] Step 8:

[1379] The device periodically retrieves real-time data, including operational information, and checks train and bus delay information. For example, it retrieves information that a train is 10 minutes late.

[1380] Step 9:

[1381] The server automatically updates schedules and alarms based on the acquired operation information. For example, it may move the departure time forward to "2:15 PM" and reset the alarm to "2:05 PM."

[1382] Step 10:

[1383] The device sends a reminder to the user to prepare for travel. For example, at 2:10 p.m., the device sends a notification saying, "There are 10 minutes until departure. Please start preparing."

[1384] Step 11:

[1385] The device checks the battery status and sends a notification to encourage charging if the remaining charge falls below a certain threshold. For example, "I detected that the battery charge is less than 20% and sent a notification saying 'Please charge.'"

[1386] Step 12:

[1387] The device will remind the user to set the silent mode 10 minutes before an important event (e.g., a meeting) starts. For example, a notification will be sent 10 minutes before the start of the meeting saying, "The meeting is about to start. Please set your smartphone to silent mode."

[1388] Through these steps, users can efficiently and accurately manage their schedules, enabling smooth travel and preparation without the need for manual operations.

[1389] Example 1

[1390] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1391] Conventional schedule management systems require users to manually check current location information and transportation options and reflect them in their schedules. This makes schedule management cumbersome and makes it difficult to efficiently complete plans. Furthermore, they often lack the ability to provide real-time traffic information or appropriate alerts or notifications based on remaining battery power, increasing the risk of users missing important appointments.

[1392] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1393] In this invention, the server includes means for acquiring the user's schedule information, means for acquiring location information, means for proposing multiple means of transportation and travel times to the next scheduled location based on the schedule information and location information, means for adding travel time to the schedule based on the transportation means selected by the user, means for setting an alarm based on the set departure time, means for acquiring traffic operation information in real time and updating the schedule and alarm, and means for monitoring the remaining battery level and sending a notification urging the user to charge the battery when the battery falls below a predetermined threshold. This allows the user to efficiently manage their schedule without having to perform complicated manual operations, significantly reducing the risk of missing important appointments.

[1394] "User schedule information" is data relating to the user's schedule, obtained from a calendar application or a schedule application.

[1395] "Location Information" means data about a user's current latitude and longitude obtained using a GPS sensor or other positioning technology.

[1396] "Transportation" refers to the means used for travel, including trains, buses, cars, and walking.

[1397] "Travel time" is the estimated time it will take to travel from the current location to the next scheduled location.

[1398] "Real-time transportation operation information" is data that obtains the operation status of transportation in real time and reflects information such as delays and cancellations.

[1399] "Battery level monitoring" is a feature that measures the remaining battery level of smartphones and other mobile devices and notifies you when it reaches a certain level.

[1400] "Add to schedule" means to reflect new schedules or information in the user's schedule.

[1401] "Set an alarm" is a function that notifies the user when a specific time or condition is met.

[1402] The "notification to encourage charging" is a notification to encourage the user to charge when the remaining battery power is low.

[1403] This invention is a smartphone schedule management system that uses AI, which acquires the user's schedule information, suggests transportation methods along with location information, adds travel time to the schedule, sets alarms, and automatically updates real-time information. The specific configuration of each part and operation of this system is explained below.

[1404] Get user's appointment information

[1405] The device accesses a calendar application (e.g., a general-purpose calendar app) based on the user's authentication information and obtains the user's event information through the API. Once this operation is complete, the obtained event data is saved in the device's database. Specifically, the device communicates with the calendar API using the user's OAuth token, receives the obtained data in JSON format, and stores it in the local database.

[1406] Get the user's current location and identify the next scheduled location

[1407] The device activates the GPS sensor to obtain the user's current location. Then, the device extracts the location information of the next scheduled event from the most recent scheduled event and determines the latitude and longitude. For example, if the current location is Chiyoda Ward, Tokyo, and the next scheduled event location is Shinjuku Ward, the device will obtain the current location using GPS and the location information of the next scheduled event through a geocoding service.

[1408] Calculate travel route and travel time

[1409] The device sends this location information to the server. The server calls a map service API (for example, a general-purpose map service API) to calculate the optimal means of transportation from the current location to the next scheduled destination and the required travel time. The calculation results are returned to the device in JSON format. For example, the device sends a request to the map service API through the server to calculate the required travel time for various modes of transportation, such as train, bus, car, and walking.

[1410] Choosing transportation methods and reflecting them in your schedule

[1411] The device displays options for transportation and travel times to the user. When the user selects their preferred transportation method, the device reflects that travel time in the schedule. For example, if the user selects traveling by train, the device calculates the travel time as 30 minutes and updates the schedule to depart at 2:30 PM. The device then sets the alarm time to 2:20 PM based on this information.

[1412] Reflecting real-time information

[1413] The device obtains real-time traffic information at regular intervals and updates schedules and alarms as necessary. Specifically, the device uses a traffic information API to check real-time traffic information, and if there is delay information, it resets the departure time or alarm and notifies the user.

[1414] Reminders and notifications

[1415] The device sends users reminders to travel and prepare. It also notifies users when the battery is low, urging them to charge the device. For example, at 2:10 p.m., it sends a notification 10 minutes before departure, and 10 minutes before a meeting starts, it sends a notification saying, "The meeting is about to start. Please set your smartphone to silent mode."

[1416] Specific examples

[1417] For example, consider a case where a user has a meeting scheduled in Shinjuku Ward at 3:00 PM. The user's current location is Chiyoda Ward, and this appointment is retrieved from the configured calendar application. The device then uses GPS to determine the user's current location, and the server uses a general-purpose map service API to calculate the optimal route and travel time from Chiyoda Ward to Shinjuku Ward.

[1418] If the user chooses to travel by train, the device calculates the travel time as 30 minutes, updates the schedule to depart at 2:30 p.m., and sets an alarm for 2:20 p.m. If the train is delayed, the device checks real-time information, automatically updates the schedule and alarm, and notifies the user.

[1419] In this way, the system of the present invention allows users to efficiently manage their schedules without having to perform complicated manual operations, significantly reducing the risk of missing important appointments.

[1420] Example prompt sentence:

[1421] "I have a meeting in Shinjuku Ward at 3 PM. My current location is in Chiyoda Ward. Please calculate and suggest the best transportation method and travel time, and update my schedule accordingly. Also, please set an alarm 10 minutes before departure time, and update my schedule as needed based on real-time traffic information."

[1422] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1423] Step 1:

[1424] The device accesses a calendar API (for example, a general-purpose calendar API) based on the user's authentication information and obtains the user's schedule information. Specifically, it makes an API call using an OAuth token and receives the obtained schedule information in JSON format. User authentication information is required as input, and event data is obtained as output. This data is saved in a database within the device.

[1425] Step 2:

[1426] The device activates the GPS sensor and obtains the user's current location information. Specifically, it uses the GPS module to measure the current latitude and longitude. The input requires location measurement using the GPS module, and the output is the latitude and longitude data of the current location.

[1427] Step 3:

[1428] The device extracts the location information of the next scheduled event from the acquired schedule information. Specifically, it parses the address information from the next scheduled event and converts it to latitude and longitude using a geocoding API. The address of the next scheduled event is required as input, and the latitude and longitude data of the scheduled location is obtained as output.

[1429] Step 4:

[1430] The device sends latitude and longitude information of the current location and the next planned location to the server. The server calls a map service API (for example, a general-purpose map service API) and calculates the multiple means of transportation from the current location to the next planned location and the required travel time. Specifically, it uses the Directions API to obtain the route and time for each means. The latitude and longitude of the current location and the planned location are required as input, and the required travel time and route information for each means of transportation are obtained as output. The calculation results are returned to the device in JSON format.

[1431] Step 5:

[1432] The device presents the user with multiple transportation options and their respective travel times. When the user selects their preferred transportation option, the device reflects the selected travel time in the schedule. Specifically, the device displays a list of options through a user interface and receives the user's selection. The inputs are the travel times for each transportation option and the user's selection, and the output is an updated schedule.

[1433] Step 6:

[1434] The terminal sets an alarm 10 minutes before the scheduled departure time. Specifically, the alarm is set using the system's alarm manager. The departure time data is required as input, and the alarm setting is output.

[1435] Step 7:

[1436] The device periodically retrieves real-time traffic information and updates schedules and alarms. Specifically, it uses a traffic information API to check the latest traffic conditions and reset schedules and alarms as needed. Real-time traffic data is required as input, and updated schedules and alarms are obtained as output.

[1437] Step 8:

[1438] The device sends travel and preparation reminders to the user. Specifically, it uses push notifications to provide pre-departure reminders and low battery notifications. The inputs required are the timing of scheduled events and battery level data, and the output is the reminder notification sent to the user.

[1439] (Application example 1)

[1440] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1441] The purpose of this invention is to efficiently manage delivery staff schedules and optimize their travel in food delivery services. When there are multiple delivery destinations, conventional methods make it cumbersome to determine routes and manage travel times, and it is difficult to quickly respond to changes in traffic conditions. Therefore, there is a need for a system that can improve delivery efficiency and accuracy.

[1442] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1443] In this invention, the server includes means for acquiring schedule information of the user, means for acquiring current location information, means for proposing multiple means of transportation and required travel times based on the schedule information and current location information, means for adding travel time to the schedule based on the means of transportation selected by the user, means for setting an alarm based on the set travel time, and means for identifying the current location of the delivery person, optimizing the route to the next delivery destination, and updating the delivery schedule in real time, thereby enabling real-time updating of the delivery schedule and efficient route proposals.

[1444] The "means for acquiring user schedule information" is a function that enables an electronic device to automatically collect event data from a user's calendar or schedule.

[1445] "Means for obtaining current location information" means a function for determining the current geographic location of a user or device using GPS or other location information systems.

[1446] The "means for proposing multiple means of transportation and required travel times" is a function for calculating options for transportation (e.g., walking, car, train, etc.) and the required travel times for each, and presenting them to the user.

[1447] The "means for adding travel time to the schedule" is a function for reflecting the travel time calculated based on the means of transportation selected by the user in the schedule.

[1448] The "means for setting an alarm" is a function for sending a notification to the user at a specified time, and serves to prompt the user to leave or get ready.

[1449] "Means for determining the delivery person's current location" refers to a function that uses GPS or a location information system to determine the delivery person's current geographic location in real time.

[1450] The "means for optimizing the route to the next delivery destination" is a function for calculating and presenting the optimal and shortest route to the delivery destination.

[1451] "Means for updating delivery schedules in real time" refers to a function for updating delivery schedules at any time based on current traffic conditions and delivery progress.

[1452] "Means for obtaining real-time operation information and updating schedules and alarms" refers to a function for obtaining traffic conditions and operation information in real time and adjusting schedule and alarm settings based on this information.

[1453] "Means for monitoring the remaining battery level and sending a notification urging the user to charge when the battery falls below a specified threshold" is a function for constantly monitoring the battery status of the device and notifying the user to charge when the remaining battery level falls below a certain level.

[1454] The present invention provides a system for efficiently managing delivery staff schedules and optimizing their travel in a food delivery service. The system includes a means for acquiring a user's schedule information, a means for acquiring current location information, a means for proposing multiple means of transportation and required travel times, a means for adding travel time to the schedule, a means for setting an alarm, a means for identifying the delivery staff's current location, a means for optimizing the route to the next delivery destination, and a means for updating the delivery schedule in real time.

[1455] Configuration and operation

[1456] 1. Obtaining user schedule information

[1457] The server receives event data from the delivery person's schedule management application via an API. This system is compatible with general calendar services, for example.

[1458] 2. Obtaining current location information

[1459] The device uses a GPS sensor to acquire the delivery person's current location, and then sends the delivery person's current location information to the server.

[1460] 3. Proposing multiple transportation methods and travel times

[1461] The server calculates multiple means of transportation (e.g., car, bicycle, walking, etc.) to the next delivery destination and the required time for each, based on the acquired schedule information and current location information. The server uses a map service API (e.g., map service API) to suggest multiple means of transportation and their required time to the delivery person.

[1462] 4. Add travel time to your schedule

[1463] When a delivery person selects their preferred mode of transportation, the server adds the travel time to the schedule. For example, if the current location is somewhere and the next delivery destination is somewhere else, and the delivery person selects a car, the calculated travel time is reflected in the schedule.

[1464] 5. Set an alarm

[1465] The server sets an alarm 10 minutes before the scheduled departure time, allowing the delivery person to prepare for the next delivery.

[1466] 6. Real-time delivery schedule updates

[1467] The device periodically checks real-time traffic information to obtain traffic conditions and delay information. The server automatically updates schedules and alarm times as needed based on the information obtained. For example, if there is traffic congestion, the server will adjust departure times and reset alarms.

[1468] Hardware and software used

[1469] This system uses smart glasses equipped with a GPS sensor as the terminal, and the server calculates routes and retrieves schedule information using the Google Maps API and general calendar APIs.

[1470] Specific examples

[1471] For example, if a delivery person has multiple delivery destinations, the system calculates the travel time from the current location to each destination in real time and suggests an efficient route. The schedule is updated based on the transportation method selected by the delivery person, and an alarm is set for the departure time. Traffic conditions and delay information are obtained in real time, and the schedule and alarms are automatically updated as needed.

[1472] Prompt Sentence Examples

[1473] Here is an example prompt:

[1474] Locate your delivery drivers with GPS and calculate the best route and travel time to their next delivery. Update schedules in real time and set alarms based on their next due time. Also, monitor traffic conditions and reschedule if delays occur.

[1475] This allows the system of the present invention to support delivery personnel in efficient delivery and improve the reliability and accuracy of delivery services.

[1476] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1477] Step 1:

[1478] The server retrieves schedule information from the delivery driver's schedule management application via an API. The inputs include the delivery driver's account information and API key. The output is the schedule information in JSON format or other structured data format. The server parses this data to obtain the delivery order and location information.

[1479] Step 2:

[1480] The terminal acquires the delivery person's current location information using a GPS sensor. The input is a signal obtained from a GPS device, and the output is real-time latitude and longitude data. The terminal then transmits the acquired geographic information to a server.

[1481] Step 3:

[1482] The server proposes multiple means of transportation to the next delivery destination and the required travel time based on the acquired schedule information and current location information. This step requires the latitude and longitude of the current location and schedule information as input, and the required travel time for each means of transportation (e.g., car, bicycle, walking) as output. The server uses a map service API to calculate the optimal route and means of transportation.

[1483] Step 4:

[1484] The user selects the desired transportation method from the presented options. The input is the user's selection information, and the output is the data of the selected transportation method. The server recalculates the required travel time based on the selected transportation method and adds the travel time to the next delivery destination to the schedule.

[1485] Step 5:

[1486] The server sets an alarm based on the set travel time. The input is the updated schedule information and travel time, and the output is the alarm setting data. Based on this, the server sets a notification to be sent before the departure time.

[1487] Step 6:

[1488] The terminal periodically checks real-time traffic information and obtains traffic conditions and delay information. As input, it sends a request to the traffic information API, and as output, it obtains the latest traffic information. The terminal then sends this to the server.

[1489] Step 7:

[1490] The server updates the schedule based on the acquired real-time operation information. This step requires the delivery schedule and real-time operation information as input, and obtains the updated schedule data as output. The server also resets the alarm time if necessary and sends a notification to the user.

[1491] This allows delivery personnel to efficiently manage their delivery schedules and respond quickly to changes in traffic conditions.

[1492] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1493] This invention is a smartphone schedule management system that uses AI. It acquires the user's schedule information, calculates travel time, adds travel time to the schedule, sets alarms, and automatically updates real-time information. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to flexibly suggest schedules and provide notifications according to the user's emotional state. The specific components and operations of this system are described below.

[1494] composition

[1495] 1. Get user schedule

[1496] The device retrieves event data from the user's calendar or schedule application via an API. This system is compatible with Google Calendar and other calendar services, for example.

[1497] 2. Travel time and route calculation

[1498] The device acquires the user's current location using the GPS sensor, then extracts the next scheduled location from the schedule and identifies its location information.

[1499] Based on this, the server accesses the API of the map service (for example, Google Maps API) and calculates the optimal means of transportation (train, bus, car, walking, etc.) from the current location to the next destination and the required time for each. After the server obtains this information, it proposes multiple means of transportation and their required times to the user.

[1500] 3. Add travel time to your schedule

[1501] When the user selects their preferred mode of transportation, the device adds the travel time to the schedule. For example, if the user's current location is Chiyoda Ward, Tokyo, and their next destination is Shinjuku Ward, and they select the train, the calculated travel time of 30 minutes will be reflected in the schedule.

[1502] 4. Set an alarm

[1503] The device will set an alarm 10 minutes before the scheduled departure time, so if the departure is scheduled for 2:30 PM, the alarm will be set for 2:20 PM.

[1504] 5. Real-time information

[1505] The device periodically checks real-time train and bus information to obtain delay information. Based on this information, it automatically updates schedules and alarm times as needed. For example, if a train is delayed by 10 minutes, the device will advance the departure time and reset the alarm.

[1506] 6. Reminder notifications

[1507] The device notifies the user of travel and preparation reminders. For example, at 2:10 p.m., it sends a notification saying, "10 minutes until departure. Please start getting ready." The device also sends a notification urging the user to charge the battery if the battery level is low. 10 minutes before the start of a meeting, it also sends a notification saying, "The meeting is about to start. Please set your smartphone to silent mode."

[1508] Combining Emotion Engines

[1509] 1. Emotion Engine Configuration

[1510] The device is equipped with an emotion engine that uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice, recognizing the user's emotional state (e.g., joy, anger, sadness, surprise, etc.) in real time.

[1511] 2. Emotion-based transportation suggestions

[1512] The server analyzes the user's emotional state and suggests transportation options based on the results. For example, if the server detects that the user is feeling stressed, it will prioritize more comfortable transportation options (such as taxis).

[1513] 3. Adjust your schedule based on emotions

[1514] The device will flexibly adjust the schedule based on the user's emotional state. For example, if it detects that the user is tired, it will suggest flexible schedules and break times.

[1515] 4. Tailoring notification content based on emotions

[1516] The device changes the content of the reminder notification depending on the user's emotional state. For example, if the device detects that the user is nervous, it will soften the tone of the reminder.

[1517] Specific examples

[1518] For example, suppose a user has a meeting scheduled in Shinjuku Ward at 3:00 PM. The user's current location is Chiyoda Ward. If the user's emotion engine detects high stress, the server suggests taking a taxi in addition to the train. If the user chooses to take a taxi, the device adds the taxi's travel time to the schedule and sets the departure time to 1:45 PM. The device also suggests relaxing music as a reminder before the user gets in the taxi.

[1519] In this way, the system of the present invention allows users to efficiently manage their schedules without hassle and allows them to respond flexibly to their emotional state. This system reduces users' stress and helps them travel and prepare comfortably.

[1520] The processing flow will be explained below.

[1521] Step 1:

[1522] The device retrieves event data from the user's calendar or schedule application via an API. For example, it sends a request to the calendar API to retrieve a schedule for a meeting starting at 3:00 PM on October 15, 2023.

[1523] Step 2:

[1524] The device acquires the user's current location information using a GPS sensor. For example, the device identifies that the user's current location is Chiyoda Ward, Tokyo.

[1525] Step 3:

[1526] The device identifies the next scheduled location from the schedule data and extracts its location information. For example, it recognizes that the next scheduled location is a "conference room in Shinjuku Ward, Tokyo."

[1527] Step 4:

[1528] The server accesses the API of a map service (e.g., Google Maps API) based on the current location and the next planned destination, and calculates the optimal means of transportation and the required time for each. For example, it obtains information such as 30 minutes by train, 45 minutes by bus, and 50 minutes by bicycle.

[1529] Step 5:

[1530] The terminal presents the acquired information on the means of transportation and the required time to the user, and the user selects the desired means. For example, the user selects "train."

[1531] Step 6:

[1532] The device will then add the travel time to the schedule based on the selected mode of transportation, updating the schedule to, for example, "Depart at 2:30 PM."

[1533] Step 7:

[1534] The device will set an alarm 10 minutes before departure time, for example, "Set alarm for 2:20 p.m."

[1535] Step 8:

[1536] The device periodically checks the traffic information API to get real-time service information, for example, every 10 minutes to check train and bus delay information.

[1537] Step 9:

[1538] The server automatically updates schedules and alarms based on real-time train information. For example, if a train is 10 minutes late, the departure time will be moved forward to 14:10 and the alarm will be reset to 14:00.

[1539] Step 10:

[1540] The device sends a reminder to the user to prepare for travel. For example, at 2:10 p.m., the device sends a notification saying, "There are 10 minutes until departure. Please start preparing."

[1541] Step 11:

[1542] The device checks the battery status and sends a notification to encourage charging if the remaining charge falls below a certain threshold. For example, "I detected that the battery charge is less than 20% and sent a notification saying 'Please charge.'"

[1543] Step 12:

[1544] The device will remind the user to set the silent mode 10 minutes before an important event (e.g., a meeting) starts. For example, a notification will be sent 10 minutes before the start of the meeting saying, "The meeting is about to start. Please set your smartphone to silent mode."

[1545] Adding an Emotion Engine

[1546] Step 13:

[1547] The device activates an emotion engine that uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice. For example, the camera captures the user's face and applies emotion analysis algorithms.

[1548] Step 14:

[1549] The emotion engine analyzes the user's emotional state (e.g., joy, anger, sadness, surprise, etc.) and sends the results to the server. For example, it detects that the user is feeling stressed.

[1550] Step 15:

[1551] The server suggests optimal transportation methods to users based on their emotional state: for example, if a user is feeling stressed, it will prioritize taxis over walking or buses.

[1552] Step 16:

[1553] The user selects the desired transportation method from the suggested transportation methods, for example, "taxi."

[1554] Step 17:

[1555] The device will add travel time to the schedule based on the selected mode of transportation and send notifications that take emotional state into account, such as adding a taxi ride time to the schedule and sending a relaxing music reminder.

[1556] By implementing these steps, users can efficiently manage their schedules without any hassle and can respond flexibly to their emotional state. This system reduces users' stress and supports comfortable travel and preparation.

[1557] Example 2

[1558] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1559] Conventional schedule management systems can suggest daily plans and transportation options to users, but they lack the functionality to respond to real-time transportation information or make flexible suggestions based on the user's emotional state. As a result, they face challenges in smoothly planning trips and adjusting schedules when users feel stressed or when unexpected delays occur. Furthermore, they do not take into account the inconvenience of low battery levels while traveling. There is a need to address these challenges.

[1560] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring schedule information of the user, means for acquiring current location information, means for proposing multiple means of transportation and required times to the next scheduled location based on the schedule information and the current location information, means for adding travel time to the schedule based on the means of transportation selected by the user, means for setting a warning based on the set travel time, means for recognizing the emotional state of the user using emotion analysis means, and means for proposing adjustments to the means of transportation and the schedule based on the emotional state. This allows the user to select the optimal means of transportation based on real-time operation information, and further enables flexible suggestions based on the user's emotional state, thereby reducing stress and enabling smooth schedule management.

[1561] "User schedule information" refers to information about events and schedules that a user has registered in a schedule management system or calendar application.

[1562] "Current Location Information" means real-time geographic location data obtained by a User's Device through GPS or other location services.

[1563] "Multiple means of transportation" refers to the means of transportation available for the user to travel (e.g., train, bus, car, walking, etc.).

[1564] "Travel time" refers to the time required to travel from the current location to the destination, and is calculated based on calculation results that differ for each means of transportation.

[1565] "Travel time" refers to the time required to travel by the means of transportation selected by the user.

[1566] "Warnings" are alarms or notifications that are set to help users stay on schedule, and are issued before departure time or just before an event.

[1567] "Emotion analysis means" is a technology that uses sensors such as cameras and microphones to analyze the user's facial expressions and tone of voice, and recognizes the user's emotional state (e.g., joy, anger, sadness, surprise, etc.).

[1568] "Emotional state" refers to the user's psychological and emotional state as detected using emotion analysis means.

[1569] "Suggestions" are notifications and advice that suggest optimal travel options or schedule adjustments based on the user's emotional state and other real-time information.

[1570] System Overview

[1571] This invention is a schedule management system that uses AI, and provides a means to acquire a user's schedule information, suggest transportation methods and travel times based on current location information, and add them to the schedule. This system is composed of a server, terminals, and users, and by incorporating emotion analysis means, it is also possible to make flexible suggestions based on the user's emotional state.

[1572] System Configuration

[1573] The system is configured using the following hardware and software.

[1574] Device: Mobile devices such as smartphones and tablets

[1575] Server: Cloud Server

[1576] Sentiment analysis tools: camera, microphone, and sentiment analysis algorithms

[1577] Location information acquisition method: GPS sensor

[1578] Data acquisition and proposal method: Data communication using API (e.g., Google Calendar API, map service API)

[1579] Real-time information acquisition method: API that periodically acquires operation information (e.g., map service API)

[1580] System Operation

[1581] The device retrieves event data from the user's calendar or schedule application via an API. This system works with Google Calendar and other calendar services. The retrieved data includes the event start time, end time, location, and event name.

[1582] The device acquires the user's current location using a GPS sensor and stores the acquired information in its internal memory. The device then sends the user's current location to a server. The server extracts the next planned destination from a database and calculates the means of transportation and route based on the location information using a map service API (e.g., Google Maps API). The server then sends the calculation results to the device, which then suggests multiple means of transportation and their estimated travel times to the user.

[1583] When the user selects their preferred mode of transportation on the device, the information is sent to the server, which adds the travel time of the selected mode to the schedule, and the updated schedule is returned to the device, which notifies the user.

[1584] In addition, the device will set a warning 10 minutes before the scheduled departure time and notify the user of the set warning. The device will periodically check the map service API and update the schedule and warning time based on real-time service information.

[1585] Using emotion analysis, the device analyzes the user's facial expressions and voice data through a camera and microphone to recognize the user's emotional state in real time. The analysis results are sent to a cloud server, which then suggests transportation options and adjusts the schedule based on the user's emotional state.

[1586] Specific examples

[1587] For example, consider a case where a user travels to Shinjuku Ward for a meeting scheduled for 3 p.m. The user's current location is Chiyoda Ward, and if the emotion analysis means detects the user's stress, the server will suggest taking a taxi in addition to the train. If the user selects a taxi, the estimated taxi travel time is calculated to be 1:45 p.m., and the device will set an alert accordingly. Before leaving, the device will make suggestions such as "listen to relaxing music."

[1588] Prompt Sentence Examples

[1589] "Create a program that provides users with schedules and transportation options. The system also makes suggestions based on the user's emotional state."

[1590] "Please explain in detail how your schedule management system works, combined with the emotion engine. Please also include the optimal route from the user's current location to their destination."

[1591] In this way, the system of the present invention allows users to efficiently manage their schedules without any hassle. Furthermore, flexible suggestions based on the user's emotional state reduce stress and allow for comfortable travel and preparation.

[1592] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1593] Program processing flow

[1594] Step 1: Get the user's schedule information

[1595] Process flow:

[1596] The device accesses the user's calendar application and obtains schedule information through the API.

[1597] input:

[1598] The user's calendar application access permissions and API key.

[1599] Data processing / calculation:

[1600] The schedule data obtained through the API is analyzed and stored in an internal database.

[1601] output:

[1602] Schedule data captured (e.g., event name, start time, end time, location).

[1603] Specific behavior:

[1604] The device uses the Google Calendar API to retrieve the user's schedule information and store it in an internal database.

[1605] Step 2: Get your current location

[1606] Process flow:

[1607] The device uses the GPS sensor to obtain the user's current location and stores it in its internal memory.

[1608] input:

[1609] GPS sensor data.

[1610] Data processing / calculation:

[1611] The acquired latitude and longitude information is analyzed to determine the current location.

[1612] output:

[1613] Current location information (latitude and longitude).

[1614] Specific behavior:

[1615] The device periodically uses the GPS sensor to obtain the user's current location and stores it in its internal memory.

[1616] Step 3: Identify the next planned location and calculate the optimal route

[1617] Process flow:

[1618] The server identifies the next planned location and uses the map service's API to calculate the optimal route and travel time from the current location to the next planned location.

[1619] input:

[1620] Current location information, schedule information, map service API key.

[1621] Data processing / calculation:

[1622] Use the map service API to calculate multiple means of transportation (e.g., train, bus, car, walking) and the travel time required for each.

[1623] output:

[1624] Travel time and route information for each mode of transportation.

[1625] Specific behavior:

[1626] The server sends a request to the Google Maps API to calculate the optimal route from the current location to the next destination and the travel time for multiple modes of transportation.

[1627] Step 4: Select your mode of transport and add travel time

[1628] Process flow:

[1629] The terminal displays a notification prompting the user to select a means of transportation and adds travel time to the schedule based on the user's selection.

[1630] input:

[1631] Travel time and route information for each mode of transportation.

[1632] Data processing / calculation:

[1633] Add and recalculate travel time to the schedule based on the user's selected mode of travel.

[1634] output:

[1635] Updated schedule information.

[1636] Specific behavior:

[1637] The terminal displays a notification to the user to select a mode of transportation such as "train," "bus," or "taxi," and updates the schedule based on the selected mode of transportation.

[1638] Step 5: Set an alarm

[1639] Process flow:

[1640] The device will set an alarm 10 minutes before the scheduled departure time.

[1641] input:

[1642] Updated schedule information.

[1643] Data processing / calculation:

[1644] Calculate 10 minutes before departure time and set the alarm.

[1645] output:

[1646] Pre-departure warning alarm.

[1647] Specific behavior:

[1648] The device will set a warning alarm 10 minutes before departure time and send a notification to the user saying, "10 minutes left until departure."

[1649] Step 6: Reflecting real-time information

[1650] Process flow:

[1651] The device periodically checks real-time operation information and updates schedules and alarm times.

[1652] input:

[1653] Real-time information, updated schedules.

[1654] Data processing / calculation:

[1655] Recalculate and update schedules and alarms based on delay and operational information.

[1656] output:

[1657] Updated schedule information and reminders.

[1658] Specific behavior:

[1659] The device checks the map service API every 15 minutes to obtain delay and operation information and update schedules and alarms.

[1660] Step 7: Reminders

[1661] Process flow:

[1662] The device will notify the user of travel and preparation reminders.

[1663] input:

[1664] Updated schedule information, alarm 2 hours before departure.

[1665] Data processing / calculation:

[1666] Generate reminders before you leave or before an event.

[1667] output:

[1668] Reminder notifications.

[1669] Specific behavior:

[1670] The device will send reminder notifications such as "Leave in 10 minutes. Start getting ready" or "Meeting is about to start. Set your smartphone to silent mode."

[1671] Step 8: Emotion Recognition with the Emotion Engine

[1672] Process flow:

[1673] The device uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize their emotional state.

[1674] input:

[1675] Sensor data from cameras and microphones.

[1676] Data processing / calculation:

[1677] Emotion analysis algorithms are used to determine the user's emotional state in real time.

[1678] output:

[1679] The user's emotional state (e.g., happy, anger, sadness, surprise).

[1680] Specific behavior:

[1681] The device analyzes the user's facial expressions and tone of voice through a camera and microphone, and recognizes the user's emotional state, such as "feeling stressed."

[1682] Step 9: Emotion-Based Suggestions

[1683] Process flow:

[1684] The server generates optimal travel and schedule adjustment suggestions based on the emotional state.

[1685] input:

[1686] User's emotional state, schedule information, real-time traffic information.

[1687] Data processing / calculation:

[1688] Apply algorithms that recommend transportation and schedule adjustments based on emotional state.

[1689] output:

[1690] Proposal Notice.

[1691] Specific behavior:

[1692] Based on the analysis result that "the user is feeling stressed," the server generates suggestions such as "taking a taxi" or "listening to relaxing music" and notifies the user via the terminal.

[1693] (Application example 2)

[1694] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1695] While conventional schedule management systems can efficiently adjust schedules by taking into account users' travel times and real-time information, they have the problem of being unable to flexibly respond to users' emotional states. In particular, when managing schedules for store staff, if staff stress and fatigue are not properly managed, work efficiency and the quality of service may decline.

[1696] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring the user's schedule information, means for acquiring current location information, means for proposing multiple means of transportation and required times to the next scheduled location based on the schedule information and the current location information, means for adding travel time to the schedule based on the transportation means selected by the user, means for setting an alarm based on the set travel time, means for performing emotion analysis using a camera and microphone to recognize the user's emotional state, and means for adjusting the schedule and changing the notification content based on the user's emotional state. This enables flexible schedule adjustment and notification that takes the user's emotional state into consideration, enabling store staff to perform their work efficiently and provide high-quality services.

[1697] "User" refers to a person who uses the system.

[1698] "Schedule information" refers to data that records a user's schedules and appointments.

[1699] "Current location information" refers to information about the user's current location, and is mainly obtained using a GPS sensor.

[1700] "Transportation" refers to the means of transportation used by the user to travel to the next scheduled location.

[1701] "Travel time" refers to the time it takes for the user to travel from their current location to their next scheduled location.

[1702] "Emotional state" refers to a user's mental and emotional state, including emotions such as joy, anger, sadness, and surprise.

[1703] A "camera" is a device that captures images or videos using an optical sensor.

[1704] A "microphone" is a device for picking up sound, and the user's voice is used for emotion analysis.

[1705] "Emotion analysis" refers to the process of analyzing a user's facial expressions and tone of voice to recognize their emotional state.

[1706] An "alarm" is a function that notifies the user at a set time.

[1707] "Notification content" refers to the content of the message sent to the user as an alarm or reminder.

[1708] "Real-time information" refers to data that changes over time, such as the latest operation and traffic information.

[1709] The term "system" refers to a collection of hardware and software required to realize the various functions of the present invention.

[1710] The present invention is a system for improving the efficiency of a user's schedule management, and is embodied in the following specific form.

[1711] System configuration

[1712] The system includes means for acquiring the user's schedule information, means for acquiring current location information, means for suggesting multiple means of transportation and travel times to the next scheduled location, means for adding travel time to the schedule based on the means of transportation selected by the user, means for setting an alarm based on the set travel time, a camera and microphone and emotion analysis means for recognizing the user's emotional state, means for adjusting the schedule based on the user's emotional state, and means for changing the content of notifications.

[1713] Program processing overview

[1714] The system performs processing using the following hardware and software.

[1715] Hardware

[1716] 1. Smart glasses: display information and track user behavior.

[1717] 2. Camera: Used to recognize the user's facial expressions.

[1718] 3. Microphone: Uses the user's tone of voice for sentiment analysis.

[1719] 4. GPS sensor: Used to obtain current location information.

[1720] software

[1721] 1. Schedule management API: Obtain and manage user schedules (e.g., Google Calendar API).

[1722] 2. Map service API: Calculates the means of transportation and travel time to the next scheduled destination (e.g., Google Maps API).

[1723] 3. Emotion Recognition API: Analyze the user's emotional state (e.g., Emotion Recognition API).

[1724] 4. Notification system: Send reminders and alarms to users.

[1725] Data processing and calculation

[1726] The system processes and calculates the data as follows:

[1727] 1. Get user schedule:

[1728] The server retrieves the user's schedule information by calling the schedule management API, which downloads the user's schedule and appointment details.

[1729] 2. Obtaining current location information:

[1730] The server uses a GPS sensor to obtain the user's current location, which provides real-time location information about where the user is currently located.

[1731] 3. Travel options and travel times:

[1732] The server calls the map service API and calculates the travel time and options for travelling from the current location to the next destination, such as train, bus, car, and walking.

[1733] 4. Emotional state recognition and analysis:

[1734] The server collects data from the camera and microphone and uses an emotion recognition API to analyze the user's emotional state, thereby identifying the emotion the user is currently feeling (e.g., joy, anger, sadness, surprise, etc.).

[1735] 5. Adjust schedule and notification content:

[1736] The server adjusts the schedule based on the user's emotional state, for example by adding breaks to the schedule if it senses that the user is tired, and it also uses a notification system to send emotionally sensitive notifications.

[1737] Specific examples

[1738] As a specific example, consider a situation where store staff member A is scheduled to work the cash register from 2 p.m. and put merchandise on display from 3 p.m. The user's smart glasses calculate the time it takes to move around the store and the store's congestion status in real time, and reflect this in the schedule. Furthermore, if the analysis shows that staff member A is feeling "tired" or "stressed," the glasses will suggest adding more break time or notify the employee of an adjustment to the division of labor.

[1739] Prompt Sentence Examples

[1740] "Based on the new schedule, Staff A's task starting at 3 p.m. is to stock the shelves. However, if Staff A is feeling stressed, how do you adjust their schedule and alarms?"

[1741] This allows users to efficiently manage their schedules without hassle and allows them to respond flexibly to their emotional state. This system reduces stress for users and enables them to carry out their work comfortably.

[1742] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1743] Step 1:

[1744] The device calls the schedule management API to obtain the user's schedule information. This process involves sending a request to the API to obtain data including details of the user's schedule and appointments. Once the data is obtained, the device stores it in its internal database.

[1745] Step 2:

[1746] The device uses the GPS sensor to obtain the user's current location information. In this process, the current latitude and longitude data is obtained from the GPS sensor to obtain the coordinates of the user's current location. The obtained coordinate data becomes the input for proceeding to the next processing step.

[1747] Step 3:

[1748] The server proposes multiple means of transportation and the required travel time to the next scheduled location based on the schedule information and current location information. In this process, it calls the map service API to obtain the route and means of transportation (e.g., train, bus, car, walking, etc.) from the current location to the scheduled location, and calculates the required travel time for each. The server sends the obtained route information to the terminal and proposes multiple means of transportation to the user.

[1749] Step 4:

[1750] The user selects a desired transportation method from the transportation methods proposed through the terminal. In this process, travel time data is acquired based on the transportation method selected by the user. The terminal adds the selected transportation method and travel time to the schedule and updates it.

[1751] Step 5:

[1752] The device sets an alarm based on the set travel time. In this process, the alarm is set a certain time (for example, 10 minutes) before the scheduled departure time based on the travel time data. The set alarm information is saved in the device, and the user is notified when the departure time approaches.

[1753] Step 6:

[1754] The server uses a camera and microphone to capture the user's emotional state. This process involves capturing facial expression data through the camera and recording voice tone through the microphone. It then calls an emotion recognition API to analyze these data and identify the user's emotional state (e.g., joy, anger, sadness, surprise, etc.).

[1755] Step 7:

[1756] The server adjusts the schedule based on the user's emotional state. This process involves making necessary schedule changes depending on the user's emotional state. For example, if the server determines that the user is "tired," it adds a break to the schedule. It also changes work allocation and adjusts tasks as necessary.

[1757] Step 8:

[1758] The device changes the notification content according to the user's emotional state. This process flexibly adjusts the content of reminder and alarm notifications based on the user's emotional state. For example, if the device determines that the user is "feeling stressed," it sends a message to soften the tone of the notification.

[1759] Step 9:

[1760] The device obtains real-time traffic information and updates schedules and alarms. This process involves calling the traffic information API to obtain the latest traffic information and checking whether changes to the mode of travel or travel time are necessary. If necessary, the device automatically updates schedules and alarms and notifies the user.

[1761] Step 10:

[1762] The device monitors the remaining battery level and sends a notification to the user urging charging if it falls below a predetermined threshold. The battery monitoring system periodically checks the remaining battery level and, if it falls below a threshold, sends a notification to the user saying, "The battery is low. Please start charging."

[1763] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1764] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1765] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1766] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1767] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1768] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1769] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1770] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1771] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1772] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1773] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1774] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1775] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1776] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1777] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1778] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1779] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1780] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1781] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1782] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be...

Claims

1. A means for obtaining schedule information of a user; A means for obtaining current location information; A means for proposing multiple means of transportation and required travel times to the next scheduled location based on schedule information and current location information; means for adding travel time to the schedule based on the mode of travel selected by the user; and means for setting an alarm based on the set travel time.

2. 10. The system of claim 1, further comprising means for obtaining real-time operational information and updating schedules and alarms.

3. 10. The system of claim 1, further comprising means for monitoring a remaining battery charge and sending a notification to charge the battery when the battery falls below a predetermined threshold.

Images