System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting position information and flow line information of a user; means for analyzing the collected position information and flow line information; and means for proposing an optimal next action for the user based on a result of the analysis.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In everyday life, it is important for users to use their time efficiently and minimize wasted movement and downtime. However, manually optimizing individual movement paths and downtime is difficult and labor-intensive. To solve this problem, a system is needed that can monitor and analyze user behavior in real time and automatically suggest the optimal next action. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems with a system that includes a means for collecting user location information and movement path information, a means for analyzing this information, and a means for proposing the optimal next action to the user based on the analysis results. Specifically, the system collects user location information and movement path information in real time and also records the user's downtime. This data is analyzed to identify wasted time and efficient movement paths, and the system proposes the most appropriate next action to the user in real time. These proposals are then sent to the user's wearable device, allowing the user to take efficient action.

[0006] "User location information" is data that indicates where the user is currently located.

[0007] "Flow line information" is data that indicates the route taken by a user.

[0008] "Dwell time" is data that indicates how long a user stayed at a particular location.

[0009] "Means of analysis" refers to the process of analyzing collected location information, movement information, and stop times, and extracting meaningful results and patterns based on that information.

[0010] "Next action" is the optimal next action that the user should take, derived from the analysis results.

[0011] "Suggestion means" refers to methods and techniques for informing users of the next action based on the analysis results.

[0012] A "system" is a set of interrelated elements that work together to achieve a specific purpose. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention provides a system that functions as an assistant to help users lead their daily lives efficiently. It collects and analyzes the user's location information, movement path information, and downtime, and suggests the optimal next action. This system consists of three main components: the terminal (wearable device), the server, and the user.

[0035] System Overview

[0036] 1. Data Collection

[0037] The wearable device uses GPS and acceleration sensors to collect real-time location and movement information for the user. For example, it records current location data and movement information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room."

[0038] The device also records the user's downtime, for example, "staying in the kitchen" from 8:00 AM to 8:05 AM.

[0039] 2. Data Transmission

[0040] The device sends the collected location information, movement information, and stop time data to the server at regular intervals (e.g., every 5 minutes), where the data is accumulated and used for analysis.

[0041] 3. Data analysis

[0042] The server receives the data and first stores it in a database. It then converts it into an analytical format and analyzes the user's movement patterns and downtime. For example, it can identify a pattern such as "every morning, the user goes to the kitchen at 8:00 AM."

[0043] The server uses the collected data to run algorithms that identify wasted time and efficient routes for users, generating specific action suggestions such as "Take out the trash during the two minutes you wait for the microwave."

[0044] 4. Action proposals

[0045] The server then selects the optimal next action based on the analysis results, such as suggesting other housework that would be best suited to the time the user is waiting in the kitchen.

[0046] The server generates notification data that suggests the selected action, and generates a message such as "Take out the garbage while the microwave is in standby mode."

[0047] 5. Notice to Users

[0048] The server sends the generated notification data to the device, which then displays the received notification data to the user in real time. For example, it displays a message such as "Take out the trash while the microwave is in standby mode."

[0049] 6. User execution

[0050] The user follows the notification to take out the trash. This suggestion helps users reduce wasted time and do their housework efficiently.

[0051] Specific examples

[0052] Situation

[0053] A user is preparing breakfast and has to wait a few minutes while the ingredients heat up in the microwave.

[0054] Data collection and transmission

[0055] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[0056] Data analysis and recommendations

[0057] The server identifies the amount of time a user waits while using a microwave in the kitchen, and also determines from past logs that a nearby trash can is full.

[0058] The server selects "take out the garbage while the microwave is in standby mode" as the optimal action and generates notification data.

[0059] User Notification

[0060] Notification data is sent from the server to the device, and the device displays a message to the user saying, "Take out the trash while the microwave is in standby mode."

[0061] User execution

[0062] The user follows the notification and takes out the trash while the microwave is waiting.

[0063] This system allows users to effectively utilize their time and improve daily efficiency.The effects of the present invention are realized by such specific embodiments.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] The device collects the user's current location and movement information in real time using GPS and acceleration sensors. For example, if the user is in the kitchen, the device will record the location information as "08:00 AM - Kitchen."

[0067] Step 2:

[0068] The device records the user's downtime. If the user stays in a specific location for a certain amount of time, the device saves the downtime as data. For example, it can collect data such as "stayed in the kitchen from 8:00 AM to 8:05 AM."

[0069] Step 3:

[0070] The device sends the collected location information and stop time data to the server at regular intervals (for example, every 5 minutes).

[0071] Step 4:

[0072] The server receives the data sent from the terminal, stores it in a database, and converts the received data into a format for analysis.

[0073] Step 5:

[0074] The server analyzes location information, movement information, and downtime patterns, for example, identifying a pattern that "the user is in the kitchen at 8:00 AM every morning."

[0075] Step 6:

[0076] Based on the analysis results, the server runs an algorithm to identify unnecessary movements and efficient routes for users, such as identifying specific data such as "there is a two-minute wait time for the microwave."

[0077] Step 7:

[0078] The server uses an optimization algorithm to select the best next action for the user, for example, suggesting an action such as "taking out the trash while the microwave is waiting."

[0079] Step 8:

[0080] The server generates data to notify the user of the selected action, summarizing the suggestion as a message such as "Take out the trash while the microwave is in standby mode."

[0081] Step 9:

[0082] The server transmits this notification data to the terminal.

[0083] Step 10:

[0084] The device displays the received notification data to the user in real time, for example, by sending a message to the user saying, "Take out the trash while the microwave is in standby mode."

[0085] Step 11:

[0086] Users can then take specific actions based on the notification, such as taking out the trash while the microwave is waiting. This allows users to live their lives efficiently without wasting time.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] While conventional assistant systems collect information on the user's location and movement, there are no systems that can effectively analyze this information and suggest beneficial next actions for the user in real time. As a result, users often waste time, which reduces the efficiency of their daily lives.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes means for collecting user location information and movement line information, means for transmitting the collected location information and movement line information to a central processing unit at regular intervals, means for the central processing unit to store the collected location information and movement line information, convert the collected information into an analytical format and analyze it, means for selecting the optimal next action for the user based on the analysis results and generating notification data, and means for transmitting the generated notification data to the user's information terminal and displaying it to the user in real time, thereby enabling users to use their time more efficiently and significantly improve the efficiency of their daily lives.

[0092] "User" refers to an individual or group of people who use the System.

[0093] "Location Information" is data that indicates a user's current geographic location.

[0094] "Flow line information" is data that indicates the trajectory or route that a user has taken.

[0095] "Dwell time" is data that indicates the amount of time a user remains in a particular location.

[0096] A "terminal" is a device carried or worn by a user, examples of which include wearable devices.

[0097] A "server" is a central computer that receives collected data, analyzes it, and sends the results to the user's device.

[0098] "Central processing unit" is a general term for the hardware and software installed on a server that stores, converts, and analyzes data.

[0099] "Notification data" refers to data that includes suggestions and instructions for the user that are generated based on the analysis results.

[0100] An "information terminal" is a device that receives notification data sent from a server and displays it to the user.

[0101] "Interval" refers to the time interval during which data is collected or transmitted.

[0102] This invention is an assistant system that helps users lead their daily lives efficiently. It collects and analyzes the user's location information, movement information, and downtime, and suggests the optimal next action. This system consists of three main components: the terminal (wearable device), the server, and the user.

[0103] Data collection

[0104] The device (wearable device) uses GPS and acceleration sensors to collect the user's location and movement information in real time. Specifically, if the user is in the kitchen at 08:00 AM, the data will be recorded as "08:00 AM - Kitchen." If the user moves to the living room at 08:05 AM, the data will be recorded as "08:05 AM - Living Room." The device also records the time the user is stationary in a particular location. For example, if the user is in the kitchen from 08:00 AM to 08:05 AM, the data will be collected as "Stayed in the kitchen from 08:00 AM to 08:05 AM."

[0105] Data transmission

[0106] The device sends the collected location information, movement information, and stop time data to a server at regular intervals (e.g., every 5 minutes). The data is sent using wireless communication technology (e.g., Wi-Fi or Bluetooth). This allows the data to be accumulated on the server and used for later analysis.

[0107] Data reception and storage

[0108] The server receives the data sent from the device. The received data is first stored in a database. For example, data such as "08:00 AM - Kitchen" and "08:05 AM - Living Room" is stored on the server.

[0109] Data conversion and analysis

[0110] The server converts the stored data into a format for analysis. This format conversion involves shaping the data and removing unnecessary information. The server then analyzes movement patterns and downtime. For example, it might analyze a pattern such as "every morning, the user goes to the kitchen at 8:00 AM."

[0111] Action proposal generation

[0112] The server selects the optimal next action based on the analysis results. For example, while the user is waiting for food to heat up in the microwave, it will suggest an action to use that waiting time efficiently. Specifically, it generates a suggestion such as "Take out the trash while the microwave is on." This suggestion is selected based on the user's past behavioral patterns and current situation (for example, whether the trash can is full).

[0113] User Notification

[0114] The server generates the selected action suggestion as notification data and sends it to the device. For example, it generates a message saying, "Take out the trash while the microwave is waiting." The device then receives the notification and displays it to the user in real time.

[0115] User execution

[0116] Users can check notifications from their devices and take suggested actions, such as taking out the trash while the microwave is running. This action allows users to use their time efficiently and get on with their daily tasks.

[0117] Specific examples

[0118] Situation

[0119] When a user is preparing breakfast, there is a few minutes of waiting while the ingredients heat up in the microwave.

[0120] Specific examples of data collection and transmission

[0121] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[0122] The device sends this data to the server every five minutes.

[0123] Examples of data reception and storage

[0124] The server receives data such as "07:30 - 07:33 Kitchen" and stores it in a database.

[0125] Specific examples of data conversion and analysis

[0126] The server converts the stored data into an analytical format and determines that "the user was using the microwave in the kitchen between 07:30 and 07:33."

[0127] Example of action proposal generation

[0128] The server determines that the user is waiting in the kitchen between 07:30 and 07:33, and uses this waiting time to suggest taking out the trash.

[0129] Examples of user notifications

[0130] The server generates notification data saying "Take out the garbage while the microwave is in standby mode" and transmits it to the terminal.

[0131] User execution example

[0132] Users can follow the notification and take out the trash while the microwave is waiting, allowing them to use their time effectively and improve their daily efficiency.

[0133] Example prompts to input to the generative AI model

[0134] User: "Can you give me some suggestions on how to make the most of my time waiting in the kitchen each morning?"

[0135] Prompt for generative AI model: "Suggest efficient actions that the user can take while waiting in the kitchen each morning, such as taking out the trash or preparing ingredients."

[0136] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0137] Step 1: Data collection

[0138] The device collects the user's location information and movement path information. Specifically, it uses GPS and an acceleration sensor to record the user's current location and movement trajectory. For example, if the user is in the kitchen at 8:00 AM, the data "8:00 AM - Kitchen" is collected. In addition, the device also records the user's downtime. For example, the data collected might be "Stayed in the kitchen from 8:00 AM to 8:05 AM." This data will be used as input for analysis in later steps.

[0139] Step 2: Send data

[0140] The device sends the collected location information, movement information, and stop time data to the server at regular intervals (e.g., every 5 minutes). Specifically, data such as "08:00 AM - kitchen" and "08:05 AM - living room" is sent to the server every 5 minutes. This transmission uses wireless communication technology (e.g., Wi-Fi, Bluetooth). This data is accumulated on the server and becomes input data for later analysis.

[0141] Step 3: Receiving and storing data

[0142] The server receives the data sent from the device. The received data is first stored in a database. For example, data such as "08:00 AM - Kitchen" and "08:05 AM - Living Room" is accumulated in the database. The output of this step is the location information and movement line information stored in the database.

[0143] Step 4: Data conversion and analysis

[0144] The server converts the stored data into a format for analysis. Specifically, it formats the data and deletes unnecessary information. Next, the server analyzes movement patterns and downtime. For example, it analyzes patterns such as "every morning, the user goes to the kitchen at 8:00 AM" and "there is a two-minute wait time for the microwave." The input data is the location information and movement information stored in the database, and the output data is the analysis results.

[0145] Step 5: Action proposal generation

[0146] The server selects the optimal next action based on the analysis results. Specifically, it suggests actions to use the user's waiting time efficiently while they wait for food to heat up in the microwave. For example, it generates a suggestion such as "Take out the trash while the microwave is on." The input to this step is the analysis results, and the output is the selected action suggestion (notification data).

[0147] Step 6: Notify users

[0148] The server sends the generated notification data to the user's device. For example, a notification message saying "Take out the trash while the microwave is in standby mode" is sent to the device. The device displays the received notification data in real time. The input data is the notification data, and the output is the message to be displayed to the user.

[0149] Step 7: Run the user

[0150] The user checks the notification from the device and performs the suggested action. For example, the user takes out the trash while the microwave is waiting. The input of this step is the notification message from the device, and the output is the user's action. This allows the user to use their time efficiently and improve the efficiency of their daily life.

[0151] (Application example 1)

[0152] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0153] In modern food delivery services, delivery workers must efficiently travel to multiple destinations. However, current systems do not provide suggestions for optimal routes or appropriate break times, resulting in inefficiency. They also have difficulty responding to real-time changes in the situation. This can lead to delivery delays and fatigue among delivery workers, resulting in a decline in overall service quality. Therefore, a system is needed that allows delivery workers to work efficiently and receive suggestions for optimal routes and break times in real time.

[0154] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0155] In this invention, the server includes means for collecting user location information, flow line information, and stop time, means for analyzing the collected location information, flow line information, and stop time, means for proposing an efficient delivery route based on the analysis results, and means for notifying the user of optimal actions and rest times in real time. This allows delivery personnel to receive suggestions for optimal routes and rest times in real time, thereby enabling delivery work to be performed efficiently, preventing delivery delays, and reducing fatigue of delivery personnel.

[0156] "Location Information" means data that describes the current location of a user or device, obtained using GPS or other location measurement technologies.

[0157] "Traffic information" is data showing the route taken by a user or device, and is generated from continuous recording of location information.

[0158] "Downtime" is data that shows the amount of time a user or device spends in a particular location.

[0159] "Analysis tools" are algorithms and software used to extract meaningful information and patterns from collected data.

[0160] "Means for suggesting the optimal next action" refers to systems or software that suggest the most efficient next action to the user based on the analysis results.

[0161] A "delivery route" refers to the route of locations that should be visited in the most efficient order when making a delivery.

[0162] "Real-time" means that processing and response occur immediately at the moment the data is generated.

[0163] "Means of notification" refers to devices or software that notify users of information or actions, including the ability to send notifications to smartphones or wearable devices.

[0164] A "prompt" is a textual suggestion or instruction that a generative AI model provides to a user.

[0165] A "generative AI model" is a machine learning algorithm or artificial intelligence technology that generates new information or suggestions based on data.

[0166] This invention relates to an assistance system that enables delivery personnel to deliver food efficiently. Specifically, this system collects and analyzes delivery personnel's location information, movement line information, and stop time, and proposes optimal delivery routes and break times in real time.

[0167] The system mainly consists of three main components: terminals, servers, and users.

[0168] Terminal

[0169] The terminal is a wearable device such as a smartphone or smart glasses. This terminal uses GPS and an acceleration sensor to collect real-time location information and movement information of the delivery person. It also records the time the delivery person stays in a specific location (stop time). For example, the terminal records location information such as "10:00 AM - Cafe" and "10:15 AM - Pizza place."

[0170] server

[0171] The server receives and stores the location information, movement information, and stop time data sent from the device. The server analyzes this data and generates the optimal next action for the delivery person. The analysis uses Python and R languages ​​to analyze movement patterns and propose efficient routes. It also uses generative AI models (e.g., TensorFlow, Scikit-learn) to prompt the user on the optimal course of action.

[0172] For example, the server generates a prompt such as, "The delivery person's current location is latitude 35.6895, longitude 139.6917. Please suggest the next best delivery destination. Please take traffic conditions and stop time into consideration." and notifies the delivery person of the analysis results.

[0173] user

[0174] Delivery workers receive suggestions from their devices. For example, they can receive a notification such as, "Your next delivery destination is the nearest cafe. After that, deliver to a pizza place." New suggestions are sent in real time as needed during delivery, allowing for efficient delivery route management.

[0175] As a concrete example, the terminal works as follows:

[0176] 1. The device periodically collects the delivery person's location information.

[0177] 2. The collected data is sent to the server and stored in a database (e.g., MySQL, MongoDB).

[0178] 3. The server analyzes the data using Python or R, and uses TensorFlow or Scikit-learn to predict the optimal next action using an AI model.

[0179] 4. A prompt is generated based on the analysis results and sent to the delivery person's device.

[0180] 5. Delivery personnel act on notifications and complete deliveries efficiently.

[0181] For example, a notification will appear on the delivery person's device stating, "The next best delivery from your current location is point A, based on distance, followed by a delivery at point B, taking current traffic conditions into account."

[0182] In this way, the present invention allows delivery personnel to work efficiently in real time, preventing delivery delays and contributing to an overall improvement in service quality.

[0183] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0184] Step 1:

[0185] The terminal obtains the delivery person's location information using a GPS sensor. It periodically collects location data (latitude and longitude) from the GPS sensor. The input is location information (latitude and longitude), and the output is the collected location data. Specifically, the GPS module inside the terminal reads the current latitude and longitude and temporarily stores the location information in memory.

[0186] Step 2:

[0187] The device sends the collected location information to the server at regular intervals (e.g., every 5 minutes). The data is sent using a RESTful API as the communication protocol. The input is the location data, and the output is the status of transmission to the server. Specifically, the device packages the location information in JSON format and sends the data to the server using an HTTP POST request.

[0188] Step 3:

[0189] The server stores the received location information in a database, using MySQL or MongoDB as the database. The input is the location data, and the output is the record status in the database. Specifically, the server parses the received location data, converts it into an appropriate format, and inserts it into the database using SQL queries or NoSQL operations.

[0190] Step 4:

[0191] The server analyzes the location information, movement line information, and stop times stored in the database. The analysis uses Python or R programming language to analyze movement line patterns and identify inefficient routes. The input is the stored location information, and the output is the analysis results (for example, movement line patterns). Specifically, the server runs an analysis script and extracts patterns of movement lines and stop times based on past location data.

[0192] Step 5:

[0193] The server uses a generative AI model based on the analysis results to calculate the next optimal action and route. TensorFlow and Scikit-learn are used as the generative AI model. The input is the analysis results, and the output is the proposed next action and route. Specifically, the server inputs the analysis results into the AI ​​model and predicts the optimal route and action.

[0194] Step 6:

[0195] The server notifies the device of the generated action proposal as a prompt. The prompt is created based on the output from the generative AI model and sent to the device in real time. The input is the action proposal, and the output is a notification to the device. Specifically, the server converts the proposal into a text prompt and notifies the delivery person's device using Firebase Cloud Messaging (FCM).

[0196] Step 7:

[0197] The user (delivery person) acts according to the suggestions notified to the terminal. For example, the user confirms the notification that "The next delivery destination is the nearest cafe. After that, deliver to the pizza place," and makes the delivery along the specified route. The input is the notified prompt sentence, and the output is the executed action. In concrete terms, the delivery person confirms the contents of the notification and moves and delivers according to the instructions.

[0198] This allows delivery personnel to deliver efficiently, shortening delivery times and improving work efficiency.

[0199] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0200] The present invention provides a system for collecting location information, movement information, and emotional information of a user to improve the efficiency of the user's life. This system recognizes the user's emotions and, taking those emotions into consideration, suggests the optimal next action, thereby reducing the user's psychological stress. Specific embodiments of the system are described below.

[0201] System Overview

[0202] 1. Data Collection

[0203] The wearable device uses GPS and acceleration sensors to collect real-time location and movement information for the user. For example, it records current location data and movement information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room."

[0204] The device records the user's downtime, for example, collecting data such as "stayed in the kitchen" from "08:00 AM" to "08:05 AM."

[0205] The device uses its built-in camera, microphone, and biometric sensors to collect biometric signals such as the user's facial expressions, voice, and heart rate, and transmits them to the emotion engine.

[0206] 2. Emotion recognition

[0207] The server analyzes facial expressions, voice, and biometric signals sent from the device to identify the user's emotional state, for example, recognizing whether the user is in a specific emotional state such as anger, sadness, or joy.

[0208] 3. Data Transmission

[0209] The terminal transmits the collected location information, movement line information, and stop time data to the server at regular intervals (for example, every 5 minutes).

[0210] 4. Data Analysis

[0211] The server receives the transmitted data, stores it in a database, and converts the received data into a format for analysis.

[0212] The server analyzes location information, movement information, and downtime patterns, for example, identifying a pattern that "the user is in the kitchen every morning at 8:00 AM."

[0213] 5. Emotion-based action suggestions

[0214] The server then takes into account the collected emotional information and runs an algorithm to identify unnecessary movements and efficient routes for the user, making specific suggestions such as "If the user is tired, they should take some time to relax."

[0215] The server uses an optimization algorithm to select the next action, taking emotional information into account. For example, it might suggest an action such as "take a deep breath while waiting for the microwave to turn on."

[0216] 6. Notification of Proposed Actions

[0217] The server generates data to notify the user of the selected action. For example, in addition to the message "Take out the trash while the microwave is in standby mode," it generates a message such as "Take a deep breath and relax."

[0218] The server transmits this notification data to the terminal.

[0219] 7. Notice to Users

[0220] The device displays the received notification data to the user in real time, for example, notifying the user of multiple messages such as "Take out the trash while the microwave is in standby mode" and "Take a deep breath and relax."

[0221] 8. User execution

[0222] Users can then take specific actions based on the notification, such as taking out the trash or taking a deep breath while waiting for the microwave to heat up. This allows users to live more efficiently without wasting time and reduces psychological stress.

[0223] Specific examples

[0224] Situation

[0225] When a user is preparing breakfast, they have to wait a few minutes while the food heats up in the microwave, which can be stressful for the user.

[0226] Data collection and transmission

[0227] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[0228] The device recognizes signs of stress from the user's facial expressions and heart rate.

[0229] Data analysis and sentiment-based recommendations

[0230] The server identifies the waiting time while the user is using the microwave in the kitchen and recognizes when the user is stressed based on the emotion engine.

[0231] The server selects "take out the trash while the microwave is waiting" and "take a deep breath and relax" as optimal actions and generates notification data.

[0232] User Notification

[0233] Notification data is sent from the server to the device, and the device displays to the user "Take out the trash while the microwave is waiting" and "Take a deep breath and relax."

[0234] User execution

[0235] The user follows the notification, takes out the trash while the microwave is waiting, and takes a deep breath, allowing the user to use their time efficiently and reduce stress.

[0236] By combining this system with an emotion engine, users can enjoy the benefits of effectively utilizing their time while reducing their psychological stress.

[0237] The processing flow will be explained below.

[0238] Step 1:

[0239] The device uses GPS and acceleration sensors to collect real-time location and movement information of the user. For example, if the user is in the kitchen at 8:00 AM, the device will record the location information as "8:00 AM - Kitchen."

[0240] Step 2:

[0241] The device records the user's downtime. If the user stays in a specific location for a certain amount of time, the device saves the downtime as data. For example, data such as "stayed in the kitchen from 8:00 AM to 8:05 AM" can be collected.

[0242] Step 3:

[0243] The device uses its built-in camera, microphone, and biometric sensors to collect the user's facial expressions, voice, heart rate, and other biometric signals in real time, and transmits this data to the emotion engine.

[0244] Step 4:

[0245] The server's emotion engine analyzes the transmitted facial expressions, voice, and biometric signals to determine the user's emotional state, for example, determining that the user is feeling stressed.

[0246] Step 5:

[0247] The terminal transmits the collected location information, movement information, and stop time data to the server at regular intervals (for example, every 5 minutes).

[0248] Step 6:

[0249] The server receives the data and stores it in a database. It converts the received data into an analytical format and analyzes the location and movement patterns. For example, it can identify a pattern that "the user is in the kitchen every morning at 8:00 AM."

[0250] Step 7:

[0251] The server then runs an algorithm that uses the analysis results to identify unnecessary movements and efficient routes, taking into account the user's emotional information and identifying specific data, such as "the microwave has a two-minute waiting time."

[0252] Step 8:

[0253] The server uses an optimization algorithm to select the best next action for the user based on the emotional information. For example, it could suggest actions such as "Take out the trash while the microwave is running" and "Take a deep breath and relax."

[0254] Step 9:

[0255] The server generates data to notify the user of the selected action, summarizing specific suggestions as messages such as "Take out the trash while the microwave is on standby" and "Take a deep breath and relax."

[0256] Step 10:

[0257] The server transmits the generated notification data to the terminal.

[0258] Step 11:

[0259] The device displays the received notification data to the user in real time, for example, notifying the user of multiple messages such as "Take out the trash while the microwave is in standby mode" and "Take a deep breath and relax."

[0260] Step 12:

[0261] Users can then take specific actions based on the notification, such as taking out the trash while the microwave is running or taking deep breaths to relax. This allows users to reduce wasted time, live more efficiently, and simultaneously reduce psychological stress.

[0262] Example 2

[0263] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0264] In modern life, users are expected to reduce unnecessary movements and use their time efficiently. However, conventional systems suggest next actions based on simple location and movement information without considering the user's emotions or psychological state, making it difficult to reduce the user's psychological stress.

[0265] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting location information, movement line information, and emotion information of the user, means for analyzing the collected location information, movement line information, and emotion information, means for proposing an optimal next action for the user based on the analysis results, means for notifying the user of the proposed action, and means for evaluating the user's behavior in accordance with the proposed action. This makes it possible to use time efficiently while reducing the user's psychological stress.

[0266] "User location information" means data that indicates the user's current geographic location.

[0267] "Traffic information" is data that indicates the route or trajectory of a user's movement when moving from one place to another.

[0268] "Emotional information" is data that indicates the user's psychological state and emotional fluctuations, and is extracted from facial expressions, voice, biometric signals, etc.

[0269] "Downtime" is data that records the length of time a user stays in a particular location.

[0270] "Analytics" refers to the algorithms and software used to process and analyze collected data to derive insights.

[0271] "Means of suggestion" refers to algorithms or software that specifically suggest optimal actions for users based on the analysis results.

[0272] "Notification means" refers to a device or communication means that notifies the user of the proposed action, such as a smartphone or smartwatch.

[0273] "Means for evaluating behavior" refers to algorithms or software that evaluate the results of a suggested action after the user performs it and use that information to make future suggestions.

[0274] The present invention is a system for collecting location information, movement information, and emotional information of a user to improve the efficiency of the user's life. This system recognizes the user's emotions and takes those emotions into consideration to suggest the optimal next action, thereby reducing the user's psychological stress. Specific embodiments of the system are described below.

[0275] Data collection

[0276] The device (wearable device) uses a GPS sensor and an acceleration sensor to collect real-time location and movement information of the user. For example, it records current location data and movement information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room." The device also records the user's inactivity time. Furthermore, it uses a built-in camera, microphone, and biometric sensors to collect biometric signals such as the user's facial expressions, voice, and heart rate.

[0277] Data transmission

[0278] The device sends the collected location information, movement information, and biometric signals to a server at regular intervals (for example, every 5 minutes) using Wi-Fi or mobile data communication.

[0279] emotion recognition

[0280] The server receives facial expression data, voice data, and biometric signals sent from the device and uses an emotion recognition algorithm to identify the user's emotional state. For example, if the user's facial expression is grim, it will be recognized as "anger."

[0281] Data analysis

[0282] The server stores the received data in a database, converts the stored data into an analytical format, and analyzes the location and movement patterns. This process includes a data cleansing process to remove noise.

[0283] Suggesting actions based on emotions

[0284] The server runs an algorithm to suggest the best next action that takes emotional information into account, for example, "If the user is tired, take some time to relax." It also selects the next action based on the optimization algorithm, choosing an action such as "take some deep breaths while waiting for the microwave to heat up."

[0285] Proposed action notifications

[0286] The server generates message data to notify the selected action and sends it to the device. For example, it creates messages such as "Take out the trash while the microwave is in standby mode" and "Take a deep breath and relax."

[0287] User Notification

[0288] The device analyzes the notification data received from the server and displays it on the display screen in real time. For example, messages such as "Take out the trash while the microwave is on standby" and "Take a deep breath and relax" may be displayed on the screen of a smartwatch or smartphone.

[0289] User execution

[0290] Users can take specific actions based on notifications from their device, such as taking out the trash while the microwave is running and taking deep breaths, allowing them to use their time efficiently and reduce psychological stress.

[0291] Specific examples

[0292] Situation

[0293] When a user is preparing breakfast, they have to wait a few minutes while the food heats up in the microwave, which can be stressful for the user.

[0294] Data collection and transmission

[0295] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[0296] The device recognizes signs of stress from the user's facial expressions and heart rate.

[0297] Data analysis and sentiment-based recommendations

[0298] The server identifies the waiting time while the user is using the microwave in the kitchen and recognizes when the user is stressed based on the emotion engine.

[0299] The server selects "take out the trash while the microwave is waiting" and "take a deep breath and relax" as optimal actions and generates notification data.

[0300] User Notification

[0301] Notification data is sent from the server to the device, and the device displays to the user "Take out the trash while the microwave is waiting" and "Take a deep breath and relax."

[0302] User execution

[0303] The user follows the notification, takes out the trash while the microwave is waiting, and takes a deep breath, allowing the user to use their time efficiently and reduce stress.

[0304] By combining this system with an emotion engine, users can enjoy the benefits of effectively utilizing their time while reducing their psychological stress.

[0305] Prompt Sentence Examples

[0306] "Please explain a system that collects location, movement, and emotional information from users and suggests optimal actions."

[0307] "Please tell me more about how you use the Emotion Engine to reduce user stress."

[0308] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0309] Step 1: Data collection

[0310] The device uses a GPS sensor and an acceleration sensor to obtain real-time location and movement information for the user. As input, the sensor receives the user's current location and route traveled, collecting information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room." As output, this data is stored in the device's internal memory. Additionally, the built-in camera captures the user's facial expressions, the microphone collects the user's voice in real time, and the biometric sensor captures vital signs such as heart rate and body temperature.

[0311] Step 2: Send data

[0312] The device transmits the collected location information, movement information, and biometric signals to the server at regular intervals (e.g., every 5 minutes). As input, data stored in the internal memory is retrieved and transmitted via Wi-Fi or mobile data communication. As output, data packets are generated from the device to the server and transferred to the server.

[0313] Step 3: Emotion Recognition

[0314] The server receives facial expression data, voice data, and biometric signals sent from the device. It receives these various data as input and runs an emotion recognition algorithm to identify the user's emotional state. For example, if the user's facial expression is grim, it will be recognized as "anger." The output is the identified emotional state, which is then stored in a database.

[0315] Step 4: Data analysis

[0316] The server stores the received data in a database. As input, it takes the received data and stores it in the database in SQL or NoSQL format. It then converts the stored data into a format for analysis. This includes a data cleansing process, for example, removing noise from location information. As output, clean data is generated and passed to the next analysis process.

[0317] Step 5: Consider emotions and suggest actions

[0318] The server runs an algorithm that takes emotional information into account to optimize the user's movement path. Location information, movement path information, pause time, and emotional information are taken as inputs and fed into the analysis algorithm. For example, a suggestion such as "If the user is tired, have them take some time to relax" is made. Optimized action suggestions are generated as output.

[0319] Step 6: Notification of proposed action

[0320] The server generates message data to notify the selected action. As input, it receives the action proposal generated in step 5 and creates a specific message. For example, messages such as "Take out the trash while the microwave is waiting" and "Take a deep breath and relax" are generated. As output, this notification data is sent to the device.

[0321] Step 7: Notify users

[0322] The device analyzes the notification data received from the server and displays it on the display screen in real time. As input, it takes the message received from the server and displays it on the user interface (UI). For example, messages such as "Take out the trash while the microwave is waiting" and "Take a deep breath and relax" are displayed on the screen of a smartwatch or smartphone. As output, specific instructions for action are presented to the user.

[0323] Step 8: Run the user

[0324] The user takes specific actions according to the notifications from the device. As input, the user receives the notification message from the device and starts the action according to the instructions. For example, the user takes out the trash while waiting for the microwave to run and takes a deep breath. As output, the user's action is completed, and efficient time utilization and reduction of psychological stress are achieved.

[0325] (Application example 2)

[0326] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0327] Conventional systems only collect and analyze user location and movement information, and are limited in their ability to suggest specific next actions that take into account the user's emotions and psychological state. This makes it difficult to encourage efficient behavior while reducing the user's psychological stress in scenarios that involve waiting, such as food delivery. The objective of the present invention is to solve this problem and provide a system that can suggest optimal actions that take into account the user's emotions and waiting time.

[0328] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user location information and movement line information, means for analyzing the collected location information, movement line information, and emotional information, means for proposing the optimal next action for the user based on the analysis results, and means for notifying the user in real time of the optimal action to take while waiting for food delivery. This makes it possible to efficiently utilize the waiting time while reducing the user's psychological stress.

[0329] "User Location Information" means the geographic coordinate information of the User's current location obtained by a GPS device or other location measurement means.

[0330] "Traffic information" is data that indicates the route and direction a user travels within a specific period of time.

[0331] "Emotional information" is information about the user's emotional state obtained by analyzing the user's facial expressions, voice, biometric signals, etc.

[0332] "Means for analyzing" refers to hardware or software functionality for processing and analyzing collected data to extract useful information.

[0333] "Means for suggesting next actions" refers to the function of an algorithm or system that suggests optimal actions or activities for users based on the analysis results.

[0334] "Waiting for food delivery" refers to the time while a user is waiting for food delivery.

[0335] "Means of notifying users of optimal actions in real time" refers to digital communication methods that take into account the user's current situation and emotional state and instantly inform the user of suggested actions.

[0336] This invention is a system that collects and analyzes a user's location information, movement information, and emotional information to suggest optimal actions for the user, thereby promoting efficient behavior and reducing psychological stress, particularly while waiting for food delivery.

[0337] System Overview

[0338] 1. Data Collection

[0339] The server uses GPS devices and other location measurement means to collect user location and movement information.

[0340] The server uses software for facial recognition, voice analysis, and biometric signal analysis to obtain the user's emotional information.

[0341] 2. Data Analysis

[0342] The server analyzes the collected location information, movement information, and emotional information and stores it in a database using an analysis platform and AI model.

[0343] 3. Action suggestions

[0344] The server then suggests the best next action for the user based on the analysis results, which includes an algorithm to reduce the user's psychological stress.

[0345] 4. Real-time notifications

[0346] The server communicates with the user's smartphone or appropriate end device using a notification API to notify the user of the optimal action in real time.

[0347] Specific examples

[0348] For example, if a user orders food delivery and is waiting for it to arrive, the system will follow these steps:

[0349] 1. Data Collection

[0350] The device collects the user's location information (e.g., "08:00 AM - Home"), movement information (e.g., "08:10 AM - Kitchen"), and emotional information (e.g., "stress" state based on facial expression analysis).

[0351] 2. Data Analysis

[0352] The server receives this information and uses an emotion engine to determine the user's psychological state and waiting status.

[0353] 3. Action suggestions

[0354] The server generates the best suggestion: "Your food delivery will arrive in 20 minutes. Take a deep breath."

[0355] 4. Notification

[0356] The server sends a notification to the device, and the user receives the suggestion via their smartphone.

[0357] This system allows users to use their time efficiently while reducing psychological stress while waiting for food delivery.

[0358] Prompt Sentence Examples

[0359] "Given the following setup, create an app that suggests optimal actions while waiting for a food delivery, taking into account the user's emotions and location. The user's emotional state can be 'stress', 'neutral', or 'happy'. The location can be 'home' or 'office'. Start 30 minutes before the scheduled delivery time."

[0360] This prompt can be used to leverage a generative AI model to flexibly generate more detailed suggestions and notification content.

[0361] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0362] Step 1:

[0363] Data collection

[0364] Specific behavior:

[0365] The device collects real-time location, movement, and emotional information from the user. Location information is obtained using GPS devices and other location measurement methods, movement information is obtained by tracking the user's movement path, and emotional information is obtained using facial recognition, voice analysis, and biometric signal analysis.

[0366] input:

[0367] User location information (e.g., "08:00 AM - home"), movement path information (e.g., "08:10 AM - kitchen"), and emotional information (e.g., "stressed" state based on facial expression analysis).

[0368] output:

[0369] A dataset of collected location, movement, and emotion information.

[0370] Step 2:

[0371] Data transmission

[0372] Specific behavior:

[0373] The device transmits the collected location information, movement information, and emotion information to a server at regular intervals (e.g., every 5 minutes) using a wireless communication module.

[0374] input:

[0375] A dataset of collected location, movement, and emotion information.

[0376] output:

[0377] The dataset sent to the server.

[0378] Step 3:

[0379] Data reception and storage

[0380] Specific behavior:

[0381] The server receives the data sent from the terminal and stores it in a database. The received data is converted into an analysis format.

[0382] input:

[0383] A dataset of location information, movement information, and emotional information sent from the device.

[0384] output:

[0385] Datasets for analysis stored in a database.

[0386] Step 4:

[0387] Data analysis

[0388] Specific behavior:

[0389] The server analyzes the user's behavioral patterns and emotional state based on location, movement, and emotional information stored in the database, and uses a generative AI model to evaluate the user's psychological state and suggest actions.

[0390] input:

[0391] A dataset for analyzing location information, movement information, and emotion information stored in a database.

[0392] output:

[0393] Analysis results (e.g., user is stressed, in the kitchen, waiting for food delivery).

[0394] Step 5:

[0395] Action suggestion generation

[0396] Specific behavior:

[0397] Based on the results of the data analysis, the server generates optimal actions to suggest to the user, such as "take deep breaths" or "listen to music to relax" to reduce stress.

[0398] input:

[0399] Analysis results (e.g., user is stressed, in the kitchen, waiting for food delivery).

[0400] output:

[0401] Specific suggestions for action (e.g., "Your food delivery will arrive in 20 minutes. Take a deep breath.").

[0402] Step 6:

[0403] Proposal Notification

[0404] Specific behavior:

[0405] The server then creates notification data based on the generated action suggestions and sends it to the device, which then displays the notification data to the user in real time.

[0406] input:

[0407] Action suggestions (e.g., "Your food delivery will arrive in 20 minutes. Take a deep breath.").

[0408] output:

[0409] Suggestions notified to users.

[0410] Step 7:

[0411] User execution

[0412] Specific behavior:

[0413] The user receives a notification from the device and performs a suggested action, such as "take a deep breath" or "stretch."

[0414] input:

[0415] Notified suggestions (e.g., "Your food delivery will arrive in 20 minutes. Take a deep breath.").

[0416] output:

[0417] The action taken (e.g., the user took a deep breath).

[0418] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0419] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0420] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0421] [Second embodiment]

[0422] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0423] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0424] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0425] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0426] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0427] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0428] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0429] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0430] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0431] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0432] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0433] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0434] This invention provides a system that functions as an assistant to help users lead their daily lives efficiently. It collects and analyzes the user's location information, movement path information, and downtime, and suggests the optimal next action. This system consists of three main components: the terminal (wearable device), the server, and the user.

[0435] System Overview

[0436] 1. Data Collection

[0437] The wearable device uses GPS and acceleration sensors to collect real-time location and movement information for the user. For example, it records current location data and movement information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room."

[0438] The device also records the user's downtime, for example, "staying in the kitchen" from 8:00 AM to 8:05 AM.

[0439] 2. Data Transmission

[0440] The device sends the collected location information, movement information, and stop time data to the server at regular intervals (e.g., every 5 minutes), where the data is accumulated and used for analysis.

[0441] 3. Data analysis

[0442] The server receives the data and first stores it in a database. It then converts it into an analytical format and analyzes the user's movement patterns and downtime. For example, it can identify a pattern such as "every morning, the user goes to the kitchen at 8:00 AM."

[0443] The server uses the collected data to run algorithms that identify wasted time and efficient routes for users, generating specific action suggestions such as "Take out the trash during the two minutes you wait for the microwave."

[0444] 4. Action proposals

[0445] The server then selects the optimal next action based on the analysis results, such as suggesting other housework that would be best suited to the time the user is waiting in the kitchen.

[0446] The server generates notification data that suggests the selected action, and generates a message such as "Take out the garbage while the microwave is in standby mode."

[0447] 5. Notice to Users

[0448] The server sends the generated notification data to the device, which then displays the received notification data to the user in real time. For example, it displays a message such as "Take out the trash while the microwave is in standby mode."

[0449] 6. User execution

[0450] The user follows the notification to take out the trash. This suggestion helps users reduce wasted time and do their housework efficiently.

[0451] Specific examples

[0452] Situation

[0453] A user is preparing breakfast and has to wait a few minutes while the ingredients heat up in the microwave.

[0454] Data collection and transmission

[0455] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[0456] Data analysis and recommendations

[0457] The server identifies the amount of time a user waits while using a microwave in the kitchen, and also determines from past logs that a nearby trash can is full.

[0458] The server selects "take out the garbage while the microwave is in standby mode" as the optimal action and generates notification data.

[0459] User Notification

[0460] Notification data is sent from the server to the device, and the device displays a message to the user saying, "Take out the trash while the microwave is in standby mode."

[0461] User execution

[0462] The user follows the notification and takes out the trash while the microwave is waiting.

[0463] This system allows users to effectively utilize their time and improve daily efficiency.The effects of the present invention are realized by such specific embodiments.

[0464] The processing flow will be explained below.

[0465] Step 1:

[0466] The device collects the user's current location and movement information in real time using GPS and acceleration sensors. For example, if the user is in the kitchen, the device will record the location information as "08:00 AM - Kitchen."

[0467] Step 2:

[0468] The device records the user's downtime. If the user stays in a specific location for a certain amount of time, the device saves the downtime as data. For example, it can collect data such as "stayed in the kitchen from 8:00 AM to 8:05 AM."

[0469] Step 3:

[0470] The device sends the collected location information and stop time data to the server at regular intervals (for example, every 5 minutes).

[0471] Step 4:

[0472] The server receives the data sent from the terminal, stores it in a database, and converts the received data into a format for analysis.

[0473] Step 5:

[0474] The server analyzes location information, movement information, and downtime patterns, for example, identifying a pattern that "the user is in the kitchen at 8:00 AM every morning."

[0475] Step 6:

[0476] Based on the analysis results, the server runs an algorithm to identify unnecessary movements and efficient routes for users, such as identifying specific data such as "there is a two-minute wait time for the microwave."

[0477] Step 7:

[0478] The server uses an optimization algorithm to select the best next action for the user, for example, suggesting an action such as "taking out the trash while the microwave is waiting."

[0479] Step 8:

[0480] The server generates data to notify the user of the selected action, summarizing the suggestion as a message such as "Take out the trash while the microwave is in standby mode."

[0481] Step 9:

[0482] The server transmits this notification data to the terminal.

[0483] Step 10:

[0484] The device displays the received notification data to the user in real time, for example, by sending a message to the user saying, "Take out the trash while the microwave is in standby mode."

[0485] Step 11:

[0486] Users can then take specific actions based on the notification, such as taking out the trash while the microwave is waiting. This allows users to live their lives efficiently without wasting time.

[0487] Example 1

[0488] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0489] While conventional assistant systems collect information on the user's location and movement, there are no systems that can effectively analyze this information and suggest beneficial next actions for the user in real time. As a result, users often waste time, which reduces the efficiency of their daily lives.

[0490] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0491] In this invention, the server includes means for collecting user location information and movement line information, means for transmitting the collected location information and movement line information to a central processing unit at regular intervals, means for the central processing unit to store the collected location information and movement line information, convert the collected information into an analytical format and analyze it, means for selecting the optimal next action for the user based on the analysis results and generating notification data, and means for transmitting the generated notification data to the user's information terminal and displaying it to the user in real time, thereby enabling users to use their time more efficiently and significantly improve the efficiency of their daily lives.

[0492] "User" refers to an individual or group of people who use the System.

[0493] "Location Information" is data that indicates a user's current geographic location.

[0494] "Flow line information" is data that indicates the trajectory or route that a user has taken.

[0495] "Dwell time" is data that indicates the amount of time a user remains in a particular location.

[0496] A "terminal" is a device carried or worn by a user, examples of which include wearable devices.

[0497] A "server" is a central computer that receives collected data, analyzes it, and sends the results to the user's device.

[0498] "Central processing unit" is a general term for the hardware and software installed on a server that stores, converts, and analyzes data.

[0499] "Notification data" refers to data that includes suggestions and instructions for the user that are generated based on the analysis results.

[0500] An "information terminal" is a device that receives notification data sent from a server and displays it to the user.

[0501] "Interval" refers to the time interval during which data is collected or transmitted.

[0502] This invention is an assistant system that helps users lead their daily lives efficiently. It collects and analyzes the user's location information, movement information, and downtime, and suggests the optimal next action. This system consists of three main components: the terminal (wearable device), the server, and the user.

[0503] Data collection

[0504] The device (wearable device) uses GPS and acceleration sensors to collect the user's location and movement information in real time. Specifically, if the user is in the kitchen at 08:00 AM, the data will be recorded as "08:00 AM - Kitchen." If the user moves to the living room at 08:05 AM, the data will be recorded as "08:05 AM - Living Room." The device also records the time the user is stationary in a particular location. For example, if the user is in the kitchen from 08:00 AM to 08:05 AM, the data will be collected as "Stayed in the kitchen from 08:00 AM to 08:05 AM."

[0505] Data transmission

[0506] The device sends the collected location information, movement information, and stop time data to a server at regular intervals (e.g., every 5 minutes). The data is sent using wireless communication technology (e.g., Wi-Fi or Bluetooth). This allows the data to be accumulated on the server and used for later analysis.

[0507] Data reception and storage

[0508] The server receives the data sent from the device. The received data is first stored in a database. For example, data such as "08:00 AM - Kitchen" and "08:05 AM - Living Room" is stored on the server.

[0509] Data conversion and analysis

[0510] The server converts the stored data into a format for analysis. This format conversion involves shaping the data and removing unnecessary information. The server then analyzes movement patterns and downtime. For example, it might analyze a pattern such as "every morning, the user goes to the kitchen at 8:00 AM."

[0511] Action proposal generation

[0512] The server selects the optimal next action based on the analysis results. For example, while the user is waiting for food to heat up in the microwave, it will suggest an action to use that waiting time efficiently. Specifically, it generates a suggestion such as "Take out the trash while the microwave is on." This suggestion is selected based on the user's past behavioral patterns and current situation (for example, whether the trash can is full).

[0513] User Notification

[0514] The server generates the selected action suggestion as notification data and sends it to the device. For example, it generates a message saying, "Take out the trash while the microwave is waiting." The device then receives the notification and displays it to the user in real time.

[0515] User execution

[0516] Users can check notifications from their devices and take suggested actions, such as taking out the trash while the microwave is running. This action allows users to use their time efficiently and get on with their daily tasks.

[0517] Specific examples

[0518] Situation

[0519] When a user is preparing breakfast, there is a few minutes of waiting while the ingredients heat up in the microwave.

[0520] Specific examples of data collection and transmission

[0521] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[0522] The device sends this data to the server every five minutes.

[0523] Examples of data reception and storage

[0524] The server receives data such as "07:30 - 07:33 Kitchen" and stores it in a database.

[0525] Specific examples of data conversion and analysis

[0526] The server converts the stored data into an analytical format and determines that "the user was using the microwave in the kitchen between 07:30 and 07:33."

[0527] Example of action proposal generation

[0528] The server determines that the user is waiting in the kitchen between 07:30 and 07:33, and uses this waiting time to suggest taking out the trash.

[0529] Examples of user notifications

[0530] The server generates notification data saying "Take out the garbage while the microwave is in standby mode" and transmits it to the terminal.

[0531] User execution example

[0532] Users can follow the notification and take out the trash while the microwave is waiting, allowing them to use their time effectively and improve their daily efficiency.

[0533] Example prompts to input to the generative AI model

[0534] User: "Can you give me some suggestions on how to make the most of my time waiting in the kitchen each morning?"

[0535] Prompt for generative AI model: "Suggest efficient actions that the user can take while waiting in the kitchen each morning, such as taking out the trash or preparing ingredients."

[0536] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0537] Step 1: Data collection

[0538] The device collects the user's location information and movement path information. Specifically, it uses GPS and an acceleration sensor to record the user's current location and movement trajectory. For example, if the user is in the kitchen at 8:00 AM, the data "8:00 AM - Kitchen" is collected. In addition, the device also records the user's downtime. For example, the data collected might be "Stayed in the kitchen from 8:00 AM to 8:05 AM." This data will be used as input for analysis in later steps.

[0539] Step 2: Send data

[0540] The device sends the collected location information, movement information, and stop time data to the server at regular intervals (e.g., every 5 minutes). Specifically, data such as "08:00 AM - kitchen" and "08:05 AM - living room" is sent to the server every 5 minutes. This transmission uses wireless communication technology (e.g., Wi-Fi, Bluetooth). This data is accumulated on the server and becomes input data for later analysis.

[0541] Step 3: Receiving and storing data

[0542] The server receives the data sent from the device. The received data is first stored in a database. For example, data such as "08:00 AM - Kitchen" and "08:05 AM - Living Room" is accumulated in the database. The output of this step is the location information and movement line information stored in the database.

[0543] Step 4: Data conversion and analysis

[0544] The server converts the stored data into a format for analysis. Specifically, it formats the data and deletes unnecessary information. Next, the server analyzes movement patterns and downtime. For example, it analyzes patterns such as "every morning, the user goes to the kitchen at 8:00 AM" and "there is a two-minute wait time for the microwave." The input data is the location information and movement information stored in the database, and the output data is the analysis results.

[0545] Step 5: Action proposal generation

[0546] The server selects the optimal next action based on the analysis results. Specifically, it suggests actions to use the user's waiting time efficiently while they wait for food to heat up in the microwave. For example, it generates a suggestion such as "Take out the trash while the microwave is on." The input to this step is the analysis results, and the output is the selected action suggestion (notification data).

[0547] Step 6: Notify users

[0548] The server sends the generated notification data to the user's device. For example, a notification message saying "Take out the trash while the microwave is in standby mode" is sent to the device. The device displays the received notification data in real time. The input data is the notification data, and the output is the message to be displayed to the user.

[0549] Step 7: Run the user

[0550] The user checks the notification from the device and performs the suggested action. For example, the user takes out the trash while the microwave is waiting. The input of this step is the notification message from the device, and the output is the user's action. This allows the user to use their time efficiently and improve the efficiency of their daily life.

[0551] (Application example 1)

[0552] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0553] In modern food delivery services, delivery workers must efficiently travel to multiple destinations. However, current systems do not provide suggestions for optimal routes or appropriate break times, resulting in inefficiency. They also have difficulty responding to real-time changes in the situation. This can lead to delivery delays and fatigue among delivery workers, resulting in a decline in overall service quality. Therefore, a system is needed that allows delivery workers to work efficiently and receive suggestions for optimal routes and break times in real time.

[0554] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0555] In this invention, the server includes means for collecting user location information, flow line information, and stop time, means for analyzing the collected location information, flow line information, and stop time, means for proposing an efficient delivery route based on the analysis results, and means for notifying the user of optimal actions and rest times in real time. This allows delivery personnel to receive suggestions for optimal routes and rest times in real time, thereby enabling delivery work to be performed efficiently, preventing delivery delays, and reducing fatigue of delivery personnel.

[0556] "Location Information" means data that describes the current location of a user or device, obtained using GPS or other location measurement technologies.

[0557] "Traffic information" is data showing the route taken by a user or device, and is generated from continuous recording of location information.

[0558] "Downtime" is data that shows the amount of time a user or device spends in a particular location.

[0559] "Analysis tools" are algorithms and software used to extract meaningful information and patterns from collected data.

[0560] "Means for suggesting the optimal next action" refers to systems or software that suggest the most efficient next action to the user based on the analysis results.

[0561] A "delivery route" refers to the route of locations that should be visited in the most efficient order when making a delivery.

[0562] "Real-time" means that processing and response occur immediately at the moment the data is generated.

[0563] "Means of notification" refers to devices or software that notify users of information or actions, including the ability to send notifications to smartphones or wearable devices.

[0564] A "prompt" is a textual suggestion or instruction that a generative AI model provides to a user.

[0565] A "generative AI model" is a machine learning algorithm or artificial intelligence technology that generates new information or suggestions based on data.

[0566] This invention relates to an assistance system that enables delivery personnel to deliver food efficiently. Specifically, this system collects and analyzes delivery personnel's location information, movement line information, and stop time, and proposes optimal delivery routes and break times in real time.

[0567] The system mainly consists of three main components: terminals, servers, and users.

[0568] Terminal

[0569] The terminal is a wearable device such as a smartphone or smart glasses. This terminal uses GPS and an acceleration sensor to collect real-time location information and movement information of the delivery person. It also records the time the delivery person stays in a specific location (stop time). For example, the terminal records location information such as "10:00 AM - Cafe" and "10:15 AM - Pizza place."

[0570] server

[0571] The server receives and stores the location information, movement information, and stop time data sent from the device. The server analyzes this data and generates the optimal next action for the delivery person. The analysis uses Python and R languages ​​to analyze movement patterns and propose efficient routes. It also uses generative AI models (e.g., TensorFlow, Scikit-learn) to prompt the user on the optimal course of action.

[0572] For example, the server generates a prompt such as, "The delivery person's current location is latitude 35.6895, longitude 139.6917. Please suggest the next best delivery destination. Please take traffic conditions and stop time into consideration." and notifies the delivery person of the analysis results.

[0573] user

[0574] Delivery workers receive suggestions from their devices. For example, they can receive a notification such as, "Your next delivery destination is the nearest cafe. After that, deliver to a pizza place." New suggestions are sent in real time as needed during delivery, allowing for efficient delivery route management.

[0575] As a concrete example, the terminal works as follows:

[0576] 1. The device periodically collects the delivery person's location information.

[0577] 2. The collected data is sent to the server and stored in a database (e.g., MySQL, MongoDB).

[0578] 3. The server analyzes the data using Python or R, and uses TensorFlow or Scikit-learn to predict the optimal next action using an AI model.

[0579] 4. A prompt is generated based on the analysis results and sent to the delivery person's device.

[0580] 5. Delivery personnel act on notifications and complete deliveries efficiently.

[0581] For example, a notification will appear on the delivery person's device stating, "The next best delivery from your current location is point A, based on distance, followed by a delivery at point B, taking current traffic conditions into account."

[0582] In this way, the present invention allows delivery personnel to work efficiently in real time, preventing delivery delays and contributing to an overall improvement in service quality.

[0583] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0584] Step 1:

[0585] The terminal obtains the delivery person's location information using a GPS sensor. It periodically collects location data (latitude and longitude) from the GPS sensor. The input is location information (latitude and longitude), and the output is the collected location data. Specifically, the GPS module inside the terminal reads the current latitude and longitude and temporarily stores the location information in memory.

[0586] Step 2:

[0587] The device sends the collected location information to the server at regular intervals (e.g., every 5 minutes). The data is sent using a RESTful API as the communication protocol. The input is the location data, and the output is the status of transmission to the server. Specifically, the device packages the location information in JSON format and sends the data to the server using an HTTP POST request.

[0588] Step 3:

[0589] The server stores the received location information in a database, using MySQL or MongoDB as the database. The input is the location data, and the output is the record status in the database. Specifically, the server parses the received location data, converts it into an appropriate format, and inserts it into the database using SQL queries or NoSQL operations.

[0590] Step 4:

[0591] The server analyzes the location information, movement line information, and stop times stored in the database. The analysis uses Python or R programming language to analyze movement line patterns and identify inefficient routes. The input is the stored location information, and the output is the analysis results (for example, movement line patterns). Specifically, the server runs an analysis script and extracts patterns of movement lines and stop times based on past location data.

[0592] Step 5:

[0593] The server uses a generative AI model based on the analysis results to calculate the next optimal action and route. TensorFlow and Scikit-learn are used as the generative AI model. The input is the analysis results, and the output is the proposed next action and route. Specifically, the server inputs the analysis results into the AI ​​model and predicts the optimal route and action.

[0594] Step 6:

[0595] The server notifies the device of the generated action proposal as a prompt. The prompt is created based on the output from the generative AI model and sent to the device in real time. The input is the action proposal, and the output is a notification to the device. Specifically, the server converts the proposal into a text prompt and notifies the delivery person's device using Firebase Cloud Messaging (FCM).

[0596] Step 7:

[0597] The user (delivery person) acts according to the suggestions notified to the terminal. For example, the user confirms the notification that "The next delivery destination is the nearest cafe. After that, deliver to the pizza place," and makes the delivery along the specified route. The input is the notified prompt sentence, and the output is the executed action. In concrete terms, the delivery person confirms the contents of the notification and moves and delivers according to the instructions.

[0598] This allows delivery personnel to deliver efficiently, shortening delivery times and improving work efficiency.

[0599] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0600] The present invention provides a system for collecting location information, movement information, and emotional information of a user to improve the efficiency of the user's life. This system recognizes the user's emotions and, taking those emotions into consideration, suggests the optimal next action, thereby reducing the user's psychological stress. Specific embodiments of the system are described below.

[0601] System Overview

[0602] 1. Data Collection

[0603] The wearable device uses GPS and acceleration sensors to collect real-time location and movement information for the user. For example, it records current location data and movement information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room."

[0604] The device records the user's downtime, for example, collecting data such as "stayed in the kitchen" from "08:00 AM" to "08:05 AM."

[0605] The device uses its built-in camera, microphone, and biometric sensors to collect biometric signals such as the user's facial expressions, voice, and heart rate, and transmits them to the emotion engine.

[0606] 2. Emotion recognition

[0607] The server analyzes facial expressions, voice, and biometric signals sent from the device to identify the user's emotional state, for example, recognizing whether the user is in a specific emotional state such as anger, sadness, or joy.

[0608] 3. Data Transmission

[0609] The terminal transmits the collected location information, movement line information, and stop time data to the server at regular intervals (for example, every 5 minutes).

[0610] 4. Data Analysis

[0611] The server receives the transmitted data, stores it in a database, and converts the received data into a format for analysis.

[0612] The server analyzes location information, movement information, and downtime patterns, for example, identifying a pattern that "the user is in the kitchen every morning at 8:00 AM."

[0613] 5. Emotion-based action suggestions

[0614] The server then takes into account the collected emotional information and runs an algorithm to identify unnecessary movements and efficient routes for the user, making specific suggestions such as "If the user is tired, they should take some time to relax."

[0615] The server uses an optimization algorithm to select the next action, taking emotional information into account. For example, it might suggest an action such as "take a deep breath while waiting for the microwave to turn on."

[0616] 6. Notification of Proposed Actions

[0617] The server generates data to notify the user of the selected action. For example, in addition to the message "Take out the trash while the microwave is in standby mode," it generates a message such as "Take a deep breath and relax."

[0618] The server transmits this notification data to the terminal.

[0619] 7. Notice to Users

[0620] The device displays the received notification data to the user in real time, for example, notifying the user of multiple messages such as "Take out the trash while the microwave is in standby mode" and "Take a deep breath and relax."

[0621] 8. User execution

[0622] Users can then take specific actions based on the notification, such as taking out the trash or taking a deep breath while waiting for the microwave to heat up. This allows users to live more efficiently without wasting time and reduces psychological stress.

[0623] Specific examples

[0624] Situation

[0625] When a user is preparing breakfast, they have to wait a few minutes while the food heats up in the microwave, which can be stressful for the user.

[0626] Data collection and transmission

[0627] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[0628] The device recognizes signs of stress from the user's facial expressions and heart rate.

[0629] Data analysis and sentiment-based recommendations

[0630] The server identifies the waiting time while the user is using the microwave in the kitchen and recognizes when the user is stressed based on the emotion engine.

[0631] The server selects "take out the trash while the microwave is waiting" and "take a deep breath and relax" as optimal actions and generates notification data.

[0632] User Notification

[0633] Notification data is sent from the server to the device, and the device displays to the user "Take out the trash while the microwave is waiting" and "Take a deep breath and relax."

[0634] User execution

[0635] The user follows the notification, takes out the trash while the microwave is waiting, and takes a deep breath, allowing the user to use their time efficiently and reduce stress.

[0636] By combining this system with an emotion engine, users can enjoy the benefits of effectively utilizing their time while reducing their psychological stress.

[0637] The processing flow will be explained below.

[0638] Step 1:

[0639] The device uses GPS and acceleration sensors to collect real-time location and movement information of the user. For example, if the user is in the kitchen at 8:00 AM, the device will record the location information as "8:00 AM - Kitchen."

[0640] Step 2:

[0641] The device records the user's downtime. If the user stays in a specific location for a certain amount of time, the device saves the downtime as data. For example, data such as "stayed in the kitchen from 8:00 AM to 8:05 AM" can be collected.

[0642] Step 3:

[0643] The device uses its built-in camera, microphone, and biometric sensors to collect the user's facial expressions, voice, heart rate, and other biometric signals in real time, and transmits this data to the emotion engine.

[0644] Step 4:

[0645] The server's emotion engine analyzes the transmitted facial expressions, voice, and biometric signals to determine the user's emotional state, for example, determining that the user is feeling stressed.

[0646] Step 5:

[0647] The terminal transmits the collected location information, movement information, and stop time data to the server at regular intervals (for example, every 5 minutes).

[0648] Step 6:

[0649] The server receives the data and stores it in a database. It converts the received data into an analytical format and analyzes the location and movement patterns. For example, it can identify a pattern that "the user is in the kitchen every morning at 8:00 AM."

[0650] Step 7:

[0651] The server then runs an algorithm that uses the analysis results to identify unnecessary movements and efficient routes, taking into account the user's emotional information and identifying specific data, such as "the microwave has a two-minute waiting time."

[0652] Step 8:

[0653] The server uses an optimization algorithm to select the best next action for the user based on the emotional information. For example, it could suggest actions such as "Take out the trash while the microwave is running" and "Take a deep breath and relax."

[0654] Step 9:

[0655] The server generates data to notify the user of the selected action, summarizing specific suggestions as messages such as "Take out the trash while the microwave is on standby" and "Take a deep breath and relax."

[0656] Step 10:

[0657] The server transmits the generated notification data to the terminal.

[0658] Step 11:

[0659] The device displays the received notification data to the user in real time, for example, notifying the user of multiple messages such as "Take out the trash while the microwave is in standby mode" and "Take a deep breath and relax."

[0660] Step 12:

[0661] Users can then take specific actions based on the notification, such as taking out the trash while the microwave is running or taking deep breaths to relax. This allows users to reduce wasted time, live more efficiently, and simultaneously reduce psychological stress.

[0662] Example 2

[0663] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0664] In modern life, users are expected to reduce unnecessary movements and use their time efficiently. However, conventional systems suggest next actions based on simple location and movement information without considering the user's emotions or psychological state, making it difficult to reduce the user's psychological stress.

[0665] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting location information, movement line information, and emotion information of the user, means for analyzing the collected location information, movement line information, and emotion information, means for proposing an optimal next action for the user based on the analysis results, means for notifying the user of the proposed action, and means for evaluating the user's behavior in accordance with the proposed action. This makes it possible to use time efficiently while reducing the user's psychological stress.

[0666] "User location information" means data that indicates the user's current geographic location.

[0667] "Traffic information" is data that indicates the route or trajectory of a user's movement when moving from one place to another.

[0668] "Emotional information" is data that indicates the user's psychological state and emotional fluctuations, and is extracted from facial expressions, voice, biometric signals, etc.

[0669] "Downtime" is data that records the length of time a user stays in a particular location.

[0670] "Analytics" refers to the algorithms and software used to process and analyze collected data to derive insights.

[0671] "Means of suggestion" refers to algorithms or software that specifically suggest optimal actions for users based on the analysis results.

[0672] "Notification means" refers to a device or communication means that notifies the user of the proposed action, such as a smartphone or smartwatch.

[0673] "Means for evaluating behavior" refers to algorithms or software that evaluate the results of a suggested action after the user performs it and use that information to make future suggestions.

[0674] The present invention is a system for collecting location information, movement information, and emotional information of a user to improve the efficiency of the user's life. This system recognizes the user's emotions and takes those emotions into consideration to suggest the optimal next action, thereby reducing the user's psychological stress. Specific embodiments of the system are described below.

[0675] Data collection

[0676] The device (wearable device) uses a GPS sensor and an acceleration sensor to collect real-time location and movement information of the user. For example, it records current location data and movement information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room." The device also records the user's inactivity time. Furthermore, it uses a built-in camera, microphone, and biometric sensors to collect biometric signals such as the user's facial expressions, voice, and heart rate.

[0677] Data transmission

[0678] The device sends the collected location information, movement information, and biometric signals to a server at regular intervals (for example, every 5 minutes) using Wi-Fi or mobile data communication.

[0679] emotion recognition

[0680] The server receives facial expression data, voice data, and biometric signals sent from the device and uses an emotion recognition algorithm to identify the user's emotional state. For example, if the user's facial expression is grim, it will be recognized as "anger."

[0681] Data analysis

[0682] The server stores the received data in a database, converts the stored data into an analytical format, and analyzes the location and movement patterns. This process includes a data cleansing process to remove noise.

[0683] Suggesting actions based on emotions

[0684] The server runs an algorithm to suggest the best next action that takes emotional information into account, for example, "If the user is tired, take some time to relax." It also selects the next action based on the optimization algorithm, choosing an action such as "take some deep breaths while waiting for the microwave to heat up."

[0685] Proposed action notifications

[0686] The server generates message data to notify the selected action and sends it to the device. For example, it creates messages such as "Take out the trash while the microwave is in standby mode" and "Take a deep breath and relax."

[0687] User Notification

[0688] The device analyzes the notification data received from the server and displays it on the display screen in real time. For example, messages such as "Take out the trash while the microwave is on standby" and "Take a deep breath and relax" may be displayed on the screen of a smartwatch or smartphone.

[0689] User execution

[0690] Users can take specific actions based on notifications from their device, such as taking out the trash while the microwave is running and taking deep breaths, allowing them to use their time efficiently and reduce psychological stress.

[0691] Specific examples

[0692] Situation

[0693] When a user is preparing breakfast, they have to wait a few minutes while the food heats up in the microwave, which can be stressful for the user.

[0694] Data collection and transmission

[0695] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[0696] The device recognizes signs of stress from the user's facial expressions and heart rate.

[0697] Data analysis and sentiment-based recommendations

[0698] The server identifies the waiting time while the user is using the microwave in the kitchen and recognizes when the user is stressed based on the emotion engine.

[0699] The server selects "take out the trash while the microwave is waiting" and "take a deep breath and relax" as optimal actions and generates notification data.

[0700] User Notification

[0701] Notification data is sent from the server to the device, and the device displays to the user "Take out the trash while the microwave is waiting" and "Take a deep breath and relax."

[0702] User execution

[0703] The user follows the notification, takes out the trash while the microwave is waiting, and takes a deep breath, allowing the user to use their time efficiently and reduce stress.

[0704] By combining this system with an emotion engine, users can enjoy the benefits of effectively utilizing their time while reducing their psychological stress.

[0705] Prompt Sentence Examples

[0706] "Please explain a system that collects location, movement, and emotional information from users and suggests optimal actions."

[0707] "Please tell me more about how you use the Emotion Engine to reduce user stress."

[0708] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0709] Step 1: Data collection

[0710] The device uses a GPS sensor and an acceleration sensor to obtain real-time location and movement information for the user. As input, the sensor receives the user's current location and route traveled, collecting information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room." As output, this data is stored in the device's internal memory. Additionally, the built-in camera captures the user's facial expressions, the microphone collects the user's voice in real time, and the biometric sensor captures vital signs such as heart rate and body temperature.

[0711] Step 2: Send data

[0712] The device transmits the collected location information, movement information, and biometric signals to the server at regular intervals (e.g., every 5 minutes). As input, data stored in the internal memory is retrieved and transmitted via Wi-Fi or mobile data communication. As output, data packets are generated from the device to the server and transferred to the server.

[0713] Step 3: Emotion Recognition

[0714] The server receives facial expression data, voice data, and biometric signals sent from the device. It receives these various data as input and runs an emotion recognition algorithm to identify the user's emotional state. For example, if the user's facial expression is grim, it will be recognized as "anger." The output is the identified emotional state, which is then stored in a database.

[0715] Step 4: Data analysis

[0716] The server stores the received data in a database. As input, it takes the received data and stores it in the database in SQL or NoSQL format. It then converts the stored data into a format for analysis. This includes a data cleansing process, for example, removing noise from location information. As output, clean data is generated and passed to the next analysis process.

[0717] Step 5: Consider emotions and suggest actions

[0718] The server runs an algorithm that takes emotional information into account to optimize the user's movement path. Location information, movement path information, pause time, and emotional information are taken as inputs and fed into the analysis algorithm. For example, a suggestion such as "If the user is tired, have them take some time to relax" is made. Optimized action suggestions are generated as output.

[0719] Step 6: Notification of proposed action

[0720] The server generates message data to notify the selected action. As input, it receives the action proposal generated in step 5 and creates a specific message. For example, messages such as "Take out the trash while the microwave is waiting" and "Take a deep breath and relax" are generated. As output, this notification data is sent to the device.

[0721] Step 7: Notify users

[0722] The device analyzes the notification data received from the server and displays it on the display screen in real time. As input, it takes the message received from the server and displays it on the user interface (UI). For example, messages such as "Take out the trash while the microwave is waiting" and "Take a deep breath and relax" are displayed on the screen of a smartwatch or smartphone. As output, specific instructions for action are presented to the user.

[0723] Step 8: Run the user

[0724] The user takes specific actions according to the notifications from the device. As input, the user receives the notification message from the device and starts the action according to the instructions. For example, the user takes out the trash while waiting for the microwave to run and takes a deep breath. As output, the user's action is completed, and efficient time utilization and reduction of psychological stress are achieved.

[0725] (Application example 2)

[0726] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0727] Conventional systems only collect and analyze user location and movement information, and are limited in their ability to suggest specific next actions that take into account the user's emotions and psychological state. This makes it difficult to encourage efficient behavior while reducing the user's psychological stress in scenarios that involve waiting, such as food delivery. The objective of the present invention is to solve this problem and provide a system that can suggest optimal actions that take into account the user's emotions and waiting time.

[0728] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user location information and movement line information, means for analyzing the collected location information, movement line information, and emotional information, means for proposing the optimal next action for the user based on the analysis results, and means for notifying the user in real time of the optimal action to take while waiting for food delivery. This makes it possible to efficiently utilize the waiting time while reducing the user's psychological stress.

[0729] "User Location Information" means the geographic coordinate information of the User's current location obtained by a GPS device or other location measurement means.

[0730] "Traffic information" is data that indicates the route and direction a user travels within a specific period of time.

[0731] "Emotional information" is information about the user's emotional state obtained by analyzing the user's facial expressions, voice, biometric signals, etc.

[0732] "Means for analyzing" refers to hardware or software functionality for processing and analyzing collected data to extract useful information.

[0733] "Means for suggesting next actions" refers to the function of an algorithm or system that suggests optimal actions or activities for users based on the analysis results.

[0734] "Waiting for food delivery" refers to the time while a user is waiting for food delivery.

[0735] "Means of notifying users of optimal actions in real time" refers to digital communication methods that take into account the user's current situation and emotional state and instantly inform the user of suggested actions.

[0736] This invention is a system that collects and analyzes a user's location information, movement information, and emotional information to suggest optimal actions for the user, thereby promoting efficient behavior and reducing psychological stress, particularly while waiting for food delivery.

[0737] System Overview

[0738] 1. Data Collection

[0739] The server uses GPS devices and other location measurement means to collect user location and movement information.

[0740] The server uses software for facial recognition, voice analysis, and biometric signal analysis to obtain the user's emotional information.

[0741] 2. Data Analysis

[0742] The server analyzes the collected location information, movement information, and emotional information and stores it in a database using an analysis platform and AI model.

[0743] 3. Action suggestions

[0744] The server then suggests the best next action for the user based on the analysis results, which includes an algorithm to reduce the user's psychological stress.

[0745] 4. Real-time notifications

[0746] The server communicates with the user's smartphone or appropriate end device using a notification API to notify the user of the optimal action in real time.

[0747] Specific examples

[0748] For example, if a user orders food delivery and is waiting for it to arrive, the system will follow these steps:

[0749] 1. Data Collection

[0750] The device collects the user's location information (e.g., "08:00 AM - Home"), movement information (e.g., "08:10 AM - Kitchen"), and emotional information (e.g., "stress" state based on facial expression analysis).

[0751] 2. Data Analysis

[0752] The server receives this information and uses an emotion engine to determine the user's psychological state and waiting status.

[0753] 3. Action suggestions

[0754] The server generates the best suggestion: "Your food delivery will arrive in 20 minutes. Take a deep breath."

[0755] 4. Notification

[0756] The server sends a notification to the device, and the user receives the suggestion via their smartphone.

[0757] This system allows users to use their time efficiently while reducing psychological stress while waiting for food delivery.

[0758] Prompt Sentence Examples

[0759] "Given the following setup, create an app that suggests optimal actions while waiting for a food delivery, taking into account the user's emotions and location. The user's emotional state can be 'stress', 'neutral', or 'happy'. The location can be 'home' or 'office'. Start 30 minutes before the scheduled delivery time."

[0760] This prompt can be used to leverage a generative AI model to flexibly generate more detailed suggestions and notification content.

[0761] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0762] Step 1:

[0763] Data collection

[0764] Specific behavior:

[0765] The device collects real-time location, movement, and emotional information from the user. Location information is obtained using GPS devices and other location measurement methods, movement information is obtained by tracking the user's movement path, and emotional information is obtained using facial recognition, voice analysis, and biometric signal analysis.

[0766] input:

[0767] User location information (e.g., "08:00 AM - home"), movement path information (e.g., "08:10 AM - kitchen"), and emotional information (e.g., "stressed" state based on facial expression analysis).

[0768] output:

[0769] A dataset of collected location, movement, and emotion information.

[0770] Step 2:

[0771] Data transmission

[0772] Specific behavior:

[0773] The device transmits the collected location information, movement information, and emotion information to a server at regular intervals (e.g., every 5 minutes) using a wireless communication module.

[0774] input:

[0775] A dataset of collected location, movement, and emotion information.

[0776] output:

[0777] The dataset sent to the server.

[0778] Step 3:

[0779] Data reception and storage

[0780] Specific behavior:

[0781] The server receives the data sent from the terminal and stores it in a database. The received data is converted into an analysis format.

[0782] input:

[0783] A dataset of location information, movement information, and emotional information sent from the device.

[0784] output:

[0785] Datasets for analysis stored in a database.

[0786] Step 4:

[0787] Data analysis

[0788] Specific behavior:

[0789] The server analyzes the user's behavioral patterns and emotional state based on location, movement, and emotional information stored in the database, and uses a generative AI model to evaluate the user's psychological state and suggest actions.

[0790] input:

[0791] A dataset for analyzing location information, movement information, and emotion information stored in a database.

[0792] output:

[0793] Analysis results (e.g., user is stressed, in the kitchen, waiting for food delivery).

[0794] Step 5:

[0795] Action suggestion generation

[0796] Specific behavior:

[0797] Based on the results of the data analysis, the server generates optimal actions to suggest to the user, such as "take deep breaths" or "listen to music to relax" to reduce stress.

[0798] input:

[0799] Analysis results (e.g., user is stressed, in the kitchen, waiting for food delivery).

[0800] output:

[0801] Specific suggestions for action (e.g., "Your food delivery will arrive in 20 minutes. Take a deep breath.").

[0802] Step 6:

[0803] Proposal Notification

[0804] Specific behavior:

[0805] The server then creates notification data based on the generated action suggestions and sends it to the device, which then displays the notification data to the user in real time.

[0806] input:

[0807] Action suggestions (e.g., "Your food delivery will arrive in 20 minutes. Take a deep breath.").

[0808] output:

[0809] Suggestions notified to users.

[0810] Step 7:

[0811] User execution

[0812] Specific behavior:

[0813] The user receives a notification from the device and performs a suggested action, such as "take a deep breath" or "stretch."

[0814] input:

[0815] Notified suggestions (e.g., "Your food delivery will arrive in 20 minutes. Take a deep breath.").

[0816] output:

[0817] The action taken (e.g., the user took a deep breath).

[0818] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0819] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0820] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0821] [Third embodiment]

[0822] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0823] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0824] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0825] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0826] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0827] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0828] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0829] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0830] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0831] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0832] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0833] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0834] This invention provides a system that functions as an assistant to help users lead their daily lives efficiently. It collects and analyzes the user's location information, movement path information, and downtime, and suggests the optimal next action. This system consists of three main components: the terminal (wearable device), the server, and the user.

[0835] System Overview

[0836] 1. Data Collection

[0837] The wearable device uses GPS and acceleration sensors to collect real-time location and movement information for the user. For example, it records current location data and movement information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room."

[0838] The device also records the user's downtime, for example, "staying in the kitchen" from 8:00 AM to 8:05 AM.

[0839] 2. Data Transmission

[0840] The device sends the collected location information, movement information, and stop time data to the server at regular intervals (e.g., every 5 minutes), where the data is accumulated and used for analysis.

[0841] 3. Data analysis

[0842] The server receives the data and first stores it in a database. It then converts it into an analytical format and analyzes the user's movement patterns and downtime. For example, it can identify a pattern such as "every morning, the user goes to the kitchen at 8:00 AM."

[0843] The server uses the collected data to run algorithms that identify wasted time and efficient routes for users, generating specific action suggestions such as "Take out the trash during the two minutes you wait for the microwave."

[0844] 4. Action proposals

[0845] The server then selects the optimal next action based on the analysis results, such as suggesting other housework that would be best suited to the time the user is waiting in the kitchen.

[0846] The server generates notification data that suggests the selected action, and generates a message such as "Take out the garbage while the microwave is in standby mode."

[0847] 5. Notice to Users

[0848] The server sends the generated notification data to the device, which then displays the received notification data to the user in real time. For example, it displays a message such as "Take out the trash while the microwave is in standby mode."

[0849] 6. User execution

[0850] The user follows the notification to take out the trash. This suggestion helps users reduce wasted time and do their housework efficiently.

[0851] Specific examples

[0852] Situation

[0853] A user is preparing breakfast and has to wait a few minutes while the ingredients heat up in the microwave.

[0854] Data collection and transmission

[0855] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[0856] Data analysis and recommendations

[0857] The server identifies the amount of time a user waits while using a microwave in the kitchen, and also determines from past logs that a nearby trash can is full.

[0858] The server selects "take out the garbage while the microwave is in standby mode" as the optimal action and generates notification data.

[0859] User Notification

[0860] Notification data is sent from the server to the device, and the device displays a message to the user saying, "Take out the trash while the microwave is in standby mode."

[0861] User execution

[0862] The user follows the notification and takes out the trash while the microwave is waiting.

[0863] This system allows users to effectively utilize their time and improve daily efficiency.The effects of the present invention are realized by such specific embodiments.

[0864] The processing flow will be explained below.

[0865] Step 1:

[0866] The device collects the user's current location and movement information in real time using GPS and acceleration sensors. For example, if the user is in the kitchen, the device will record the location information as "08:00 AM - Kitchen."

[0867] Step 2:

[0868] The device records the user's downtime. If the user stays in a specific location for a certain amount of time, the device saves the downtime as data. For example, it can collect data such as "stayed in the kitchen from 8:00 AM to 8:05 AM."

[0869] Step 3:

[0870] The device sends the collected location information and stop time data to the server at regular intervals (for example, every 5 minutes).

[0871] Step 4:

[0872] The server receives the data sent from the terminal, stores it in a database, and converts the received data into a format for analysis.

[0873] Step 5:

[0874] The server analyzes location information, movement information, and downtime patterns, for example, identifying a pattern that "the user is in the kitchen at 8:00 AM every morning."

[0875] Step 6:

[0876] Based on the analysis results, the server runs an algorithm to identify unnecessary movements and efficient routes for users, such as identifying specific data such as "there is a two-minute wait time for the microwave."

[0877] Step 7:

[0878] The server uses an optimization algorithm to select the best next action for the user, for example, suggesting an action such as "taking out the trash while the microwave is waiting."

[0879] Step 8:

[0880] The server generates data to notify the user of the selected action, summarizing the suggestion as a message such as "Take out the trash while the microwave is in standby mode."

[0881] Step 9:

[0882] The server transmits this notification data to the terminal.

[0883] Step 10:

[0884] The device displays the received notification data to the user in real time, for example, by sending a message to the user saying, "Take out the trash while the microwave is in standby mode."

[0885] Step 11:

[0886] Users can then take specific actions based on the notification, such as taking out the trash while the microwave is waiting. This allows users to live their lives efficiently without wasting time.

[0887] Example 1

[0888] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0889] While conventional assistant systems collect information on the user's location and movement, there are no systems that can effectively analyze this information and suggest beneficial next actions for the user in real time. As a result, users often waste time, which reduces the efficiency of their daily lives.

[0890] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0891] In this invention, the server includes means for collecting user location information and movement line information, means for transmitting the collected location information and movement line information to a central processing unit at regular intervals, means for the central processing unit to store the collected location information and movement line information, convert the collected information into an analytical format and analyze it, means for selecting the optimal next action for the user based on the analysis results and generating notification data, and means for transmitting the generated notification data to the user's information terminal and displaying it to the user in real time, thereby enabling users to use their time more efficiently and significantly improve the efficiency of their daily lives.

[0892] "User" refers to an individual or group of people who use the System.

[0893] "Location Information" is data that indicates a user's current geographic location.

[0894] "Flow line information" is data that indicates the trajectory or route that a user has taken.

[0895] "Dwell time" is data that indicates the amount of time a user remains in a particular location.

[0896] A "terminal" is a device carried or worn by a user, examples of which include wearable devices.

[0897] A "server" is a central computer that receives collected data, analyzes it, and sends the results to the user's device.

[0898] "Central processing unit" is a general term for the hardware and software installed on a server that stores, converts, and analyzes data.

[0899] "Notification data" refers to data that includes suggestions and instructions for the user that are generated based on the analysis results.

[0900] An "information terminal" is a device that receives notification data sent from a server and displays it to the user.

[0901] "Interval" refers to the time interval during which data is collected or transmitted.

[0902] This invention is an assistant system that helps users lead their daily lives efficiently. It collects and analyzes the user's location information, movement information, and downtime, and suggests the optimal next action. This system consists of three main components: the terminal (wearable device), the server, and the user.

[0903] Data collection

[0904] The device (wearable device) uses GPS and acceleration sensors to collect the user's location and movement information in real time. Specifically, if the user is in the kitchen at 08:00 AM, the data will be recorded as "08:00 AM - Kitchen." If the user moves to the living room at 08:05 AM, the data will be recorded as "08:05 AM - Living Room." The device also records the time the user is stationary in a particular location. For example, if the user is in the kitchen from 08:00 AM to 08:05 AM, the data will be collected as "Stayed in the kitchen from 08:00 AM to 08:05 AM."

[0905] Data transmission

[0906] The device sends the collected location information, movement information, and stop time data to a server at regular intervals (e.g., every 5 minutes). The data is sent using wireless communication technology (e.g., Wi-Fi or Bluetooth). This allows the data to be accumulated on the server and used for later analysis.

[0907] Data reception and storage

[0908] The server receives the data sent from the device. The received data is first stored in a database. For example, data such as "08:00 AM - Kitchen" and "08:05 AM - Living Room" is stored on the server.

[0909] Data conversion and analysis

[0910] The server converts the stored data into a format for analysis. This format conversion involves shaping the data and removing unnecessary information. The server then analyzes movement patterns and downtime. For example, it might analyze a pattern such as "every morning, the user goes to the kitchen at 8:00 AM."

[0911] Action proposal generation

[0912] The server selects the optimal next action based on the analysis results. For example, while the user is waiting for food to heat up in the microwave, it will suggest an action to use that waiting time efficiently. Specifically, it generates a suggestion such as "Take out the trash while the microwave is on." This suggestion is selected based on the user's past behavioral patterns and current situation (for example, whether the trash can is full).

[0913] User Notification

[0914] The server generates the selected action suggestion as notification data and sends it to the device. For example, it generates a message saying, "Take out the trash while the microwave is waiting." The device then receives the notification and displays it to the user in real time.

[0915] User execution

[0916] Users can check notifications from their devices and take suggested actions, such as taking out the trash while the microwave is running. This action allows users to use their time efficiently and get on with their daily tasks.

[0917] Specific examples

[0918] Situation

[0919] When a user is preparing breakfast, there is a few minutes of waiting while the ingredients heat up in the microwave.

[0920] Specific examples of data collection and transmission

[0921] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[0922] The device sends this data to the server every five minutes.

[0923] Examples of data reception and storage

[0924] The server receives data such as "07:30 - 07:33 Kitchen" and stores it in a database.

[0925] Specific examples of data conversion and analysis

[0926] The server converts the stored data into an analytical format and determines that "the user was using the microwave in the kitchen between 07:30 and 07:33."

[0927] Example of action proposal generation

[0928] The server determines that the user is waiting in the kitchen between 07:30 and 07:33, and uses this waiting time to suggest taking out the trash.

[0929] Examples of user notifications

[0930] The server generates notification data saying "Take out the garbage while the microwave is in standby mode" and transmits it to the terminal.

[0931] User execution example

[0932] Users can follow the notification and take out the trash while the microwave is waiting, allowing them to use their time effectively and improve their daily efficiency.

[0933] Example prompts to input to the generative AI model

[0934] User: "Can you give me some suggestions on how to make the most of my time waiting in the kitchen each morning?"

[0935] Prompt for generative AI model: "Suggest efficient actions that the user can take while waiting in the kitchen each morning, such as taking out the trash or preparing ingredients."

[0936] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0937] Step 1: Data collection

[0938] The device collects the user's location information and movement path information. Specifically, it uses GPS and an acceleration sensor to record the user's current location and movement trajectory. For example, if the user is in the kitchen at 8:00 AM, the data "8:00 AM - Kitchen" is collected. In addition, the device also records the user's downtime. For example, the data collected might be "Stayed in the kitchen from 8:00 AM to 8:05 AM." This data will be used as input for analysis in later steps.

[0939] Step 2: Send data

[0940] The device sends the collected location information, movement information, and stop time data to the server at regular intervals (e.g., every 5 minutes). Specifically, data such as "08:00 AM - kitchen" and "08:05 AM - living room" is sent to the server every 5 minutes. This transmission uses wireless communication technology (e.g., Wi-Fi, Bluetooth). This data is accumulated on the server and becomes input data for later analysis.

[0941] Step 3: Receiving and storing data

[0942] The server receives the data sent from the device. The received data is first stored in a database. For example, data such as "08:00 AM - Kitchen" and "08:05 AM - Living Room" is accumulated in the database. The output of this step is the location information and movement line information stored in the database.

[0943] Step 4: Data conversion and analysis

[0944] The server converts the stored data into a format for analysis. Specifically, it formats the data and deletes unnecessary information. Next, the server analyzes movement patterns and downtime. For example, it analyzes patterns such as "every morning, the user goes to the kitchen at 8:00 AM" and "there is a two-minute wait time for the microwave." The input data is the location information and movement information stored in the database, and the output data is the analysis results.

[0945] Step 5: Action proposal generation

[0946] The server selects the optimal next action based on the analysis results. Specifically, it suggests actions to use the user's waiting time efficiently while they wait for food to heat up in the microwave. For example, it generates a suggestion such as "Take out the trash while the microwave is on." The input to this step is the analysis results, and the output is the selected action suggestion (notification data).

[0947] Step 6: Notify users

[0948] The server sends the generated notification data to the user's device. For example, a notification message saying "Take out the trash while the microwave is in standby mode" is sent to the device. The device displays the received notification data in real time. The input data is the notification data, and the output is the message to be displayed to the user.

[0949] Step 7: Run the user

[0950] The user checks the notification from the device and performs the suggested action. For example, the user takes out the trash while the microwave is waiting. The input of this step is the notification message from the device, and the output is the user's action. This allows the user to use their time efficiently and improve the efficiency of their daily life.

[0951] (Application example 1)

[0952] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0953] In modern food delivery services, delivery workers must efficiently travel to multiple destinations. However, current systems do not provide suggestions for optimal routes or appropriate break times, resulting in inefficiency. They also have difficulty responding to real-time changes in the situation. This can lead to delivery delays and fatigue among delivery workers, resulting in a decline in overall service quality. Therefore, a system is needed that allows delivery workers to work efficiently and receive suggestions for optimal routes and break times in real time.

[0954] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0955] In this invention, the server includes means for collecting user location information, flow line information, and stop time, means for analyzing the collected location information, flow line information, and stop time, means for proposing an efficient delivery route based on the analysis results, and means for notifying the user of optimal actions and rest times in real time. This allows delivery personnel to receive suggestions for optimal routes and rest times in real time, thereby enabling delivery work to be performed efficiently, preventing delivery delays, and reducing fatigue of delivery personnel.

[0956] "Location Information" means data that describes the current location of a user or device, obtained using GPS or other location measurement technologies.

[0957] "Traffic information" is data showing the route taken by a user or device, and is generated from continuous recording of location information.

[0958] "Downtime" is data that shows the amount of time a user or device spends in a particular location.

[0959] "Analysis tools" are algorithms and software used to extract meaningful information and patterns from collected data.

[0960] "Means for suggesting the optimal next action" refers to systems or software that suggest the most efficient next action to the user based on the analysis results.

[0961] A "delivery route" refers to the route of locations that should be visited in the most efficient order when making a delivery.

[0962] "Real-time" means that processing and response occur immediately at the moment the data is generated.

[0963] "Means of notification" refers to devices or software that notify users of information or actions, including the ability to send notifications to smartphones or wearable devices.

[0964] A "prompt" is a textual suggestion or instruction that a generative AI model provides to a user.

[0965] A "generative AI model" is a machine learning algorithm or artificial intelligence technology that generates new information or suggestions based on data.

[0966] This invention relates to an assistance system that enables delivery personnel to deliver food efficiently. Specifically, this system collects and analyzes delivery personnel's location information, movement line information, and stop time, and proposes optimal delivery routes and break times in real time.

[0967] The system mainly consists of three main components: terminals, servers, and users.

[0968] Terminal

[0969] The terminal is a wearable device such as a smartphone or smart glasses. This terminal uses GPS and an acceleration sensor to collect real-time location information and movement information of the delivery person. It also records the time the delivery person stays in a specific location (stop time). For example, the terminal records location information such as "10:00 AM - Cafe" and "10:15 AM - Pizza place."

[0970] server

[0971] The server receives and stores the location information, movement information, and stop time data sent from the device. The server analyzes this data and generates the optimal next action for the delivery person. The analysis uses Python and R languages ​​to analyze movement patterns and propose efficient routes. It also uses generative AI models (e.g., TensorFlow, Scikit-learn) to prompt the user on the optimal course of action.

[0972] For example, the server generates a prompt such as, "The delivery person's current location is latitude 35.6895, longitude 139.6917. Please suggest the next best delivery destination. Please take traffic conditions and stop time into consideration." and notifies the delivery person of the analysis results.

[0973] user

[0974] Delivery workers receive suggestions from their devices. For example, they can receive a notification such as, "Your next delivery destination is the nearest cafe. After that, deliver to a pizza place." New suggestions are sent in real time as needed during delivery, allowing for efficient delivery route management.

[0975] As a concrete example, the terminal works as follows:

[0976] 1. The device periodically collects the delivery person's location information.

[0977] 2. The collected data is sent to the server and stored in a database (e.g., MySQL, MongoDB).

[0978] 3. The server analyzes the data using Python or R, and uses TensorFlow or Scikit-learn to predict the optimal next action using an AI model.

[0979] 4. A prompt is generated based on the analysis results and sent to the delivery person's device.

[0980] 5. Delivery personnel act on notifications and complete deliveries efficiently.

[0981] For example, a notification will appear on the delivery person's device stating, "The next best delivery from your current location is point A, based on distance, followed by a delivery at point B, taking current traffic conditions into account."

[0982] In this way, the present invention allows delivery personnel to work efficiently in real time, preventing delivery delays and contributing to an overall improvement in service quality.

[0983] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0984] Step 1:

[0985] The terminal obtains the delivery person's location information using a GPS sensor. It periodically collects location data (latitude and longitude) from the GPS sensor. The input is location information (latitude and longitude), and the output is the collected location data. Specifically, the GPS module inside the terminal reads the current latitude and longitude and temporarily stores the location information in memory.

[0986] Step 2:

[0987] The device sends the collected location information to the server at regular intervals (e.g., every 5 minutes). The data is sent using a RESTful API as the communication protocol. The input is the location data, and the output is the status of transmission to the server. Specifically, the device packages the location information in JSON format and sends the data to the server using an HTTP POST request.

[0988] Step 3:

[0989] The server stores the received location information in a database, using MySQL or MongoDB as the database. The input is the location data, and the output is the record status in the database. Specifically, the server parses the received location data, converts it into an appropriate format, and inserts it into the database using SQL queries or NoSQL operations.

[0990] Step 4:

[0991] The server analyzes the location information, movement line information, and stop times stored in the database. The analysis uses Python or R programming language to analyze movement line patterns and identify inefficient routes. The input is the stored location information, and the output is the analysis results (for example, movement line patterns). Specifically, the server runs an analysis script and extracts patterns of movement lines and stop times based on past location data.

[0992] Step 5:

[0993] The server uses a generative AI model based on the analysis results to calculate the next optimal action and route. TensorFlow and Scikit-learn are used as the generative AI model. The input is the analysis results, and the output is the proposed next action and route. Specifically, the server inputs the analysis results into the AI ​​model and predicts the optimal route and action.

[0994] Step 6:

[0995] The server notifies the device of the generated action proposal as a prompt. The prompt is created based on the output from the generative AI model and sent to the device in real time. The input is the action proposal, and the output is a notification to the device. Specifically, the server converts the proposal into a text prompt and notifies the delivery person's device using Firebase Cloud Messaging (FCM).

[0996] Step 7:

[0997] The user (delivery person) acts according to the suggestions notified to the terminal. For example, the user confirms the notification that "The next delivery destination is the nearest cafe. After that, deliver to the pizza place," and makes the delivery along the specified route. The input is the notified prompt sentence, and the output is the executed action. In concrete terms, the delivery person confirms the contents of the notification and moves and delivers according to the instructions.

[0998] This allows delivery personnel to deliver efficiently, shortening delivery times and improving work efficiency.

[0999] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1000] The present invention provides a system for collecting location information, movement information, and emotional information of a user to improve the efficiency of the user's life. This system recognizes the user's emotions and, taking those emotions into consideration, suggests the optimal next action, thereby reducing the user's psychological stress. Specific embodiments of the system are described below.

[1001] System Overview

[1002] 1. Data Collection

[1003] The wearable device uses GPS and acceleration sensors to collect real-time location and movement information for the user. For example, it records current location data and movement information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room."

[1004] The device records the user's downtime, for example, collecting data such as "stayed in the kitchen" from "08:00 AM" to "08:05 AM."

[1005] The device uses its built-in camera, microphone, and biometric sensors to collect biometric signals such as the user's facial expressions, voice, and heart rate, and transmits them to the emotion engine.

[1006] 2. Emotion recognition

[1007] The server analyzes facial expressions, voice, and biometric signals sent from the device to identify the user's emotional state, for example, recognizing whether the user is in a specific emotional state such as anger, sadness, or joy.

[1008] 3. Data Transmission

[1009] The terminal transmits the collected location information, movement line information, and stop time data to the server at regular intervals (for example, every 5 minutes).

[1010] 4. Data Analysis

[1011] The server receives the transmitted data, stores it in a database, and converts the received data into a format for analysis.

[1012] The server analyzes location information, movement information, and downtime patterns, for example, identifying a pattern that "the user is in the kitchen every morning at 8:00 AM."

[1013] 5. Emotion-based action suggestions

[1014] The server then takes into account the collected emotional information and runs an algorithm to identify unnecessary movements and efficient routes for the user, making specific suggestions such as "If the user is tired, they should take some time to relax."

[1015] The server uses an optimization algorithm to select the next action, taking emotional information into account. For example, it might suggest an action such as "take a deep breath while waiting for the microwave to turn on."

[1016] 6. Notification of Proposed Actions

[1017] The server generates data to notify the user of the selected action. For example, in addition to the message "Take out the trash while the microwave is in standby mode," it generates a message such as "Take a deep breath and relax."

[1018] The server transmits this notification data to the terminal.

[1019] 7. Notice to Users

[1020] The device displays the received notification data to the user in real time, for example, notifying the user of multiple messages such as "Take out the trash while the microwave is in standby mode" and "Take a deep breath and relax."

[1021] 8. User execution

[1022] Users can then take specific actions based on the notification, such as taking out the trash or taking a deep breath while waiting for the microwave to heat up. This allows users to live more efficiently without wasting time and reduces psychological stress.

[1023] Specific examples

[1024] Situation

[1025] When a user is preparing breakfast, they have to wait a few minutes while the food heats up in the microwave, which can be stressful for the user.

[1026] Data collection and transmission

[1027] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[1028] The device recognizes signs of stress from the user's facial expressions and heart rate.

[1029] Data analysis and sentiment-based recommendations

[1030] The server identifies the waiting time while the user is using the microwave in the kitchen and recognizes when the user is stressed based on the emotion engine.

[1031] The server selects "take out the trash while the microwave is waiting" and "take a deep breath and relax" as optimal actions and generates notification data.

[1032] User Notification

[1033] Notification data is sent from the server to the device, and the device displays to the user "Take out the trash while the microwave is waiting" and "Take a deep breath and relax."

[1034] User execution

[1035] The user follows the notification, takes out the trash while the microwave is waiting, and takes a deep breath, allowing the user to use their time efficiently and reduce stress.

[1036] By combining this system with an emotion engine, users can enjoy the benefits of effectively utilizing their time while reducing their psychological stress.

[1037] The processing flow will be explained below.

[1038] Step 1:

[1039] The device uses GPS and acceleration sensors to collect real-time location and movement information of the user. For example, if the user is in the kitchen at 8:00 AM, the device will record the location information as "8:00 AM - Kitchen."

[1040] Step 2:

[1041] The device records the user's downtime. If the user stays in a specific location for a certain amount of time, the device saves the downtime as data. For example, data such as "stayed in the kitchen from 8:00 AM to 8:05 AM" can be collected.

[1042] Step 3:

[1043] The device uses its built-in camera, microphone, and biometric sensors to collect the user's facial expressions, voice, heart rate, and other biometric signals in real time, and transmits this data to the emotion engine.

[1044] Step 4:

[1045] The server's emotion engine analyzes the transmitted facial expressions, voice, and biometric signals to determine the user's emotional state, for example, determining that the user is feeling stressed.

[1046] Step 5:

[1047] The terminal transmits the collected location information, movement information, and stop time data to the server at regular intervals (for example, every 5 minutes).

[1048] Step 6:

[1049] The server receives the data and stores it in a database. It converts the received data into an analytical format and analyzes the location and movement patterns. For example, it can identify a pattern that "the user is in the kitchen every morning at 8:00 AM."

[1050] Step 7:

[1051] The server then runs an algorithm that uses the analysis results to identify unnecessary movements and efficient routes, taking into account the user's emotional information and identifying specific data, such as "the microwave has a two-minute waiting time."

[1052] Step 8:

[1053] The server uses an optimization algorithm to select the best next action for the user based on the emotional information. For example, it could suggest actions such as "Take out the trash while the microwave is running" and "Take a deep breath and relax."

[1054] Step 9:

[1055] The server generates data to notify the user of the selected action, summarizing specific suggestions as messages such as "Take out the trash while the microwave is on standby" and "Take a deep breath and relax."

[1056] Step 10:

[1057] The server transmits the generated notification data to the terminal.

[1058] Step 11:

[1059] The device displays the received notification data to the user in real time, for example, notifying the user of multiple messages such as "Take out the trash while the microwave is in standby mode" and "Take a deep breath and relax."

[1060] Step 12:

[1061] Users can then take specific actions based on the notification, such as taking out the trash while the microwave is running or taking deep breaths to relax. This allows users to reduce wasted time, live more efficiently, and simultaneously reduce psychological stress.

[1062] Example 2

[1063] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1064] In modern life, users are expected to reduce unnecessary movements and use their time efficiently. However, conventional systems suggest next actions based on simple location and movement information without considering the user's emotions or psychological state, making it difficult to reduce the user's psychological stress.

[1065] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting location information, movement line information, and emotion information of the user, means for analyzing the collected location information, movement line information, and emotion information, means for proposing an optimal next action for the user based on the analysis results, means for notifying the user of the proposed action, and means for evaluating the user's behavior in accordance with the proposed action. This makes it possible to use time efficiently while reducing the user's psychological stress.

[1066] "User location information" means data that indicates the user's current geographic location.

[1067] "Traffic information" is data that indicates the route or trajectory of a user's movement when moving from one place to another.

[1068] "Emotional information" is data that indicates the user's psychological state and emotional fluctuations, and is extracted from facial expressions, voice, biometric signals, etc.

[1069] "Downtime" is data that records the length of time a user stays in a particular location.

[1070] "Analytics" refers to the algorithms and software used to process and analyze collected data to derive insights.

[1071] "Means of suggestion" refers to algorithms or software that specifically suggest optimal actions for users based on the analysis results.

[1072] "Notification means" refers to a device or communication means that notifies the user of the proposed action, such as a smartphone or smartwatch.

[1073] "Means for evaluating behavior" refers to algorithms or software that evaluate the results of a suggested action after the user performs it and use that information to make future suggestions.

[1074] The present invention is a system for collecting location information, movement information, and emotional information of a user to improve the efficiency of the user's life. This system recognizes the user's emotions and takes those emotions into consideration to suggest the optimal next action, thereby reducing the user's psychological stress. Specific embodiments of the system are described below.

[1075] Data collection

[1076] The device (wearable device) uses a GPS sensor and an acceleration sensor to collect real-time location and movement information of the user. For example, it records current location data and movement information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room." The device also records the user's inactivity time. Furthermore, it uses a built-in camera, microphone, and biometric sensors to collect biometric signals such as the user's facial expressions, voice, and heart rate.

[1077] Data transmission

[1078] The device sends the collected location information, movement information, and biometric signals to a server at regular intervals (for example, every 5 minutes) using Wi-Fi or mobile data communication.

[1079] emotion recognition

[1080] The server receives facial expression data, voice data, and biometric signals sent from the device and uses an emotion recognition algorithm to identify the user's emotional state. For example, if the user's facial expression is grim, it will be recognized as "anger."

[1081] Data analysis

[1082] The server stores the received data in a database, converts the stored data into an analytical format, and analyzes the location and movement patterns. This process includes a data cleansing process to remove noise.

[1083] Suggesting actions based on emotions

[1084] The server runs an algorithm to suggest the best next action that takes emotional information into account, for example, "If the user is tired, take some time to relax." It also selects the next action based on the optimization algorithm, choosing an action such as "take some deep breaths while waiting for the microwave to heat up."

[1085] Proposed action notifications

[1086] The server generates message data to notify the selected action and sends it to the device. For example, it creates messages such as "Take out the trash while the microwave is in standby mode" and "Take a deep breath and relax."

[1087] User Notification

[1088] The device analyzes the notification data received from the server and displays it on the display screen in real time. For example, messages such as "Take out the trash while the microwave is on standby" and "Take a deep breath and relax" may be displayed on the screen of a smartwatch or smartphone.

[1089] User execution

[1090] Users can take specific actions based on notifications from their device, such as taking out the trash while the microwave is running and taking deep breaths, allowing them to use their time efficiently and reduce psychological stress.

[1091] Specific examples

[1092] Situation

[1093] When a user is preparing breakfast, they have to wait a few minutes while the food heats up in the microwave, which can be stressful for the user.

[1094] Data collection and transmission

[1095] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[1096] The device recognizes signs of stress from the user's facial expressions and heart rate.

[1097] Data analysis and sentiment-based recommendations

[1098] The server identifies the waiting time while the user is using the microwave in the kitchen and recognizes when the user is stressed based on the emotion engine.

[1099] The server selects "take out the trash while the microwave is waiting" and "take a deep breath and relax" as optimal actions and generates notification data.

[1100] User Notification

[1101] Notification data is sent from the server to the device, and the device displays to the user "Take out the trash while the microwave is waiting" and "Take a deep breath and relax."

[1102] User execution

[1103] The user follows the notification, takes out the trash while the microwave is waiting, and takes a deep breath, allowing the user to use their time efficiently and reduce stress.

[1104] By combining this system with an emotion engine, users can enjoy the benefits of effectively utilizing their time while reducing their psychological stress.

[1105] Prompt Sentence Examples

[1106] "Please explain a system that collects location, movement, and emotional information from users and suggests optimal actions."

[1107] "Please tell me more about how you use the Emotion Engine to reduce user stress."

[1108] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1109] Step 1: Data collection

[1110] The device uses a GPS sensor and an acceleration sensor to obtain real-time location and movement information for the user. As input, the sensor receives the user's current location and route traveled, collecting information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room." As output, this data is stored in the device's internal memory. Additionally, the built-in camera captures the user's facial expressions, the microphone collects the user's voice in real time, and the biometric sensor captures vital signs such as heart rate and body temperature.

[1111] Step 2: Send data

[1112] The device transmits the collected location information, movement information, and biometric signals to the server at regular intervals (e.g., every 5 minutes). As input, data stored in the internal memory is retrieved and transmitted via Wi-Fi or mobile data communication. As output, data packets are generated from the device to the server and transferred to the server.

[1113] Step 3: Emotion Recognition

[1114] The server receives facial expression data, voice data, and biometric signals sent from the device. It receives these various data as input and runs an emotion recognition algorithm to identify the user's emotional state. For example, if the user's facial expression is grim, it will be recognized as "anger." The output is the identified emotional state, which is then stored in a database.

[1115] Step 4: Data analysis

[1116] The server stores the received data in a database. As input, it takes the received data and stores it in the database in SQL or NoSQL format. It then converts the stored data into a format for analysis. This includes a data cleansing process, for example, removing noise from location information. As output, clean data is generated and passed to the next analysis process.

[1117] Step 5: Consider emotions and suggest actions

[1118] The server runs an algorithm that takes emotional information into account to optimize the user's movement path. Location information, movement path information, pause time, and emotional information are taken as inputs and fed into the analysis algorithm. For example, a suggestion such as "If the user is tired, have them take some time to relax" is made. Optimized action suggestions are generated as output.

[1119] Step 6: Notification of proposed action

[1120] The server generates message data to notify the selected action. As input, it receives the action proposal generated in step 5 and creates a specific message. For example, messages such as "Take out the trash while the microwave is waiting" and "Take a deep breath and relax" are generated. As output, this notification data is sent to the device.

[1121] Step 7: Notify users

[1122] The device analyzes the notification data received from the server and displays it on the display screen in real time. As input, it takes the message received from the server and displays it on the user interface (UI). For example, messages such as "Take out the trash while the microwave is waiting" and "Take a deep breath and relax" are displayed on the screen of a smartwatch or smartphone. As output, specific instructions for action are presented to the user.

[1123] Step 8: Run the user

[1124] The user takes specific actions according to the notifications from the device. As input, the user receives the notification message from the device and starts the action according to the instructions. For example, the user takes out the trash while waiting for the microwave to run and takes a deep breath. As output, the user's action is completed, and efficient time utilization and reduction of psychological stress are achieved.

[1125] (Application example 2)

[1126] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1127] Conventional systems only collect and analyze user location and movement information, and are limited in their ability to suggest specific next actions that take into account the user's emotions and psychological state. This makes it difficult to encourage efficient behavior while reducing the user's psychological stress in scenarios that involve waiting, such as food delivery. The objective of the present invention is to solve this problem and provide a system that can suggest optimal actions that take into account the user's emotions and waiting time.

[1128] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user location information and movement line information, means for analyzing the collected location information, movement line information, and emotional information, means for proposing the optimal next action for the user based on the analysis results, and means for notifying the user in real time of the optimal action to take while waiting for food delivery. This makes it possible to efficiently utilize the waiting time while reducing the user's psychological stress.

[1129] "User Location Information" means the geographic coordinate information of the User's current location obtained by a GPS device or other location measurement means.

[1130] "Traffic information" is data that indicates the route and direction a user travels within a specific period of time.

[1131] "Emotional information" is information about the user's emotional state obtained by analyzing the user's facial expressions, voice, biometric signals, etc.

[1132] "Means for analyzing" refers to hardware or software functionality for processing and analyzing collected data to extract useful information.

[1133] "Means for suggesting next actions" refers to the function of an algorithm or system that suggests optimal actions or activities for users based on the analysis results.

[1134] "Waiting for food delivery" refers to the time while a user is waiting for food delivery.

[1135] "Means of notifying users of optimal actions in real time" refers to digital communication methods that take into account the user's current situation and emotional state and instantly inform the user of suggested actions.

[1136] This invention is a system that collects and analyzes a user's location information, movement information, and emotional information to suggest optimal actions for the user, thereby promoting efficient behavior and reducing psychological stress, particularly while waiting for food delivery.

[1137] System Overview

[1138] 1. Data Collection

[1139] The server uses GPS devices and other location measurement means to collect user location and movement information.

[1140] The server uses software for facial recognition, voice analysis, and biometric signal analysis to obtain the user's emotional information.

[1141] 2. Data Analysis

[1142] The server analyzes the collected location information, movement information, and emotional information and stores it in a database using an analysis platform and AI model.

[1143] 3. Action suggestions

[1144] The server then suggests the best next action for the user based on the analysis results, which includes an algorithm to reduce the user's psychological stress.

[1145] 4. Real-time notifications

[1146] The server communicates with the user's smartphone or appropriate end device using a notification API to notify the user of the optimal action in real time.

[1147] Specific examples

[1148] For example, if a user orders food delivery and is waiting for it to arrive, the system will follow these steps:

[1149] 1. Data Collection

[1150] The device collects the user's location information (e.g., "08:00 AM - Home"), movement information (e.g., "08:10 AM - Kitchen"), and emotional information (e.g., "stress" state based on facial expression analysis).

[1151] 2. Data Analysis

[1152] The server receives this information and uses an emotion engine to determine the user's psychological state and waiting status.

[1153] 3. Action suggestions

[1154] The server generates the best suggestion: "Your food delivery will arrive in 20 minutes. Take a deep breath."

[1155] 4. Notification

[1156] The server sends a notification to the device, and the user receives the suggestion via their smartphone.

[1157] This system allows users to use their time efficiently while reducing psychological stress while waiting for food delivery.

[1158] Prompt Sentence Examples

[1159] "Given the following setup, create an app that suggests optimal actions while waiting for a food delivery, taking into account the user's emotions and location. The user's emotional state can be 'stress', 'neutral', or 'happy'. The location can be 'home' or 'office'. Start 30 minutes before the scheduled delivery time."

[1160] This prompt can be used to leverage a generative AI model to flexibly generate more detailed suggestions and notification content.

[1161] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1162] Step 1:

[1163] Data collection

[1164] Specific behavior:

[1165] The device collects real-time location, movement, and emotional information from the user. Location information is obtained using GPS devices and other location measurement methods, movement information is obtained by tracking the user's movement path, and emotional information is obtained using facial recognition, voice analysis, and biometric signal analysis.

[1166] input:

[1167] User location information (e.g., "08:00 AM - home"), movement path information (e.g., "08:10 AM - kitchen"), and emotional information (e.g., "stressed" state based on facial expression analysis).

[1168] output:

[1169] A dataset of collected location, movement, and emotion information.

[1170] Step 2:

[1171] Data transmission

[1172] Specific behavior:

[1173] The device transmits the collected location information, movement information, and emotion information to a server at regular intervals (e.g., every 5 minutes) using a wireless communication module.

[1174] input:

[1175] A dataset of collected location, movement, and emotion information.

[1176] output:

[1177] The dataset sent to the server.

[1178] Step 3:

[1179] Data reception and storage

[1180] Specific behavior:

[1181] The server receives the data sent from the terminal and stores it in a database. The received data is converted into an analysis format.

[1182] input:

[1183] A dataset of location information, movement information, and emotional information sent from the device.

[1184] output:

[1185] Datasets for analysis stored in a database.

[1186] Step 4:

[1187] Data analysis

[1188] Specific behavior:

[1189] The server analyzes the user's behavioral patterns and emotional state based on location, movement, and emotional information stored in the database, and uses a generative AI model to evaluate the user's psychological state and suggest actions.

[1190] input:

[1191] A dataset for analyzing location information, movement information, and emotion information stored in a database.

[1192] output:

[1193] Analysis results (e.g., user is stressed, in the kitchen, waiting for food delivery).

[1194] Step 5:

[1195] Action suggestion generation

[1196] Specific behavior:

[1197] Based on the results of the data analysis, the server generates optimal actions to suggest to the user, such as "take deep breaths" or "listen to music to relax" to reduce stress.

[1198] input:

[1199] Analysis results (e.g., user is stressed, in the kitchen, waiting for food delivery).

[1200] output:

[1201] Specific suggestions for action (e.g., "Your food delivery will arrive in 20 minutes. Take a deep breath.").

[1202] Step 6:

[1203] Proposal Notification

[1204] Specific behavior:

[1205] The server then creates notification data based on the generated action suggestions and sends it to the device, which then displays the notification data to the user in real time.

[1206] input:

[1207] Action suggestions (e.g., "Your food delivery will arrive in 20 minutes. Take a deep breath.").

[1208] output:

[1209] Suggestions notified to users.

[1210] Step 7:

[1211] User execution

[1212] Specific behavior:

[1213] The user receives a notification from the device and performs a suggested action, such as "take a deep breath" or "stretch."

[1214] input:

[1215] Notified suggestions (e.g., "Your food delivery will arrive in 20 minutes. Take a deep breath.").

[1216] output:

[1217] The action taken (e.g., the user took a deep breath).

[1218] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1219] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1220] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1221] [Fourth embodiment]

[1222] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1223] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1224] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1225] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1226] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1227] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1228] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1229] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1230] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1231] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1232] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1233] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1234] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1235] This invention provides a system that functions as an assistant to help users lead their daily lives efficiently. It collects and analyzes the user's location information, movement path information, and downtime, and suggests the optimal next action. This system consists of three main components: the terminal (wearable device), the server, and the user.

[1236] System Overview

[1237] 1. Data Collection

[1238] The wearable device uses GPS and acceleration sensors to collect real-time location and movement information for the user. For example, it records current location data and movement information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room."

[1239] The device also records the user's downtime, for example, "staying in the kitchen" from 8:00 AM to 8:05 AM.

[1240] 2. Data Transmission

[1241] The device sends the collected location information, movement information, and stop time data to the server at regular intervals (e.g., every 5 minutes), where the data is accumulated and used for analysis.

[1242] 3. Data analysis

[1243] The server receives the data and first stores it in a database. It then converts it into an analytical format and analyzes the user's movement patterns and downtime. For example, it can identify a pattern such as "every morning, the user goes to the kitchen at 8:00 AM."

[1244] The server uses the collected data to run algorithms that identify wasted time and efficient routes for users, generating specific action suggestions such as "Take out the trash during the two minutes you wait for the microwave."

[1245] 4. Action proposals

[1246] The server then selects the optimal next action based on the analysis results, such as suggesting other housework that would be best suited to the time the user is waiting in the kitchen.

[1247] The server generates notification data that suggests the selected action, and generates a message such as "Take out the garbage while the microwave is in standby mode."

[1248] 5. Notice to Users

[1249] The server sends the generated notification data to the device, which then displays the received notification data to the user in real time. For example, it displays a message such as "Take out the trash while the microwave is in standby mode."

[1250] 6. User execution

[1251] The user follows the notification to take out the trash. This suggestion helps users reduce wasted time and do their housework efficiently.

[1252] Specific examples

[1253] Situation

[1254] A user is preparing breakfast and has to wait a few minutes while the ingredients heat up in the microwave.

[1255] Data collection and transmission

[1256] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[1257] Data analysis and recommendations

[1258] The server identifies the amount of time a user waits while using a microwave in the kitchen, and also determines from past logs that a nearby trash can is full.

[1259] The server selects "take out the garbage while the microwave is in standby mode" as the optimal action and generates notification data.

[1260] User Notification

[1261] Notification data is sent from the server to the device, and the device displays a message to the user saying, "Take out the trash while the microwave is in standby mode."

[1262] User execution

[1263] The user follows the notification and takes out the trash while the microwave is waiting.

[1264] This system allows users to effectively utilize their time and improve daily efficiency.The effects of the present invention are realized by such specific embodiments.

[1265] The processing flow will be explained below.

[1266] Step 1:

[1267] The device collects the user's current location and movement information in real time using GPS and acceleration sensors. For example, if the user is in the kitchen, the device will record the location information as "08:00 AM - Kitchen."

[1268] Step 2:

[1269] The device records the user's downtime. If the user stays in a specific location for a certain amount of time, the device saves the downtime as data. For example, it can collect data such as "stayed in the kitchen from 8:00 AM to 8:05 AM."

[1270] Step 3:

[1271] The device sends the collected location information and stop time data to the server at regular intervals (for example, every 5 minutes).

[1272] Step 4:

[1273] The server receives the data sent from the terminal, stores it in a database, and converts the received data into a format for analysis.

[1274] Step 5:

[1275] The server analyzes location information, movement information, and downtime patterns, for example, identifying a pattern that "the user is in the kitchen at 8:00 AM every morning."

[1276] Step 6:

[1277] Based on the analysis results, the server runs an algorithm to identify unnecessary movements and efficient routes for users, such as identifying specific data such as "there is a two-minute wait time for the microwave."

[1278] Step 7:

[1279] The server uses an optimization algorithm to select the best next action for the user, for example, suggesting an action such as "taking out the trash while the microwave is waiting."

[1280] Step 8:

[1281] The server generates data to notify the user of the selected action, summarizing the suggestion as a message such as "Take out the trash while the microwave is in standby mode."

[1282] Step 9:

[1283] The server transmits this notification data to the terminal.

[1284] Step 10:

[1285] The device displays the received notification data to the user in real time, for example, by sending a message to the user saying, "Take out the trash while the microwave is in standby mode."

[1286] Step 11:

[1287] Users can then take specific actions based on the notification, such as taking out the trash while the microwave is waiting. This allows users to live their lives efficiently without wasting time.

[1288] Example 1

[1289] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1290] While conventional assistant systems collect information on the user's location and movement, there are no systems that can effectively analyze this information and suggest beneficial next actions for the user in real time. As a result, users often waste time, which reduces the efficiency of their daily lives.

[1291] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1292] In this invention, the server includes means for collecting user location information and movement line information, means for transmitting the collected location information and movement line information to a central processing unit at regular intervals, means for the central processing unit to store the collected location information and movement line information, convert the collected information into an analytical format and analyze it, means for selecting the optimal next action for the user based on the analysis results and generating notification data, and means for transmitting the generated notification data to the user's information terminal and displaying it to the user in real time, thereby enabling users to use their time more efficiently and significantly improve the efficiency of their daily lives.

[1293] "User" refers to an individual or group of people who use the System.

[1294] "Location Information" is data that indicates a user's current geographic location.

[1295] "Flow line information" is data that indicates the trajectory or route that a user has taken.

[1296] "Dwell time" is data that indicates the amount of time a user remains in a particular location.

[1297] A "terminal" is a device carried or worn by a user, examples of which include wearable devices.

[1298] A "server" is a central computer that receives collected data, analyzes it, and sends the results to the user's device.

[1299] "Central processing unit" is a general term for the hardware and software installed on a server that stores, converts, and analyzes data.

[1300] "Notification data" refers to data that includes suggestions and instructions for the user that are generated based on the analysis results.

[1301] An "information terminal" is a device that receives notification data sent from a server and displays it to the user.

[1302] "Interval" refers to the time interval during which data is collected or transmitted.

[1303] This invention is an assistant system that helps users lead their daily lives efficiently. It collects and analyzes the user's location information, movement information, and downtime, and suggests the optimal next action. This system consists of three main components: the terminal (wearable device), the server, and the user.

[1304] Data collection

[1305] The device (wearable device) uses GPS and acceleration sensors to collect the user's location and movement information in real time. Specifically, if the user is in the kitchen at 08:00 AM, the data will be recorded as "08:00 AM - Kitchen." If the user moves to the living room at 08:05 AM, the data will be recorded as "08:05 AM - Living Room." The device also records the time the user is stationary in a particular location. For example, if the user is in the kitchen from 08:00 AM to 08:05 AM, the data will be collected as "Stayed in the kitchen from 08:00 AM to 08:05 AM."

[1306] Data transmission

[1307] The device sends the collected location information, movement information, and stop time data to a server at regular intervals (e.g., every 5 minutes). The data is sent using wireless communication technology (e.g., Wi-Fi or Bluetooth). This allows the data to be accumulated on the server and used for later analysis.

[1308] Data reception and storage

[1309] The server receives the data sent from the device. The received data is first stored in a database. For example, data such as "08:00 AM - Kitchen" and "08:05 AM - Living Room" is stored on the server.

[1310] Data conversion and analysis

[1311] The server converts the stored data into a format for analysis. This format conversion involves shaping the data and removing unnecessary information. The server then analyzes movement patterns and downtime. For example, it might analyze a pattern such as "every morning, the user goes to the kitchen at 8:00 AM."

[1312] Action proposal generation

[1313] The server selects the optimal next action based on the analysis results. For example, while the user is waiting for food to heat up in the microwave, it will suggest an action to use that waiting time efficiently. Specifically, it generates a suggestion such as "Take out the trash while the microwave is on." This suggestion is selected based on the user's past behavioral patterns and current situation (for example, whether the trash can is full).

[1314] User Notification

[1315] The server generates the selected action suggestion as notification data and sends it to the device. For example, it generates a message saying, "Take out the trash while the microwave is waiting." The device then receives the notification and displays it to the user in real time.

[1316] User execution

[1317] Users can check notifications from their devices and take suggested actions, such as taking out the trash while the microwave is running. This action allows users to use their time efficiently and get on with their daily tasks.

[1318] Specific examples

[1319] Situation

[1320] When a user is preparing breakfast, there is a few minutes of waiting while the ingredients heat up in the microwave.

[1321] Specific examples of data collection and transmission

[1322] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[1323] The device sends this data to the server every five minutes.

[1324] Examples of data reception and storage

[1325] The server receives data such as "07:30 - 07:33 Kitchen" and stores it in a database.

[1326] Specific examples of data conversion and analysis

[1327] The server converts the stored data into an analytical format and determines that "the user was using the microwave in the kitchen between 07:30 and 07:33."

[1328] Example of action proposal generation

[1329] The server determines that the user is waiting in the kitchen between 07:30 and 07:33, and uses this waiting time to suggest taking out the trash.

[1330] Examples of user notifications

[1331] The server generates notification data saying "Take out the garbage while the microwave is in standby mode" and transmits it to the terminal.

[1332] User execution example

[1333] Users can follow the notification and take out the trash while the microwave is waiting, allowing them to use their time effectively and improve their daily efficiency.

[1334] Example prompts to input to the generative AI model

[1335] User: "Can you give me some suggestions on how to make the most of my time waiting in the kitchen each morning?"

[1336] Prompt for generative AI model: "Suggest efficient actions that the user can take while waiting in the kitchen each morning, such as taking out the trash or preparing ingredients."

[1337] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1338] Step 1: Data collection

[1339] The device collects the user's location information and movement path information. Specifically, it uses GPS and an acceleration sensor to record the user's current location and movement trajectory. For example, if the user is in the kitchen at 8:00 AM, the data "8:00 AM - Kitchen" is collected. In addition, the device also records the user's downtime. For example, the data collected might be "Stayed in the kitchen from 8:00 AM to 8:05 AM." This data will be used as input for analysis in later steps.

[1340] Step 2: Send data

[1341] The device sends the collected location information, movement information, and stop time data to the server at regular intervals (e.g., every 5 minutes). Specifically, data such as "08:00 AM - kitchen" and "08:05 AM - living room" is sent to the server every 5 minutes. This transmission uses wireless communication technology (e.g., Wi-Fi, Bluetooth). This data is accumulated on the server and becomes input data for later analysis.

[1342] Step 3: Receiving and storing data

[1343] The server receives the data sent from the device. The received data is first stored in a database. For example, data such as "08:00 AM - Kitchen" and "08:05 AM - Living Room" is accumulated in the database. The output of this step is the location information and movement line information stored in the database.

[1344] Step 4: Data conversion and analysis

[1345] The server converts the stored data into a format for analysis. Specifically, it formats the data and deletes unnecessary information. Next, the server analyzes movement patterns and downtime. For example, it analyzes patterns such as "every morning, the user goes to the kitchen at 8:00 AM" and "there is a two-minute wait time for the microwave." The input data is the location information and movement information stored in the database, and the output data is the analysis results.

[1346] Step 5: Action proposal generation

[1347] The server selects the optimal next action based on the analysis results. Specifically, it suggests actions to use the user's waiting time efficiently while they wait for food to heat up in the microwave. For example, it generates a suggestion such as "Take out the trash while the microwave is on." The input to this step is the analysis results, and the output is the selected action suggestion (notification data).

[1348] Step 6: Notify users

[1349] The server sends the generated notification data to the user's device. For example, a notification message saying "Take out the trash while the microwave is in standby mode" is sent to the device. The device displays the received notification data in real time. The input data is the notification data, and the output is the message to be displayed to the user.

[1350] Step 7: Run the user

[1351] The user checks the notification from the device and performs the suggested action. For example, the user takes out the trash while the microwave is waiting. The input of this step is the notification message from the device, and the output is the user's action. This allows the user to use their time efficiently and improve the efficiency of their daily life.

[1352] (Application example 1)

[1353] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1354] In modern food delivery services, delivery workers must efficiently travel to multiple destinations. However, current systems do not provide suggestions for optimal routes or appropriate break times, resulting in inefficiency. They also have difficulty responding to real-time changes in the situation. This can lead to delivery delays and fatigue among delivery workers, resulting in a decline in overall service quality. Therefore, a system is needed that allows delivery workers to work efficiently and receive suggestions for optimal routes and break times in real time.

[1355] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1356] In this invention, the server includes means for collecting user location information, flow line information, and stop time, means for analyzing the collected location information, flow line information, and stop time, means for proposing an efficient delivery route based on the analysis results, and means for notifying the user of optimal actions and rest times in real time. This allows delivery personnel to receive suggestions for optimal routes and rest times in real time, thereby enabling delivery work to be performed efficiently, preventing delivery delays, and reducing fatigue of delivery personnel.

[1357] "Location Information" means data that describes the current location of a user or device, obtained using GPS or other location measurement technologies.

[1358] "Traffic information" is data showing the route taken by a user or device, and is generated from continuous recording of location information.

[1359] "Downtime" is data that shows the amount of time a user or device spends in a particular location.

[1360] "Analysis tools" are algorithms and software used to extract meaningful information and patterns from collected data.

[1361] "Means for suggesting the optimal next action" refers to systems or software that suggest the most efficient next action to the user based on the analysis results.

[1362] A "delivery route" refers to the route of locations that should be visited in the most efficient order when making a delivery.

[1363] "Real-time" means that processing and response occur immediately at the moment the data is generated.

[1364] "Means of notification" refers to devices or software that notify users of information or actions, including the ability to send notifications to smartphones or wearable devices.

[1365] A "prompt" is a textual suggestion or instruction that a generative AI model provides to a user.

[1366] A "generative AI model" is a machine learning algorithm or artificial intelligence technology that generates new information or suggestions based on data.

[1367] This invention relates to an assistance system that enables delivery personnel to deliver food efficiently. Specifically, this system collects and analyzes delivery personnel's location information, movement line information, and stop time, and proposes optimal delivery routes and break times in real time.

[1368] The system mainly consists of three main components: terminals, servers, and users.

[1369] Terminal

[1370] The terminal is a wearable device such as a smartphone or smart glasses. This terminal uses GPS and an acceleration sensor to collect real-time location information and movement information of the delivery person. It also records the time the delivery person stays in a specific location (stop time). For example, the terminal records location information such as "10:00 AM - Cafe" and "10:15 AM - Pizza place."

[1371] server

[1372] The server receives and stores the location information, movement information, and stop time data sent from the device. The server analyzes this data and generates the optimal next action for the delivery person. The analysis uses Python and R languages ​​to analyze movement patterns and propose efficient routes. It also uses generative AI models (e.g., TensorFlow, Scikit-learn) to prompt the user on the optimal course of action.

[1373] For example, the server generates a prompt such as, "The delivery person's current location is latitude 35.6895, longitude 139.6917. Please suggest the next best delivery destination. Please take traffic conditions and stop time into consideration." and notifies the delivery person of the analysis results.

[1374] user

[1375] Delivery workers receive suggestions from their devices. For example, they can receive a notification such as, "Your next delivery destination is the nearest cafe. After that, deliver to a pizza place." New suggestions are sent in real time as needed during delivery, allowing for efficient delivery route management.

[1376] As a concrete example, the terminal works as follows:

[1377] 1. The device periodically collects the delivery person's location information.

[1378] 2. The collected data is sent to the server and stored in a database (e.g., MySQL, MongoDB).

[1379] 3. The server analyzes the data using Python or R, and uses TensorFlow or Scikit-learn to predict the optimal next action using an AI model.

[1380] 4. A prompt is generated based on the analysis results and sent to the delivery person's device.

[1381] 5. Delivery personnel act on notifications and complete deliveries efficiently.

[1382] For example, a notification will appear on the delivery person's device stating, "The next best delivery from your current location is point A, based on distance, followed by a delivery at point B, taking current traffic conditions into account."

[1383] In this way, the present invention allows delivery personnel to work efficiently in real time, preventing delivery delays and contributing to an overall improvement in service quality.

[1384] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1385] Step 1:

[1386] The terminal obtains the delivery person's location information using a GPS sensor. It periodically collects location data (latitude and longitude) from the GPS sensor. The input is location information (latitude and longitude), and the output is the collected location data. Specifically, the GPS module inside the terminal reads the current latitude and longitude and temporarily stores the location information in memory.

[1387] Step 2:

[1388] The device sends the collected location information to the server at regular intervals (e.g., every 5 minutes). The data is sent using a RESTful API as the communication protocol. The input is the location data, and the output is the status of transmission to the server. Specifically, the device packages the location information in JSON format and sends the data to the server using an HTTP POST request.

[1389] Step 3:

[1390] The server stores the received location information in a database, using MySQL or MongoDB as the database. The input is the location data, and the output is the record status in the database. Specifically, the server parses the received location data, converts it into an appropriate format, and inserts it into the database using SQL queries or NoSQL operations.

[1391] Step 4:

[1392] The server analyzes the location information, movement line information, and stop times stored in the database. The analysis uses Python or R programming language to analyze movement line patterns and identify inefficient routes. The input is the stored location information, and the output is the analysis results (for example, movement line patterns). Specifically, the server runs an analysis script and extracts patterns of movement lines and stop times based on past location data.

[1393] Step 5:

[1394] The server uses a generative AI model based on the analysis results to calculate the next optimal action and route. TensorFlow and Scikit-learn are used as the generative AI model. The input is the analysis results, and the output is the proposed next action and route. Specifically, the server inputs the analysis results into the AI ​​model and predicts the optimal route and action.

[1395] Step 6:

[1396] The server notifies the device of the generated action proposal as a prompt. The prompt is created based on the output from the generative AI model and sent to the device in real time. The input is the action proposal, and the output is a notification to the device. Specifically, the server converts the proposal into a text prompt and notifies the delivery person's device using Firebase Cloud Messaging (FCM).

[1397] Step 7:

[1398] The user (delivery person) acts according to the suggestions notified to the terminal. For example, the user confirms the notification that "The next delivery destination is the nearest cafe. After that, deliver to the pizza place," and makes the delivery along the specified route. The input is the notified prompt sentence, and the output is the executed action. In concrete terms, the delivery person confirms the contents of the notification and moves and delivers according to the instructions.

[1399] This allows delivery personnel to deliver efficiently, shortening delivery times and improving work efficiency.

[1400] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1401] The present invention provides a system for collecting location information, movement information, and emotional information of a user to improve the efficiency of the user's life. This system recognizes the user's emotions and, taking those emotions into consideration, suggests the optimal next action, thereby reducing the user's psychological stress. Specific embodiments of the system are described below.

[1402] System Overview

[1403] 1. Data Collection

[1404] The wearable device uses GPS and acceleration sensors to collect real-time location and movement information for the user. For example, it records current location data and movement information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room."

[1405] The device records the user's downtime, for example, collecting data such as "stayed in the kitchen" from "08:00 AM" to "08:05 AM."

[1406] The device uses its built-in camera, microphone, and biometric sensors to collect biometric signals such as the user's facial expressions, voice, and heart rate, and transmits them to the emotion engine.

[1407] 2. Emotion recognition

[1408] The server analyzes facial expressions, voice, and biometric signals sent from the device to identify the user's emotional state, for example, recognizing whether the user is in a specific emotional state such as anger, sadness, or joy.

[1409] 3. Data Transmission

[1410] The terminal transmits the collected location information, movement line information, and stop time data to the server at regular intervals (for example, every 5 minutes).

[1411] 4. Data Analysis

[1412] The server receives the transmitted data, stores it in a database, and converts the received data into a format for analysis.

[1413] The server analyzes location information, movement information, and downtime patterns, for example, identifying a pattern that "the user is in the kitchen every morning at 8:00 AM."

[1414] 5. Emotion-based action suggestions

[1415] The server then takes into account the collected emotional information and runs an algorithm to identify unnecessary movements and efficient routes for the user, making specific suggestions such as "If the user is tired, they should take some time to relax."

[1416] The server uses an optimization algorithm to select the next action, taking emotional information into account. For example, it might suggest an action such as "take a deep breath while waiting for the microwave to turn on."

[1417] 6. Notification of Proposed Actions

[1418] The server generates data to notify the user of the selected action. For example, in addition to the message "Take out the trash while the microwave is in standby mode," it generates a message such as "Take a deep breath and relax."

[1419] The server transmits this notification data to the terminal.

[1420] 7. Notice to Users

[1421] The device displays the received notification data to the user in real time, for example, notifying the user of multiple messages such as "Take out the trash while the microwave is in standby mode" and "Take a deep breath and relax."

[1422] 8. User execution

[1423] Users can then take specific actions based on the notification, such as taking out the trash or taking a deep breath while waiting for the microwave to heat up. This allows users to live more efficiently without wasting time and reduces psychological stress.

[1424] Specific examples

[1425] Situation

[1426] When a user is preparing breakfast, they have to wait a few minutes while the food heats up in the microwave, which can be stressful for the user.

[1427] Data collection and transmission

[1428] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[1429] The device recognizes signs of stress from the user's facial expressions and heart rate.

[1430] Data analysis and sentiment-based recommendations

[1431] The server identifies the waiting time while the user is using the microwave in the kitchen and recognizes when the user is stressed based on the emotion engine.

[1432] The server selects "take out the trash while the microwave is waiting" and "take a deep breath and relax" as optimal actions and generates notification data.

[1433] User Notification

[1434] Notification data is sent from the server to the device, and the device displays to the user "Take out the trash while the microwave is waiting" and "Take a deep breath and relax."

[1435] User execution

[1436] The user follows the notification, takes out the trash while the microwave is waiting, and takes a deep breath, allowing the user to use their time efficiently and reduce stress.

[1437] By combining this system with an emotion engine, users can enjoy the benefits of effectively utilizing their time while reducing their psychological stress.

[1438] The processing flow will be explained below.

[1439] Step 1:

[1440] The device uses GPS and acceleration sensors to collect real-time location and movement information of the user. For example, if the user is in the kitchen at 8:00 AM, the device will record the location information as "8:00 AM - Kitchen."

[1441] Step 2:

[1442] The device records the user's downtime. If the user stays in a specific location for a certain amount of time, the device saves the downtime as data. For example, data such as "stayed in the kitchen from 8:00 AM to 8:05 AM" can be collected.

[1443] Step 3:

[1444] The device uses its built-in camera, microphone, and biometric sensors to collect the user's facial expressions, voice, heart rate, and other biometric signals in real time, and transmits this data to the emotion engine.

[1445] Step 4:

[1446] The server's emotion engine analyzes the transmitted facial expressions, voice, and biometric signals to determine the user's emotional state, for example, determining that the user is feeling stressed.

[1447] Step 5:

[1448] The terminal transmits the collected location information, movement information, and stop time data to the server at regular intervals (for example, every 5 minutes).

[1449] Step 6:

[1450] The server receives the data and stores it in a database. It converts the received data into an analytical format and analyzes the location and movement patterns. For example, it can identify a pattern that "the user is in the kitchen every morning at 8:00 AM."

[1451] Step 7:

[1452] The server then runs an algorithm that uses the analysis results to identify unnecessary movements and efficient routes, taking into account the user's emotional information and identifying specific data, such as "the microwave has a two-minute waiting time."

[1453] Step 8:

[1454] The server uses an optimization algorithm to select the best next action for the user based on the emotional information. For example, it could suggest actions such as "Take out the trash while the microwave is running" and "Take a deep breath and relax."

[1455] Step 9:

[1456] The server generates data to notify the user of the selected action, summarizing specific suggestions as messages such as "Take out the trash while the microwave is on standby" and "Take a deep breath and relax."

[1457] Step 10:

[1458] The server transmits the generated notification data to the terminal.

[1459] Step 11:

[1460] The device displays the received notification data to the user in real time, for example, notifying the user of multiple messages such as "Take out the trash while the microwave is in standby mode" and "Take a deep breath and relax."

[1461] Step 12:

[1462] Users can then take specific actions based on the notification, such as taking out the trash while the microwave is running or taking deep breaths to relax. This allows users to reduce wasted time, live more efficiently, and simultaneously reduce psychological stress.

[1463] Example 2

[1464] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1465] In modern life, users are expected to reduce unnecessary movements and use their time efficiently. However, conventional systems suggest next actions based on simple location and movement information without considering the user's emotions or psychological state, making it difficult to reduce the user's psychological stress.

[1466] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting location information, movement line information, and emotion information of the user, means for analyzing the collected location information, movement line information, and emotion information, means for proposing an optimal next action for the user based on the analysis results, means for notifying the user of the proposed action, and means for evaluating the user's behavior in accordance with the proposed action. This makes it possible to use time efficiently while reducing the user's psychological stress.

[1467] "User location information" means data that indicates the user's current geographic location.

[1468] "Traffic information" is data that indicates the route or trajectory of a user's movement when moving from one place to another.

[1469] "Emotional information" is data that indicates the user's psychological state and emotional fluctuations, and is extracted from facial expressions, voice, biometric signals, etc.

[1470] "Downtime" is data that records the length of time a user stays in a particular location.

[1471] "Analytics" refers to the algorithms and software used to process and analyze collected data to derive insights.

[1472] "Means of suggestion" refers to algorithms or software that specifically suggest optimal actions for users based on the analysis results.

[1473] "Notification means" refers to a device or communication means that notifies the user of the proposed action, such as a smartphone or smartwatch.

[1474] "Means for evaluating behavior" refers to algorithms or software that evaluate the results of a suggested action after the user performs it and use that information to make future suggestions.

[1475] The present invention is a system for collecting location information, movement information, and emotional information of a user to improve the efficiency of the user's life. This system recognizes the user's emotions and takes those emotions into consideration to suggest the optimal next action, thereby reducing the user's psychological stress. Specific embodiments of the system are described below.

[1476] Data collection

[1477] The device (wearable device) uses a GPS sensor and an acceleration sensor to collect real-time location and movement information of the user. For example, it records current location data and movement information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room." The device also records the user's inactivity time. Furthermore, it uses a built-in camera, microphone, and biometric sensors to collect biometric signals such as the user's facial expressions, voice, and heart rate.

[1478] Data transmission

[1479] The device sends the collected location information, movement information, and biometric signals to a server at regular intervals (for example, every 5 minutes) using Wi-Fi or mobile data communication.

[1480] emotion recognition

[1481] The server receives facial expression data, voice data, and biometric signals sent from the device and uses an emotion recognition algorithm to identify the user's emotional state. For example, if the user's facial expression is grim, it will be recognized as "anger."

[1482] Data analysis

[1483] The server stores the received data in a database, converts the stored data into an analytical format, and analyzes the location and movement patterns. This process includes a data cleansing process to remove noise.

[1484] Suggesting actions based on emotions

[1485] The server runs an algorithm to suggest the best next action that takes emotional information into account, for example, "If the user is tired, take some time to relax." It also selects the next action based on the optimization algorithm, choosing an action such as "take some deep breaths while waiting for the microwave to heat up."

[1486] Proposed action notifications

[1487] The server generates message data to notify the selected action and sends it to the device. For example, it creates messages such as "Take out the trash while the microwave is in standby mode" and "Take a deep breath and relax."

[1488] User Notification

[1489] The device analyzes the notification data received from the server and displays it on the display screen in real time. For example, messages such as "Take out the trash while the microwave is on standby" and "Take a deep breath and relax" may be displayed on the screen of a smartwatch or smartphone.

[1490] User execution

[1491] Users can take specific actions based on notifications from their device, such as taking out the trash while the microwave is running and taking deep breaths, allowing them to use their time efficiently and reduce psychological stress.

[1492] Specific examples

[1493] Situation

[1494] When a user is preparing breakfast, they have to wait a few minutes while the food heats up in the microwave, which can be stressful for the user.

[1495] Data collection and transmission

[1496] The device records "07:30 - 07:33 Kitchen" (microwave in use, waiting).

[1497] The device recognizes signs of stress from the user's facial expressions and heart rate.

[1498] Data analysis and sentiment-based recommendations

[1499] The server identifies the waiting time while the user is using the microwave in the kitchen and recognizes when the user is stressed based on the emotion engine.

[1500] The server selects "take out the trash while the microwave is waiting" and "take a deep breath and relax" as optimal actions and generates notification data.

[1501] User Notification

[1502] Notification data is sent from the server to the device, and the device displays to the user "Take out the trash while the microwave is waiting" and "Take a deep breath and relax."

[1503] User execution

[1504] The user follows the notification, takes out the trash while the microwave is waiting, and takes a deep breath, allowing the user to use their time efficiently and reduce stress.

[1505] By combining this system with an emotion engine, users can enjoy the benefits of effectively utilizing their time while reducing their psychological stress.

[1506] Prompt Sentence Examples

[1507] "Please explain a system that collects location, movement, and emotional information from users and suggests optimal actions."

[1508] "Please tell me more about how you use the Emotion Engine to reduce user stress."

[1509] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1510] Step 1: Data collection

[1511] The device uses a GPS sensor and an acceleration sensor to obtain real-time location and movement information for the user. As input, the sensor receives the user's current location and route traveled, collecting information such as "08:00 AM - Kitchen" and "08:05 AM - Living Room." As output, this data is stored in the device's internal memory. Additionally, the built-in camera captures the user's facial expressions, the microphone collects the user's voice in real time, and the biometric sensor captures vital signs such as heart rate and body temperature.

[1512] Step 2: Send data

[1513] The device transmits the collected location information, movement information, and biometric signals to the server at regular intervals (e.g., every 5 minutes). As input, data stored in the internal memory is retrieved and transmitted via Wi-Fi or mobile data communication. As output, data packets are generated from the device to the server and transferred to the server.

[1514] Step 3: Emotion Recognition

[1515] The server receives facial expression data, voice data, and biometric signals sent from the device. It receives these various data as input and runs an emotion recognition algorithm to identify the user's emotional state. For example, if the user's facial expression is grim, it will be recognized as "anger." The output is the identified emotional state, which is then stored in a database.

[1516] Step 4: Data analysis

[1517] The server stores the received data in a database. As input, it takes the received data and stores it in the database in SQL or NoSQL format. It then converts the stored data into a format for analysis. This includes a data cleansing process, for example, removing noise from location information. As output, clean data is generated and passed to the next analysis process.

[1518] Step 5: Consider emotions and suggest actions

[1519] The server runs an algorithm that takes emotional information into account to optimize the user's movement path. Location information, movement path information, pause time, and emotional information are taken as inputs and fed into the analysis algorithm. For example, a suggestion such as "If the user is tired, have them take some time to relax" is made. Optimized action suggestions are generated as output.

[1520] Step 6: Notification of proposed action

[1521] The server generates message data to notify the selected action. As input, it receives the action proposal generated in step 5 and creates a specific message. For example, messages such as "Take out the trash while the microwave is waiting" and "Take a deep breath and relax" are generated. As output, this notification data is sent to the device.

[1522] Step 7: Notify users

[1523] The device analyzes the notification data received from the server and displays it on the display screen in real time. As input, it takes the message received from the server and displays it on the user interface (UI). For example, messages such as "Take out the trash while the microwave is waiting" and "Take a deep breath and relax" are displayed on the screen of a smartwatch or smartphone. As output, specific instructions for action are presented to the user.

[1524] Step 8: Run the user

[1525] The user takes specific actions according to the notifications from the device. As input, the user receives the notification message from the device and starts the action according to the instructions. For example, the user takes out the trash while waiting for the microwave to run and takes a deep breath. As output, the user's action is completed, and efficient time utilization and reduction of psychological stress are achieved.

[1526] (Application example 2)

[1527] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1528] Conventional systems only collect and analyze user location and movement information, and are limited in their ability to suggest specific next actions that take into account the user's emotions and psychological state. This makes it difficult to encourage efficient behavior while reducing the user's psychological stress in scenarios that involve waiting, such as food delivery. The objective of the present invention is to solve this problem and provide a system that can suggest optimal actions that take into account the user's emotions and waiting time.

[1529] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user location information and movement line information, means for analyzing the collected location information, movement line information, and emotional information, means for proposing the optimal next action for the user based on the analysis results, and means for notifying the user in real time of the optimal action to take while waiting for food delivery. This makes it possible to efficiently utilize the waiting time while reducing the user's psychological stress.

[1530] "User Location Information" means the geographic coordinate information of the User's current location obtained by a GPS device or other location measurement means.

[1531] "Traffic information" is data that indicates the route and direction a user travels within a specific period of time.

[1532] "Emotional information" is information about the user's emotional state obtained by analyzing the user's facial expressions, voice, biometric signals, etc.

[1533] "Means for analyzing" refers to hardware or software functionality for processing and analyzing collected data to extract useful information.

[1534] "Means for suggesting next actions" refers to the function of an algorithm or system that suggests optimal actions or activities for users based on the analysis results.

[1535] "Waiting for food delivery" refers to the time while a user is waiting for food delivery.

[1536] "Means of notifying users of optimal actions in real time" refers to digital communication methods that take into account the user's current situation and emotional state and instantly inform the user of suggested actions.

[1537] This invention is a system that collects and analyzes a user's location information, movement information, and emotional information to suggest optimal actions for the user, thereby promoting efficient behavior and reducing psychological stress, particularly while waiting for food delivery.

[1538] System Overview

[1539] 1. Data Collection

[1540] The server uses GPS devices and other location measurement means to collect user location and movement information.

[1541] The server uses software for facial recognition, voice analysis, and biometric signal analysis to obtain the user's emotional information.

[1542] 2. Data Analysis

[1543] The server analyzes the collected location information, movement information, and emotional information and stores it in a database using an analysis platform and AI model.

[1544] 3. Action suggestions

[1545] The server then suggests the best next action for the user based on the analysis results, which includes an algorithm to reduce the user's psychological stress.

[1546] 4. Real-time notifications

[1547] The server communicates with the user's smartphone or appropriate end device using a notification API to notify the user of the optimal action in real time.

[1548] Specific examples

[1549] For example, if a user orders food delivery and is waiting for it to arrive, the system will follow these steps:

[1550] 1. Data Collection

[1551] The device collects the user's location information (e.g., "08:00 AM - Home"), movement information (e.g., "08:10 AM - Kitchen"), and emotional information (e.g., "stress" state based on facial expression analysis).

[1552] 2. Data Analysis

[1553] The server receives this information and uses an emotion engine to determine the user's psychological state and waiting status.

[1554] 3. Action suggestions

[1555] The server generates the best suggestion: "Your food delivery will arrive in 20 minutes. Take a deep breath."

[1556] 4. Notification

[1557] The server sends a notification to the device, and the user receives the suggestion via their smartphone.

[1558] This system allows users to use their time efficiently while reducing psychological stress while waiting for food delivery.

[1559] Prompt Sentence Examples

[1560] "Given the following setup, create an app that suggests optimal actions while waiting for a food delivery, taking into account the user's emotions and location. The user's emotional state can be 'stress', 'neutral', or 'happy'. The location can be 'home' or 'office'. Start 30 minutes before the scheduled delivery time."

[1561] This prompt can be used to leverage a generative AI model to flexibly generate more detailed suggestions and notification content.

[1562] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1563] Step 1:

[1564] Data collection

[1565] Specific behavior:

[1566] The device collects real-time location, movement, and emotional information from the user. Location information is obtained using GPS devices and other location measurement methods, movement information is obtained by tracking the user's movement path, and emotional information is obtained using facial recognition, voice analysis, and biometric signal analysis.

[1567] input:

[1568] User location information (e.g., "08:00 AM - home"), movement path information (e.g., "08:10 AM - kitchen"), and emotional information (e.g., "stressed" state based on facial expression analysis).

[1569] output:

[1570] A dataset of collected location, movement, and emotion information.

[1571] Step 2:

[1572] Data transmission

[1573] Specific behavior:

[1574] The device transmits the collected location information, movement information, and emotion information to a server at regular intervals (e.g., every 5 minutes) using a wireless communication module.

[1575] input:

[1576] A dataset of collected location, movement, and emotion information.

[1577] output:

[1578] The dataset sent to the server.

[1579] Step 3:

[1580] Data reception and storage

[1581] Specific behavior:

[1582] The server receives the data sent from the terminal and stores it in a database. The received data is converted into an analysis format.

[1583] input:

[1584] A dataset of location information, movement information, and emotional information sent from the device.

[1585] output:

[1586] Datasets for analysis stored in a database.

[1587] Step 4:

[1588] Data analysis

[1589] Specific behavior:

[1590] The server analyzes the user's behavioral patterns and emotional state based on location, movement, and emotional information stored in the database, and uses a generative AI model to evaluate the user's psychological state and suggest actions.

[1591] input:

[1592] A dataset for analyzing location information, movement information, and emotion information stored in a database.

[1593] output:

[1594] Analysis results (e.g., user is stressed, in the kitchen, waiting for food delivery).

[1595] Step 5:

[1596] Action suggestion generation

[1597] Specific behavior:

[1598] Based on the results of the data analysis, the server generates optimal actions to suggest to the user, such as "take deep breaths" or "listen to music to relax" to reduce stress.

[1599] input:

[1600] Analysis results (e.g., user is stressed, in the kitchen, waiting for food delivery).

[1601] output:

[1602] Specific suggestions for action (e.g., "Your food delivery will arrive in 20 minutes. Take a deep breath.").

[1603] Step 6:

[1604] Proposal Notification

[1605] Specific behavior:

[1606] The server then creates notification data based on the generated action suggestions and sends it to the device, which then displays the notification data to the user in real time.

[1607] input:

[1608] Action suggestions (e.g., "Your food delivery will arrive in 20 minutes. Take a deep breath.").

[1609] output:

[1610] Suggestions notified to users.

[1611] Step 7:

[1612] User execution

[1613] Specific behavior:

[1614] The user receives a notification from the device and performs a suggested action, such as "take a deep breath" or "stretch."

[1615] input:

[1616] Notified suggestions (e.g., "Your food delivery will arrive in 20 minutes. Take a deep breath.").

[1617] output:

[1618] The action taken (e.g., the user took a deep breath).

[1619] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1620] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1621] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1622] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1623] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1624] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1625] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1626] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1627] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1628] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1629] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1630] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1631] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1632] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1633] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1634] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1635] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1636] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1637] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1638] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1639] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1640] The following is further disclosed regarding the above embodiment.

[1641] (Claim 1)

[1642] A means for collecting user location information and movement information;

[1643] means for analyzing the collected location information and movement line information;

[1644] A means for proposing the next action that is optimal for the user based on the analysis results;

[1645] A system including:

[1646] (Claim 2)

[1647] The system according to claim 1, further comprising means for recording a user's stop time in addition to the collected location information and movement line information.

[1648] (Claim 3)

[1649] 2. The system according to claim 1, wherein the suggesting means notifies the user in real time of the optimal next action based on the collected location information, movement line information, and stop time.

[1650] "Example 1"

[1651] (Claim 1)

[1652] A means for collecting user location information and movement information;

[1653] means for transmitting the collected location information and movement line information to a central processing unit at regular intervals;

[1654] a means for storing the location information and movement line information collected by the central processing unit, converting the information into an analytical format, and analyzing the information;

[1655] a means for selecting an optimal next action for the user based on the analysis result and generating notification data;

[1656] means for transmitting the generated notification data to a user's information terminal and displaying the data to the user in real time;

[1657] A system including:

[1658] (Claim 2)

[1659] The system according to claim 1, further comprising means for recording a user's stop time in addition to the collected location information and movement line information.

[1660] (Claim 3)

[1661] 2. The system according to claim 1, wherein the suggesting means notifies the user in real time of the optimal next action based on the collected location information, movement line information, and stop time.

[1662] "Application Example 1"

[1663] (Claim 1)

[1664] A means of collecting user location information, movement information, and stop time,

[1665] A means for analyzing the collected location information, flow line information, and stop time;

[1666] A means for proposing the next action that is optimal for the user based on the analysis results;

[1667] means for proposing an efficient delivery route based on the analysis results;

[1668] A way to notify users of the best actions and break times in real time,

[1669] A system including:

[1670] (Claim 2)

[1671] 2. The system according to claim 1, wherein the suggesting means notifies the user in real time of the optimal next delivery destination and rest time based on the collected location information, traffic flow information, and stop time.

[1672] (Claim 3)

[1673] The system of claim 1, wherein the suggesting means uses a generative AI model to generate the optimal next action or break time based on the user's current state using prompt sentences and notify the user in real time.

[1674] "Example 2: Combining Emotion Engines"

[1675] (Claim 1)

[1676] A means for collecting user location information, movement information, and emotion information;

[1677] means for analyzing the collected location information, movement line information, and emotion information;

[1678] A means for proposing the next action that is optimal for the user based on the analysis results;

[1679] means for notifying a user of said proposed action;

[1680] means for evaluating user behavior according to the suggested actions;

[1681] A system including:

[1682] (Claim 2)

[1683] The system according to claim 1, further comprising means for recording a user's stop time in addition to the collected location information, movement line information and emotion information.

[1684] (Claim 3)

[1685] 2. The system according to claim 1, wherein the suggesting means notifies the user in real time of the optimal next action based on the collected location information, movement information, emotional information, and stop time.

[1686] "Application example 2 when combining emotion engines"

[1687] (Claim 1)

[1688] A means for collecting user location information and movement information;

[1689] means for analyzing the collected location information, movement line information, and emotion information;

[1690] A means for proposing the next action that is optimal for the user based on the analysis results;

[1691] A means to notify users in real time about the best course of action while waiting for food delivery,

[1692] A system including:

[1693] (Claim 2)

[1694] The system according to claim 1, further comprising means for recording a user's stop time in addition to the collected location information and movement line information.

[1695] (Claim 3)

[1696] 2. The system according to claim 1, wherein the suggesting means notifies the user in real time of the optimal next action based on the collected location information, movement line information, stop time and emotion information. [Explanation of symbols]

[1697] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for collecting user location information and movement information; means for analyzing the collected location information and movement line information; A means for proposing the next action that is optimal for the user based on the analysis results; A system including:

2. The system according to claim 1 , further comprising means for recording a user's stop time in addition to the collected location information and movement line information.

3. The system according to claim 1 , wherein the suggesting means notifies the user in real time of the optimal next action based on the collected location information, flow line information, and stop time.

Images