system

Abstract

Provide a system. [Solution] A means of collecting activity data, such as user location, app usage history, calendar events, and sleep data; means for transmitting the activity data to a server; A means for the generation AI to learn the user's behavioral patterns based on the activity data; means for suggesting a next action to be taken based on the behavior pattern; means for notifying a user terminal of the proposed action; means for reflecting the user's selection in the learning; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many small decisions are made in succession in daily life, causing psychological strain and "decision fatigue." This "decision fatigue" reduces quality of life and causes stress. While some leaders of well-known companies reduce the burden of decision-making by wearing the same clothes every day, for many people, doing the same thing every day can actually be stressful. Therefore, a system is needed to suggest actions optimized for each user and reduce the burden of daily decision-making. [Means for solving the problem]

[0005] The present invention solves the above problems by the following means.

[0006] The system provides a means for collecting activity data such as the user's location, app usage history, calendar events, and sleep data, and periodically sends this data to a server. The server uses the received data to learn the user's behavioral patterns using a generation AI. This generation AI is trained using a deep learning model. It is equipped with a means for suggesting the next action to be taken based on the learned behavioral patterns, and notifies the user of the suggested action on their device. The user's device also has a mechanism for visually displaying suggested options using the camera function, allowing the user to select by tapping. Furthermore, by providing a means for reflecting the user's selections in the generation AI's learning, the accuracy of the suggestions is continuously improved.

[0007] "User location information" is data indicating where the user is currently located and where the user has visited in the past.

[0008] "App usage history" is data that includes records of the applications that a user uses on a device such as a smartphone or tablet, as well as the amount of time they spend using them.

[0009] "Calendar events" are data that include information about schedules and events that a user has registered in a calendar application.

[0010] "Sleep data" refers to data that records information related to the user's sleep time and sleep quality.

[0011] "Activity data" is a collective term for various information related to a user's lifestyle and behavior (such as location information, app usage history, calendar events, and sleep data).

[0012] "Generative AI" is an artificial intelligence technology that learns from large amounts of data and performs pattern recognition and predictions.

[0013] A "deep learning model" is a type of machine learning that uses neural networks, has a multi-layer network structure, and is capable of learning complex data patterns.

[0014] "Behavior patterns" are data that indicate the tendencies and habits of behaviors that a user engages in in daily life.

[0015] "Action suggestions" are suggestions that show the user specific actions to take next based on the behavioral patterns learned by the generative AI.

[0016] "Device" refers to an electronic device used by a user, such as a smartphone, tablet, or PC.

[0017] The "camera function" is a function for taking pictures and videos using the camera installed on the device. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention is an action suggestion system for reducing "decision fatigue" in a user's daily life, and is implemented by the following means.

[0040] Overall system configuration

[0041] 1. Data Collection Module

[0042] Device: Your smartphone collects activity data, such as location, app usage, calendar events, and sleep data. This data is obtained from sensors and application APIs.

[0043] Example: Imagine a user commuting to work and turning on the GPS function on their smartphone, which collects location information every minute. It also records the launch time and usage time of each application they are using, as well as calendar event information.

[0044] 2. Data transmission module

[0045] On your device: We encrypt the activity data we collect and send it to our servers via a secure communication protocol (e.g., HTTPS).

[0046] Example: The acquired data is uploaded to a server in batch format at regular intervals (e.g., once an hour).

[0047] 3. Data receiving and storage module

[0048] Server: Receives data sent from the device and stores it in a database for each user. If necessary, it cleans the data (fills in missing values, corrects outliers, etc.).

[0049] Example: Data received on the server side is organized and saved in folders for each user. For example, user A's location data is stored as a dedicated table in the database.

[0050] 4. Data Learning Module

[0051] Server: Inputs the received data into the Generative AI, which learns the user's behavioral patterns. This Generative AI is trained using a deep learning model.

[0052] Example: Using location data and app usage history from the past month as training data, the app learns users' travel patterns and app usage trends.

[0053] 5. Action Proposal Generation Module

[0054] Server: Based on the learned data, predicts the next action to be taken and generates action suggestions. The generated suggestions are customized based on the user's preferences and past behavioral history.

[0055] Example: The server suggests "after-work dinner at a popular restaurant" on Friday evening, based on data about restaurants the user has frequently visited in the past.

[0056] 6. Action Suggestion Notification Module

[0057] Server: Sends customized suggestions to the user's smartphone app.

[0058] On the device: The smartphone app opens and notifies the user of the suggested action.

[0059] Example: Using the smartphone's push notification function, you can send a message to the user saying, "You can refresh yourself by jogging in a nearby park this Saturday."

[0060] 7. Choice Feedback Module

[0061] On the device: Presents specific options for suggested actions (e.g., photos of recipes or a list of activities) using a camera or touch interface.

[0062] User: Selects from the presented options by tapping, etc. The selected content is sent to the server and reflected in the learning of the generation AI.

[0063] Example: A user selects the suggested action "Go to a popular pasta restaurant" from the camera screen, and the selection is sent to the server.

[0064] Through the above-described embodiments, the present invention can reduce the decision-making burden in users' daily lives and prevent "decision fatigue." In particular, suggestions based on learned behavioral patterns are more tailored to individual users and provide higher accuracy and satisfaction than conventional general suggestion systems.

[0065] The processing flow will be explained below.

[0066] Step 1:

[0067] Activity Data Collection

[0068] Device: The user's smartphone collects location information, app usage history, calendar events, and sleep data, which are periodically retrieved from sensors and application APIs.

[0069] Specific operations: Turn on the GPS function to obtain location information, record the launch time and usage time of applications in use, extract schedule and event information from the calendar API, and obtain sleep data from a sleep tracker app.

[0070] Step 2:

[0071] Sending data to the server

[0072] On your device: Collected activity data is encrypted and sent to our servers using a secure communication protocol (e.g., HTTPS).

[0073] Specific behavior: Periodically checks the activity data stored in the local database, uploads any unsent data to the server in batches, encrypts the data using an encryption library, and sends an HTTP request.

[0074] Step 3:

[0075] Receiving and storing data

[0076] Server: Receives data sent from the device and stores it in a database for each user.

[0077] Specific operations: Receive data via API endpoint, organize the data by user ID, and save it in a database. Detect outliers and handle missing values ​​to check data integrity.

[0078] Step 4:

[0079] Data Preprocessing

[0080] Server: Cleans the received data (fills in missing values, handles outliers, etc.) and converts it into a format suitable for generative AI.

[0081] What it does: Normalize the data, sort it by time, and perform feature engineering if necessary, e.g., convert location information into geographic features and add categorical features based on time of day.

[0082] Step 5:

[0083] Generative AI training

[0084] Server: Trains the generative AI using preprocessed data to learn user behavior patterns.

[0085] Specific operation: Preprocessed data is input into the deep learning model, and it learns past behavioral patterns. During the training process, an appropriate learning rate and number of epochs are set, and adjustments are made while evaluating the model's performance.

[0086] Step 6:

[0087] Generate action suggestions

[0088] Server: Using the trained model, predict the user's next action in real time and generate action suggestions.

[0089] Specific Actions: New data points (e.g., current location or recent app usage history) are fed into the model to predict the next action, generating specific suggestions such as "get a coffee at a nearby cafe" or "relax at home."

[0090] Step 7:

[0091] Customize suggestions

[0092] Server: Customizes suggested actions based on the user's preferences and past behavior.

[0093] What it does: It applies personalization algorithms to generate recommendations that take into account the user's preferred activities and dining options. For example, it references a list of restaurants the user has visited in the past and makes new recommendations from there.

[0094] Step 8:

[0095] Submit your proposal

[0096] Server: Sends customized suggestions to the user's smartphone app.

[0097] Specific action: Using the push notification system, the suggested action is sent to the user's device in real time. For example, a notification message saying, "You should go see a movie at this theater right now" is sent.

[0098] Step 9:

[0099] View Suggestions

[0100] On the device: Display received action suggestions visually to the user, for example in the notification area or on an in-app dashboard.

[0101] What it does: Receive a push notification and display the suggestion in the app's UI, such as "Enjoy dinner at a popular restaurant nearby," in an easily accessible format for the user.

[0102] Step 10:

[0103] Presenting options

[0104] Device: Presents specific options for suggested actions (e.g., photos of dishes or a list of activities) and utilizes a camera or touch-enabled interface.

[0105] Specific operation: The camera function is used to scan the surrounding environment, and the presented options are displayed as images. The user taps to confirm the option.

[0106] Step 11:

[0107] Reflecting your selection

[0108] Device: The user's selection is sent to the server and reflected in future behavior predictions.

[0109] Specific actions: The user's selected actions are recorded on the device and sent to the server for the next learning cycle. The server then incorporates the received selections into the training data of the generative AI to improve the model's performance.

[0110] Example 1

[0111] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0112] Many modern users feel fatigued by the numerous choices and decisions they have to make in their daily lives, and are facing what is known as "decision fatigue." This "decision fatigue" reduces the quality of their daily lives and increases stress and dissatisfaction. This has led to a demand for systems that can provide personalized action suggestions and decision support for individual users. However, existing systems are unable to fully learn users' behavioral patterns, making it difficult to provide highly accurate suggestions.

[0113] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0114] In this invention, the server includes means for collecting activity data such as user location information, application usage history, calendar events, and sleep data, means for encrypting the activity data and transmitting it to the server via a secure communication protocol, means for receiving the activity data, storing it in a database for each user, and cleaning the data, means for inputting the activity data into a generative AI model and learning the user's behavioral patterns using a deep learning model, and means for suggesting the next action to be taken based on the learned behavioral patterns, thereby enabling highly accurate action suggestions based on the user's behavioral history.

[0115] "Location information" refers to data that indicates the user's current location and the route they are traveling.

[0116] "Application usage history" refers to data that records usage information of applications used by users on smartphones and other devices.

[0117] "Calendar events" refers to data that records a user's schedules and events on a calendar.

[0118] "Sleep data" refers to data that records the user's sleep time and sleep quality.

[0119] "Activity data" is a general term for data related to a user's daily activities, such as location information, application usage history, calendar events, and sleep data.

[0120] "Encryption" is a technology that converts data to transmit information securely and prevents unauthorized access and data leaks.

[0121] A "secure communication protocol" is a communication protocol for securely sending and receiving data, and a specific example is HTTPS.

[0122] "Data cleaning" is a process of improving data quality by filling in missing values ​​and correcting outliers.

[0123] A "generative AI model" is an artificial intelligence model that generates new information and suggestions based on data.

[0124] A "deep learning model" is a machine learning model that uses a multi-layer neural network to perform advanced learning and prediction.

[0125] A "behavioral pattern" is a series of data that indicates a user's daily behavior and tendencies.

[0126] "Action suggestions" are information or ideas that recommend the next action the user should take.

[0127] "Terminal" refers to a device that is directly operated by a user, such as a smartphone or tablet.

[0128] "Choices" are any list or item that allows a user to choose from multiple suggestions or options.

[0129] Overall system configuration

[0130] The present invention provides an action suggestion system for reducing "decision fatigue" in a user's daily life. This system is implemented by the following components and means.

[0131] Data Collection Module

[0132] Device: Your smartphone collects location information, application usage history, calendar events, sleep data, and other activity data. This is done using the smartphone's GPS, various sensors, and application APIs.

[0133] Example: When a user commutes to work, the GPS function on their smartphone is turned on, and location information is collected every minute. In addition, the launch time and usage time of each application being used are collected through the application API, as are events registered in the calendar.

[0134] Data Transmission Module

[0135] On your device: Collected activity data is encrypted and sent to a server via a secure communication protocol (e.g., HTTPS).

[0136] Example: The collected data is encrypted using the AES encryption algorithm and then sent to the server via HTTPS. The data is uploaded to the server in batches, for example, once an hour.

[0137] Data reception and storage module

[0138] Server: Receives data sent from the device and stores it in a database for each user. It also cleans the data as needed (filling in missing values, correcting outliers, etc.).

[0139] Example: Data received on the server side is organized and saved in a dedicated folder for each user. Before saving to the database, outliers are detected and corrected or deleted, and the data is saved as clean data. For example, the location information data of user A is stored in a dedicated table in the database.

[0140] Data Learning Module

[0141] Server: Inputs the received data into a generative AI model to learn user behavior patterns. This generative AI model is trained using a deep learning model.

[0142] Example: Location data and app usage history from the past month are input into a deep learning model as training data to learn the user's movement patterns and app usage trends. Specifically, CNN (convolutional neural network) and LSTM (long short-term memory) models are used.

[0143] Action proposal generation module

[0144] Server: Based on the learned data, the server proposes the next action to be taken. The proposed actions are customized based on the user's preferences and past behavioral history.

[0145] Example: For example, the server suggests "have dinner at a popular restaurant after work" on Friday evening. This suggestion is generated based on the data of restaurants the user has frequently visited in the past. Also, the server selects vegetarian restaurants as the user's preference.

[0146] Action Suggestion Notification Module

[0147] Server: Sends customized suggestions to the user's smartphone app.

[0148] On the device: The smartphone app opens and notifies the user of the suggested action.

[0149] Example: A smartphone app that receives a suggestion message from a server uses the push notification function to display a message to the user saying, "You can refresh yourself by jogging in a nearby park this Saturday."

[0150] Choice Feedback Module

[0151] On the device: Presents specific options for suggested actions (e.g., photos of recipes or a list of activities) using a camera or touch interface.

[0152] User: Selects from the presented options by tapping, etc. The selected content is sent to the server and reflected in the learning of the generation AI.

[0153] Example: A user selects a suggested action, such as "Go to a popular pasta restaurant," on the camera screen, and this information is sent to the server in real time, allowing the generative AI model to learn new data and provide more personalized recommendations the next time.

[0154] Examples of prompts for this system include "Predict the user's next move based on location data from the past month" and "Suggest activities for this weekend, taking into account the user's preferences."

[0155] Through these modules and processes, the present invention can significantly reduce the burden of decision-making in users' daily lives and effectively prevent "decision fatigue." By providing personalized recommendations, it increases satisfaction and achieves higher accuracy than conventional general recommendation systems.

[0156] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0157] Step 1:

[0158] Data collection

[0159] Device: Collects activity data such as user location, application usage history, calendar events, and sleep data. This data collection is done using the smartphone's GPS function, various sensors, and application APIs.

[0160] Input: Raw data collected from sensors on the user's smartphone and application APIs (location, application usage history, calendar events, sleep data).

[0161] Processing: Collecting data from sensors and APIs and organizing it into a specific format as activity data.

[0162] Output: Organized activity data.

[0163] What it does: When a user commutes to work, the GPS function on their smartphone is turned on, and location information is collected every minute, along with application launch times, usage time, and calendar events.

[0164] Step 2:

[0165] Data transmission

[0166] On your device: Collected activity data is encrypted and sent to a server via a secure communication protocol (e.g., HTTPS).

[0167] Input: Organized activity data.

[0168] Processing: The organized activity data is encrypted using the AES encryption algorithm and sent to the server via HTTPS.

[0169] Output: The encrypted data is sent to the server.

[0170] Specific operation: The collected data is periodically (for example, once an hour) encrypted in batch format and uploaded to the server via HTTPS communication.

[0171] Step 3:

[0172] Data reception and storage

[0173] Server: Receives data sent from the device and stores it in a database for each user. It also cleans the data (fills in missing values, corrects outliers, etc.).

[0174] Input: Encrypted activity data.

[0175] Processing: The encrypted data is decrypted, organized by user, and stored in a database. Data cleaning is performed.

[0176] Output: Cleaned per-user activity data.

[0177] Specific operations: Decrypts encrypted data on the server side and stores it in a folder for each user in the database. Detects and imputes outliers, and fills in missing values ​​to improve data quality.

[0178] Step 4:

[0179] Data Learning

[0180] Server: Inputs the received data into a generative AI model to learn user behavior patterns. This generative AI model is trained using a deep learning model.

[0181] Input: Cleaned per-user activity data.

[0182] Processing: The cleaned data is fed into a deep learning model to learn user behavior patterns.

[0183] Output: Learned behavioral pattern model.

[0184] How it works: The system inputs location data and application usage history from the past month into a deep learning model to learn the user's movement patterns and app usage trends, using deep learning techniques such as CNN and LSTM.

[0185] Step 5:

[0186] Action suggestion generation

[0187] Server: Based on the learned data, the server proposes the next action to be taken. The proposed actions are customized based on the user's preferences and past behavioral history.

[0188] Input: Learned behavioral pattern model.

[0189] Processing: Based on the learned model, next action suggestions are generated, which are customized based on the user's preferences and past behavioral history.

[0190] Output: Customized action suggestions.

[0191] Specific behavior: For example, the server may suggest "have dinner at a popular restaurant after work" on a Friday evening. This suggestion is generated based on data on restaurants the user has frequently visited in the past.

[0192] Step 6:

[0193] Action suggestion notification

[0194] Server: Sends customized suggestions to the user's smartphone app.

[0195] On the device: The smartphone app notifies the user of the suggested action.

[0196] Input: customized action suggestions.

[0197] Processing: Sends the proposed content to the smartphone app and notifies it.

[0198] Output: The user is notified of the suggestion.

[0199] Specific operation: The smartphone app receives a proposal message from the server and uses the push notification function to notify the user of the proposal.

[0200] Step 7:

[0201] Selection Feedback

[0202] On the device: Present specific options for the suggested action (e.g., photos of recipes or a list of activities) using a camera or touch interface.

[0203] User: Selects from the presented options by tapping, etc. The selected content is sent to the server and reflected in the learning of the generation AI.

[0204] Input: The user's selection.

[0205] Processing: The user's selection is sent to the server and reflected in the generative AI model.

[0206] Output: The updated training model.

[0207] What happens: The user taps to select the suggested action "Go to a popular pasta restaurant," which is then sent to the server, allowing the generative AI model to learn new data and provide more personalized recommendations the next time.

[0208] (Application example 1)

[0209] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0210] While modern users have access to a wide range of entertainment content, they often experience "decision fatigue" when trying to choose the best content from the diverse options. Under these circumstances, there is a need for systems that can appropriately suggest the next content to watch based on the user's preferences and behavioral history. Current systems are unable to effectively utilize the content and activity data of users' past viewings, making it difficult to make optimal suggestions for each user.

[0211] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0212] In this invention, the server includes means for collecting activity data such as a user's location information, app usage history, calendar events, and sleep data, means for transmitting the activity data to the server via a secure communication protocol, means for a generation AI to learn the user's behavioral patterns using a deep learning model based on the activity data, means for suggesting entertainment content to be viewed next based on the behavioral patterns, means for notifying the user's smartphone of the suggested content, and means for reflecting the user's selection in the learning, thereby enabling users to easily select entertainment content appropriately customized based on their preferences and behavioral history.

[0213] "Location information" is data that indicates a user's current location and is obtained using GPS or other location information technology.

[0214] "App usage history" is a record of applications used by a user on a smartphone or other device, and includes data such as the time the app was launched and the amount of time it was used.

[0215] "Calendar events" are information about events and schedules that a user registers in a calendar to manage their plans and appointments.

[0216] "Sleep data" refers to data that indicates the user's sleep time and quality, and is collected via smartphones or wearable devices.

[0217] "Activity data" is data that records various actions and conditions of a user's daily life, and includes location information, app usage history, calendar events, sleep data, and more.

[0218] A "secure communication protocol" is a safe means of communication that prevents eavesdropping or tampering by third parties when sending and receiving data, and examples include protocols such as HTTPS.

[0219] "Generative AI" is artificial intelligence that uses machine learning techniques such as deep learning to train user data to perform specific tasks.

[0220] A "deep learning model" is a type of artificial intelligence that uses multi-layered neural networks to learn complex patterns from data.

[0221] A "behavioral pattern" indicates the tendencies and habits of the user's daily behavior, and is determined based on past data.

[0222] "Entertainment content" refers to media such as movies, dramas, music, podcasts, etc. that users consume for entertainment purposes.

[0223] A "smartphone" is a multi-function mobile phone that is a device that can not only make calls but also access the Internet and use applications.

[0224] "Push notification" is a function that notifies information to smartphones and other devices in real time.

[0225] A "selection" refers to the particular item a user chooses from among the options offered.

[0226] "Means for reflecting in learning" refers to a method for feeding back user choices to the generative AI model to improve the accuracy of subsequent suggestions.

[0227] The present invention is an action suggestion system for reducing "decision fatigue" in a user's daily life, and specific implementation means are described below.

[0228] Data Collection Module

[0229] The device used is the user's smartphone, which collects location information, app usage history, calendar events, sleep data, and other activity data. This data is obtained through the smartphone's built-in sensors and application APIs.

[0230] Data Transmission Module

[0231] Activity data collected on the device is sent to a server using a secure communication protocol (e.g., HTTPS). The data is encrypted and uploaded in batches at regular intervals (e.g., once an hour).

[0232] Data reception and storage module

[0233] The server side receives the transmitted data and stores it in a database for each user. Data cleansing (filling in missing values, correcting outliers, etc.) is also performed. For example, SQLite is used as the database.

[0234] Data Learning Module

[0235] Based on the data received on the server, the generated AI uses a deep learning model to learn the user's behavioral patterns. To do this, it uses deep learning libraries such as TENSORFLOW (registered trademark). It uses past location data and app usage history as training data to learn the user's patterns and tendencies.

[0236] Action proposal generation module

[0237] The server uses the learned data to predict the next entertainment content to watch and generates action suggestions, which are customized based on the user's preferences and past behavioral history.

[0238] Action Suggestion Notification Module

[0239] The generated action suggestions are sent to the user's smartphone using the push notification function, and the notification content includes specific entertainment content such as "the next movie to watch."

[0240] Choice Feedback Module

[0241] The user taps to select an option from the suggested content. This selection is sent to the server and reflected in the learning of the generative AI. The suggested options can also be visually displayed using the smartphone's camera function.

[0242] Specific examples

[0243] For example, at 8 p.m. on a weekday while a user is relaxing at home, a push notification on their smartphone suggests a movie to watch next based on their past viewing history and activity data. This suggestion specifically suggests movies in a genre that the user previously enjoyed.

[0244] Example prompt sentence:

[0245] "Below is your past viewing history and recent activity data. Based on this data, we'd like to suggest what you should watch next."

[0246] Viewing history: [Movie A, Drama B, Music C...]

[0247] Latest activity data: {Location: Home, Time: 20:00, Date: 2023-10-02}

[0248] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0249] Step 1:

[0250] The device (user's smartphone) collects activity data such as location information, app usage history, calendar events, and sleep data.

[0251] Input: Data obtained through smartphone sensors and APIs.

[0252] Specific operation: Using the smartphone's GPS, it collects location information every minute and also periodically records app usage history and calendar events.

[0253] Output: Collected activity data (location, app usage history, etc.).

[0254] Step 2:

[0255] The device transmits the collected activity data to the server via a secure communication protocol (e.g., HTTPS).

[0256] Input: Collected activity data.

[0257] What it does: It encrypts your activity data and uploads it to a server in batches using the HTTPS protocol, for example, every hour.

[0258] Output: Activity data sent to the server.

[0259] Step 3:

[0260] The server receives the data and stores it in a database. It also performs data cleansing, such as filling in missing values ​​and correcting outliers.

[0261] Input: Submitted activity data.

[0262] Specific operation: The received data is organized and stored in a database for each user, outliers are detected and corrected, and missing values ​​are complemented.

[0263] Output: Saved and cleansed activity data.

[0264] Step 4:

[0265] The server uses generative AI (deep learning model) to learn the user's behavioral patterns based on the stored activity data.

[0266] Input: Saved activity data.

[0267] Specific operation: Using a deep learning library (such as TensorFlow), the model is trained using past location information and app usage history as training data. The following is an example of a prompt sentence to input.

[0268] Example prompt: "Below is the user's past viewing history and latest activity data. Based on this data, please suggest the next content they should watch." Viewing history: [Movie A, Drama B, Music C...] Latest activity data: {Location: Home, Time: 20:00, Date: 2023-10-02}

[0269] Output: An AI model that has learned the user's behavioral patterns.

[0270] Step 5:

[0271] The server uses the learned AI model to predict the next entertainment content to watch and generate action suggestions.

[0272] Input: An AI model that has learned your behavioral patterns, and your latest activity data.

[0273] What it does: It uses AI models to predict the best entertainment content for users based on their current situation and past behavioral patterns.

[0274] Output: Suggestions about what entertainment content to watch next.

[0275] Step 6:

[0276] The server sends the generated action suggestions to the smartphone via push notification.

[0277] Input: Suggestions for entertainment content to watch next.

[0278] Specific operation: Using a push notification library (e.g., Firebase Cloud Messaging), the suggestion content is sent to the user's smartphone.

[0279] Output: Action suggestions displayed on the user's smartphone.

[0280] Step 7:

[0281] Users tap to select from the suggested entertainment content, and the selected content is sent to the server and reflected in the learning of the generative AI.

[0282] Input: User-selected entertainment content.

[0283] Specific operation: Using the smartphone's camera function and touch operation interface, the proposed options are visually displayed and the user's tap selection information is collected. The selection information is then sent back to the server and used to update the generative AI model.

[0284] Output: The user's choices fed back to the generative AI model.

[0285] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0286] The present invention is an action suggestion system for reducing "decision fatigue" in a user's daily life, and further improves the accuracy and satisfaction of the suggestions by incorporating an emotion engine that recognizes and responds to the user's emotional state. The present invention is implemented by the following means.

[0287] Overall system configuration

[0288] 1. Data Collection Module

[0289] Device: Your smartphone collects activity data, such as location, app usage history, calendar events, and sleep data. This data is periodically retrieved from sensors and application APIs.

[0290] Example: A user turns on the GPS on their smartphone during their commute to collect location information, record app usage history, calendar events, and data from a sleep tracker.

[0291] 2. Data transmission module

[0292] On your device: Collected activity data is encrypted and sent to a server via a secure communication protocol.

[0293] Example: The acquired data is uploaded to the server in batches every hour.

[0294] 3. Data receiving and storage module

[0295] Server: Stores the received data in a database for each user, and cleans the data if necessary.

[0296] Example: Organize data by user ID on the server side, process outliers and missing values, and then save it to the database.

[0297] 4. Data Preprocessing Module

[0298] Server: Cleans the received data and converts it into a format suitable for the generative AI.

[0299] Examples: data normalization, timestamp alignment, feature engineering.

[0300] 5. Generative AI Learning Module

[0301] Server: Trains the generative AI using preprocessed data to learn user behavior patterns.

[0302] Example: Using past location information and app usage history as training data to recognize users' travel patterns and app usage tendencies.

[0303] 6. Action Proposal Generation Module

[0304] Server: Uses the trained model to predict the user's next action in real time and generate action suggestions.

[0305] Example: Feed new data points into the model to generate specific suggestions, such as "have a coffee at a nearby cafe."

[0306] 7. Proposal customization module

[0307] Server: Customizes suggested actions based on the user's preferences and past behavior.

[0308] Example: Referencing data on restaurants the user has visited in the past and suggesting "dinner at a popular restaurant after work."

[0309] 8. Action Suggestion Notification Module

[0310] Server: Sends customized suggestions to the user's smartphone app.

[0311] Terminal: Visually notify the user of the proposed action.

[0312] Example: Use a push notification to say, "You can refresh yourself by jogging in a nearby park this Saturday."

[0313] 9. Choice Feedback Module

[0314] Device: Presents specific options for suggested actions using a camera or touch interface.

[0315] User: Tap to select a presented option. The selection is sent to the server and reflected in the learning of the generation AI.

[0316] Example: A user selects the option "Go to a popular pasta restaurant" from a camera screen, and the selection is sent to a server.

[0317] 10. Emotion Engine Module

[0318] Device: Recognizes the user's emotional state by analyzing facial expressions, voice, and text messages.

[0319] Example: Using a smartphone's camera and microphone, it analyzes emotions from the user's facial expressions and tone of voice, as well as the content of text messages.

[0320] 11. Emotion-based behavior suggestion adjustment module

[0321] Server: Adjusts the recommendations based on the results from the emotion engine.

[0322] Example: If the user is feeling stressed, suggest a relaxing activity (e.g., yoga or meditation).

[0323] Through the above-described embodiments, the present invention reduces the burden of decision-making in the user's daily life, and furthermore, by using an emotion engine, it is possible to realize optimal action suggestions that take into account the user's mental state, thereby providing higher accuracy and satisfaction than conventional systems.

[0324] The processing flow will be explained below.

[0325] Step 1:

[0326] Activity Data Collection

[0327] Device: Your smartphone collects activity data, such as location, app usage history, calendar events, and sleep data. This data is periodically retrieved from sensors and application APIs.

[0328] Specific operations: Turn on the GPS function to obtain location information, record the launch time and usage time of each application in use, extract schedule and event information from the calendar API, and obtain sleep data from a sleep tracker app.

[0329] Step 2:

[0330] Sending data to the server

[0331] On your device: We encrypt the activity data we collect and send it to our servers using a secure communication protocol (e.g., HTTPS).

[0332] Specific behavior: Periodically checks the activity data stored in the local database, uploads any unsent data to the server in batches, encrypts the data using an encryption library, and sends an HTTP request.

[0333] Step 3:

[0334] Receiving and storing data

[0335] Server: Receives data sent from the device and stores it in a database for each user.

[0336] Specific operations: Receive data via API endpoint, organize the data by user ID, and save it in a database. Detect outliers and handle missing values ​​to check data integrity.

[0337] Step 4:

[0338] Data Preprocessing

[0339] Server: Cleans the received data (fills in missing values, handles outliers, etc.) and converts it into a format suitable for generative AI.

[0340] What it does: Normalizes the data, aligns it on the time axis, and performs feature engineering as needed, e.g., converting location information into geographic features and adding categorical features based on time of day.

[0341] Step 5:

[0342] Generative AI training

[0343] Server: Trains the generative AI using preprocessed data to learn user behavior patterns.

[0344] Specific operation: Preprocessed data is input into the deep learning model, and it learns past behavioral patterns. During the training process, an appropriate learning rate and number of epochs are set, and adjustments are made while evaluating the model's performance.

[0345] Step 6:

[0346] Generate action suggestions

[0347] Server: Using the trained model, predict the user's next action in real time and generate action suggestions.

[0348] Specific Actions: New data points (e.g., current location or recent app usage history) are fed into the model to predict the next action, generating specific suggestions such as "get a coffee at a nearby cafe" or "relax at home."

[0349] Step 7:

[0350] Customize suggestions

[0351] Server: Customizes suggested actions based on the user's preferences and past behavior.

[0352] What it does: It applies personalization algorithms to generate recommendations that take into account the user's preferred activities and dining options. For example, it references a list of restaurants the user has visited in the past and makes new recommendations from there.

[0353] Step 8:

[0354] Submit your proposal

[0355] Server: Sends customized suggestions to the user's smartphone app.

[0356] Specific action: Using the push notification system, the suggested action is sent to the user's device in real time. For example, a notification message saying, "You should go see a movie at this theater right now" is sent.

[0357] Step 9:

[0358] View Suggestions

[0359] On the device: Display received action suggestions visually to the user, for example in the notification area or on an in-app dashboard.

[0360] What it does: Receive a push notification and display the suggestion in the app's UI, for example, "Enjoy dinner at a popular restaurant nearby," in a format that's easily accessible to the user.

[0361] Step 10:

[0362] Presenting options

[0363] Device: Presents specific options for suggested actions (e.g., photos of dishes or a list of activities) and utilizes a camera or touch-enabled interface.

[0364] Specific operation: The camera function is used to scan the surrounding environment, and the presented options are displayed as images. The user taps to confirm the option.

[0365] Step 11:

[0366] Reflecting your selection

[0367] Device: The user's selection is sent to the server and reflected in future behavior predictions.

[0368] Specific actions: The user's selected actions are recorded on the device and sent to the server for the next learning cycle. The server then incorporates the received selections into the training data of the generative AI to improve the model's performance.

[0369] Step 12:

[0370] Collecting Emotional Data

[0371] Device: Recognizes the user's emotional state by analyzing facial expressions, voice, and text messages.

[0372] Specific operation: Using the smartphone's camera and microphone, the system analyzes the user's facial expressions and tone of voice, and uses natural language processing to analyze the content of text messages to determine their emotional state.

[0373] Step 13:

[0374] Sending emotional data

[0375] Terminal: Sends analyzed emotion data to the server.

[0376] Specific operation: The emotion data output by the emotion recognition engine is encrypted and sent to the server using a secure communication protocol.

[0377] Step 14:

[0378] Tailoring behavioral suggestions based on emotions

[0379] Server: Adjusts the recommendations based on the results from the emotion engine.

[0380] Specific actions: If the user is feeling stressed, it will suggest relaxing actions (e.g., yoga or meditation). Conversely, if the user is feeling excited, it will suggest active actions (e.g., sports or dancing).

[0381] Through the above-described embodiments, the present invention reduces the burden of decision-making in the user's daily life, and furthermore, by using an emotion engine, it is possible to realize optimal action suggestions that take into account the user's mental state, thereby providing higher accuracy and satisfaction than conventional systems.

[0382] Example 2

[0383] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0384] While current technology exists to reduce the burden of decision-making in users' daily lives, these systems are unable to take into account their emotional state when making suggestions, which can result in low satisfaction. To solve this problem, it is necessary to provide a system that recognizes the user's emotional state and suggests optimal actions.

[0385] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting activity data such as the user's location information, app usage history, calendar events, and sleep data, means for encrypting the activity data and transmitting it to the server, means for a generation AI to learn the user's behavioral patterns based on the activity data, means for suggesting a next action to be taken based on the behavioral patterns, means for recognizing the user's emotional state using an emotion engine and adjusting the suggestion content, means for notifying the user's terminal of the suggested action, and means for reflecting the user's selection in the learning. This reduces the burden of decision-making in the user's daily life and enables highly accurate and satisfying action suggestions that take the user's emotional state into consideration.

[0386] "User location information" is geographical data that indicates where the user is currently located.

[0387] "Application usage history" is data that records the names of applications used by a user, the duration of use, frequency of use, and so on.

[0388] A "calendar event" is event information entered into a calendar application that registers a user's plans and schedules.

[0389] "Sleep data" refers to data that records the user's sleep patterns, sleep duration, quality, etc.

[0390] "Activity data" refers to all data related to daily life activities, such as a user's location information, app usage history, calendar events, and sleep data.

[0391] "Encryption" means converting data into a form that is unintelligible to others using a specific algorithm.

[0392] "Generative AI" is an artificial intelligence model that can learn a user's behavioral patterns and suggest the next action to take.

[0393] A "behavioral pattern" refers to a series of actions and habits in a user's daily life.

[0394] The "emotion engine" is an engine that analyzes and identifies the user's emotional state from their facial expressions, voice, and text.

[0395] "Suggestion content" refers to a suggested action that the generation AI generates by taking into account the user's behavioral patterns and emotional state.

[0396] "Notification" refers to displaying the generated action suggestion on the user's terminal.

[0397] "Selection" refers to the item that the user selects from the suggested actions.

[0398] "Reflecting in learning" means incorporating user selections into the generative AI model to improve the accuracy of future action suggestions.

[0399] This invention is a system for reducing the burden of decision-making in a user's daily life and for suggesting optimal actions that take into account the user's emotional state.

[0400] System configuration

[0401] Data Collection Module

[0402] The user's device, specifically their smartphone, is used to collect activity data such as location information, app usage history, calendar events, and sleep data, which are periodically retrieved through the device's sensors and application APIs.

[0403] Example: A user turns on the GPS on their smartphone during their commute to collect location information, record app usage history, calendar events, and data from a sleep tracker.

[0404] Data Transmission Module

[0405] Activity data collected on the device is encrypted and sent to a server via a secure communication protocol (e.g., HTTPS).

[0406] Example: The collected data is AES encrypted in batches every hour and uploaded to the server via HTTPS protocol.

[0407] Data reception and storage module

[0408] The server stores the received data in a database for each user. Before storing, it cleans the data (dealing with outliers and imputing missing values) as necessary.

[0409] Example: The data received by the server is organized into a database by user ID, and outliers and missing values ​​are automatically processed.

[0410] Data Preprocessing Module

[0411] The server cleans the received data and converts it into a format suitable for generative AI, including normalizing the data, aligning timestamps, and extracting features.

[0412] Example: Prepare a training dataset by properly aligning timestamps, normalizing numeric data, and removing unnecessary fields.

[0413] Generative AI learning module

[0414] The server uses the preprocessed data to train a generative AI to learn the user's behavioral patterns. This generative AI is trained using a deep learning framework (e.g., TensorFlow, PyTorch).

[0415] Example: Using past location information and app usage history to learn about users' travel patterns and app usage habits.

[0416] Action proposal generation module

[0417] The server uses a trained generative AI model to predict the user's next action in real time and generate action suggestions.

[0418] Example: Newly collected data points are fed into a generative AI model to generate specific suggestions, such as "It's almost lunchtime, so suggest a nearby cafe."

[0419] Suggestion customization module

[0420] The server customizes the generated suggestions based on the user's preferences and past behavioral history.

[0421] Example: Referencing data on restaurants frequently visited by the user to provide customized suggestions such as "dinner at a popular restaurant after work."

[0422] Action Suggestion Notification Module

[0423] The server sends the customized action suggestions to the user's smartphone app, which then visually notifies the user of the suggested action.

[0424] Example: Use push notifications to notify users of something like, "You can refresh yourself by jogging in a nearby park this Saturday."

[0425] Choice Feedback Module

[0426] The device uses a camera or touch interface to present specific options for the proposed action. The user taps to select the option. The selection is sent to the server and reflected in the learning of the generative AI.

[0427] Example: A user selects the option "Go to a popular pasta restaurant" from a camera screen, and the selection is sent to a server.

[0428] Emotion Engine Module

[0429] The device recognizes the user's emotional state by analyzing their facial expressions, voice, and text messages.

[0430] Example: Using the smartphone camera and microphone, the system analyzes emotions from the user's facial expressions and tone of voice, including text messages.

[0431] Emotion-based behavior suggestion adjustment module

[0432] The server adjusts the suggestions based on the emotional state obtained from the emotion engine.

[0433] Example: If the user is feeling stressed, suggest a relaxing activity (e.g., meditation, yoga).

[0434] Examples of prompts for generative AI models

[0435] "Generate optimal action suggestions based on the user's current location, app usage history, calendar events, and sleep data."

[0436] This invention can reduce the burden of decision-making in the user's daily life and realize highly accurate and satisfying action suggestions that also take into account the user's emotional state.

[0437] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0438] Step 1:

[0439] Data collection

[0440] Input: Activity data about the user's daily life (location, app usage history, calendar events, sleep data).

[0441] Specific operation: The device (smartphone) acquires location information using GPS, records application usage history in the background, obtains the user's schedule from a calendar app, and collects sleep data from a sleep tracker app.

[0442] Output: Collected activity data.

[0443] Step 2:

[0444] Data transmission

[0445] Input: Collected activity data.

[0446] Specific operation: The device encrypts the collected data using AES and sends it to the server using a secure communication protocol (HTTPS).

[0447] Output: The encrypted data is sent to the server.

[0448] Step 3:

[0449] Data reception and storage

[0450] Input: Encrypted data.

[0451] Specific operation: The server receives the data, unlocks it, and then stores it in a database for each user ID. At that time, it cleans the data by removing outliers and filling in missing values.

[0452] Output: An organized database.

[0453] Step 4:

[0454] Data Preprocessing

[0455] Input: A curated database.

[0456] Specific operations: The server normalizes the data, aligns timestamps, removes unnecessary fields, extracts features, and converts it into a format suitable for generative AI.

[0457] Output: The preprocessed dataset.

[0458] Step 5:

[0459] Generative AI training

[0460] Input: The preprocessed dataset.

[0461] How it works: The server trains the generative AI using a deep learning framework (e.g., TensorFlow, PyTorch), learning the dataset over multiple epochs and incorporating user behavior patterns into the model.

[0462] Output: The trained model.

[0463] Step 6:

[0464] Action suggestion generation

[0465] Input: Newly collected data points, trained model.

[0466] What it does: The server uses the trained model to analyze new data points collected and predict the next action to take. It generates suggestions based on prompts (e.g., "Generate optimal action suggestions based on the current user's location, app usage history, calendar events, and sleep data.").

[0467] Output: Proposal.

[0468] Step 7:

[0469] Customize suggestions

[0470] Input: Proposal content, past behavior history.

[0471] Specific operation: The server customizes the generated suggestions based on the user's preferences and past behavior history, taking into account the places visited in the past and preferred activities to provide optimal suggestions.

[0472] Output: A customized proposal.

[0473] Step 8:

[0474] Action suggestion notification

[0475] Input: Your customized suggestion.

[0476] Specific operation: The server uses the push notification API to send customized suggestions to the device, which then visually notifies the user of the suggested action.

[0477] Output: The suggestion that was communicated to the user.

[0478] Step 9:

[0479] Selection Feedback

[0480] Input: User choices, camera and touch interfaces.

[0481] Specific actions: The device visually displays specific options for the proposed action, and the user taps to select it. The user's selection is sent to the server and reflected in the learning of the generative AI.

[0482] Output: The selections sent to the server.

[0483] Step 10:

[0484] Emotion Engine

[0485] Input: User facial expressions, voice, and text messages.

[0486] How it works: The device uses the smartphone's camera and microphone to collect the user's facial expressions and voice, and analyzes emotions from text messages.

[0487] Output: Emotional state data.

[0488] Step 11:

[0489] Emotion-based behavioral adjustment

[0490] Input: Emotional state data, customized suggestions.

[0491] Specific operation: The server further adjusts the suggested actions based on the emotional state obtained from the emotion engine. If the user is feeling stressed, the server changes the suggested actions to suggest relaxing activities.

[0492] Output: Final recommendations based on the user's emotional state.

[0493] (Application example 2)

[0494] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0495] In recent years, the proliferation of options in consumers' daily lives has led to the problem of "decision fatigue." Particularly in brick-and-mortar stores, consumers are forced to make optimal choices from a vast array of products and services, placing a psychological burden on them. Furthermore, because optimal recommendations vary depending on a consumer's emotional state, there is a need for a system that can analyze each individual's emotional state and adjust the recommendations accordingly.

[0496] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting behavioral data such as user location information, app usage history, time information, and sleep data; means for transmitting the behavioral data to the server; means for a generation AI to learn the user's behavioral patterns based on the behavioral data; means for suggesting the next action to be taken based on the behavioral patterns; means for notifying the user's device of the suggested action and making a purchase suggestion; means for reflecting the user's selection in the learning; and means including an emotion engine for collecting the user's emotional state and adjusting the suggestion content. This reduces the decision-making burden in consumers' daily lives and makes it possible to provide personalized suggestions based on each individual's emotional state in real time.

[0497] "User location information" is data that identifies the user's current geographic location.

[0498] "Application usage history" is data on applications that a user has used in the past.

[0499] "Time information" is data relating to the time and date when the user performed an action.

[0500] "Sleep data" is information related to the user's sleep, including the amount of sleep time and the quality of sleep.

[0501] "Behavioral data" is a general term for data including location information, app usage history, time information, sleep data, and the like in a user's daily life.

[0502] "Generative AI" is an artificial intelligence that learns user behavior patterns based on collected data and generates new suggestions.

[0503] "Behavioral patterns" refer to tendencies and habits extracted from a user's past behavior.

[0504] The "means for suggesting an action" is a method for indicating the next action to be taken based on the user's behavioral patterns.

[0505] A "purchase suggestion" is a suggestion that recommends specific products or services related to consumption behavior to a user.

[0506] The "emotion engine" is a function that analyzes the user's emotional state and adjusts the content of suggestions based on that state.

[0507] "Personalized offers" are individual offers tailored to a user's individual interests, preferences, and emotional state.

[0508] "Server" means a computer system that receives, stores, analyzes, and generates recommendations from user-submitted data.

[0509] This invention is an action suggestion system designed to reduce "decision fatigue" in users' daily lives. It also incorporates an emotion engine that recognizes and responds to the user's emotional state, improving the accuracy of the suggestions and satisfaction. This system collects behavioral data, such as the user's location, app usage history, time information, and sleep data, and uses this data to generate AI that learns the user's behavioral patterns. Based on the behavioral patterns, the system then suggests the next action to be taken and notifies the user's device of the suggested action.

[0510] First, user behavior data is collected from devices such as smartphones. Specifically, this includes location information using GPS, application usage history, calendar events, and data from sleep trackers. This data is collected in real time or periodically and sent to a server. The data is encrypted and transmitted via a secure communication protocol.

[0511] The server stores the received behavioral data in a database, cleans the data, and then performs preprocessing to train the generative AI model. Specifically, it normalizes the data, aligns timestamps, and performs feature engineering. The preprocessed data is then used to train the generative AI, which learns the user's behavioral patterns. This AI model uses deep learning technology.

[0512] The action suggestion generation module uses a trained generative AI model to predict the next action a user should take in real time based on their behavioral patterns, generating action suggestions. For example, if the user is in a physical store, suggestions such as "take a break at a coffee shop" or "check for new product sales" will be made. These suggestions are further refined by an emotion engine that analyzes the user's emotional state. Specifically, the system analyzes the user's facial expressions and voice using the smartphone's camera and microphone, as well as the content of text messages.

[0513] The proposed actions, including purchasing suggestions, are sent to the user's device. The user selects from the suggested options using touch operations on their smartphone, and the selection is then sent back to the server. This allows the generative AI model to constantly retrain based on the latest data, improving the accuracy of the suggestions.

[0514] For example, if the system detects that the user is feeling stressed, it will suggest "take a break in the relaxation area." If the user is feeling normal, it will suggest "try your favorite pasta restaurant." In this way, personalized suggestions based on the user's individual emotional state can be made in real time.

[0515] Example prompt sentence:

[0516] The user opens their smartphone and types, "Please suggest what I should do next." This prompt allows the server to analyze their latest behavioral patterns and emotional state and notify the user of the optimal suggested action.

[0517] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0518] Step 1:

[0519] The user's device collects behavioral data such as location information, app usage history, time information, and sleep data. This data is obtained from sensors and devices such as the smartphone's GPS, application APIs, calendar functions, and sleep trackers. Input data includes location coordinates, a list of used apps, calendar events, and sleep duration, and this is output as the initial behavioral data.

[0520] Step 2:

[0521] The behavioral data collected by the device is encrypted and sent to the server via a secure communication protocol. The input data is the behavioral data collected in step 1, and is output as transmission data after going through the encryption process. Specifically, the data is uploaded in batches every hour.

[0522] Step 3:

[0523] The server stores the received data in a database for each user. This is where data cleaning is also performed. The input data is the data sent in step 2, and outliers and missing values ​​are processed before being stored in the database. The cleaned data is output.

[0524] Step 4:

[0525] The server cleans the received data and converts it into a format suitable for generative AI. Specifically, it normalizes the data, aligns timestamps, and performs feature engineering. The input data is data stored in a database, and the output is preprocessed training data.

[0526] Step 5:

[0527] The server uses the preprocessed data to train the generative AI to learn the user's behavioral patterns. A deep learning model is used here. The input data is the training data output in step 4, and the output is a trained model.

[0528] Step 6:

[0529] The action suggestion generation module uses the trained model to predict the next action to be taken in real time based on the user's behavioral patterns and generates action suggestions. New data points (e.g., current location or app usage status) are used as input, and action suggestions are generated as output. For example, a specific suggestion such as "Take a break at a coffee shop" is generated.

[0530] Step 7:

[0531] The server customizes the generated action suggestions and refers to the user's preferences and past behavioral history to provide more personalized suggestions. The input data are the action suggestions generated in step 6 and the user's preference data, and customized suggestions based on them are output.

[0532] Step 8:

[0533] The server notifies the user's device of the customized suggestion. The user's device visually displays the notification and informs the user of the suggested action. The input data is the customized suggestion output in step 7, and the notification data is output. A push notification is used for the specific notification.

[0534] Step 9:

[0535] The user selects a specific option for the proposed action and sends the selection to the server. The input data is the selection from the device, and this is output as the data to be sent to the server. Specifically, the selection is made using touch operations or the camera function.

[0536] Step 10:

[0537] The server uses an emotion engine that collects the user's emotional state and adjusts suggestions based on that state. Input data includes the user's facial expressions, voice, and text messages, and outputs the emotional state and adjustment data based on that state. If the emotional state is "stressed," the system will make suggestions such as "take a break in the relaxation area."

[0538] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0539] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0540] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0541] [Second embodiment]

[0542] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0543] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0544] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0545] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0546] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0547] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0548] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0549] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0550] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0551] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0552] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0553] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0554] The present invention is an action suggestion system for reducing "decision fatigue" in a user's daily life, and is implemented by the following means.

[0555] Overall system configuration

[0556] 1. Data Collection Module

[0557] Device: Your smartphone collects activity data, such as location, app usage, calendar events, and sleep data. This data is obtained from sensors and application APIs.

[0558] Example: Imagine a user commuting to work and turning on the GPS function on their smartphone, which collects location information every minute. It also records the launch time and usage time of each application they are using, as well as calendar event information.

[0559] 2. Data transmission module

[0560] On your device: We encrypt the activity data we collect and send it to our servers via a secure communication protocol (e.g., HTTPS).

[0561] Example: The acquired data is uploaded to a server in batch format at regular intervals (e.g., once an hour).

[0562] 3. Data receiving and storage module

[0563] Server: Receives data sent from the device and stores it in a database for each user. If necessary, it cleans the data (fills in missing values, corrects outliers, etc.).

[0564] Example: Data received on the server side is organized and saved in folders for each user. For example, user A's location data is stored as a dedicated table in the database.

[0565] 4. Data Learning Module

[0566] Server: Inputs the received data into the Generative AI, which learns the user's behavioral patterns. This Generative AI is trained using a deep learning model.

[0567] Example: Using location data and app usage history from the past month as training data, the app learns users' travel patterns and app usage trends.

[0568] 5. Action Proposal Generation Module

[0569] Server: Based on the learned data, predicts the next action to be taken and generates action suggestions. The generated suggestions are customized based on the user's preferences and past behavioral history.

[0570] Example: The server suggests "after-work dinner at a popular restaurant" on Friday evening, based on data about restaurants the user has frequently visited in the past.

[0571] 6. Action Suggestion Notification Module

[0572] Server: Sends customized suggestions to the user's smartphone app.

[0573] On the device: The smartphone app opens and notifies the user of the suggested action.

[0574] Example: Using the smartphone's push notification function, you can send a message to the user saying, "You can refresh yourself by jogging in a nearby park this Saturday."

[0575] 7. Choice Feedback Module

[0576] On the device: Presents specific options for suggested actions (e.g., photos of recipes or a list of activities) using a camera or touch interface.

[0577] User: Selects from the presented options by tapping, etc. The selected content is sent to the server and reflected in the learning of the generation AI.

[0578] Example: A user selects the suggested action "Go to a popular pasta restaurant" from the camera screen, and the selection is sent to the server.

[0579] Through the above-described embodiments, the present invention can reduce the decision-making burden in users' daily lives and prevent "decision fatigue." In particular, suggestions based on learned behavioral patterns are more tailored to individual users and provide higher accuracy and satisfaction than conventional general suggestion systems.

[0580] The processing flow will be explained below.

[0581] Step 1:

[0582] Activity Data Collection

[0583] Device: The user's smartphone collects location information, app usage history, calendar events, and sleep data, which are periodically retrieved from sensors and application APIs.

[0584] Specific operations: Turn on the GPS function to obtain location information, record the launch time and usage time of applications in use, extract schedule and event information from the calendar API, and obtain sleep data from a sleep tracker app.

[0585] Step 2:

[0586] Sending data to the server

[0587] On your device: Collected activity data is encrypted and sent to our servers using a secure communication protocol (e.g., HTTPS).

[0588] Specific behavior: Periodically checks the activity data stored in the local database, uploads any unsent data to the server in batches, encrypts the data using an encryption library, and sends an HTTP request.

[0589] Step 3:

[0590] Receiving and storing data

[0591] Server: Receives data sent from the device and stores it in a database for each user.

[0592] Specific operations: Receive data via API endpoint, organize the data by user ID, and save it in a database. Detect outliers and handle missing values ​​to check data integrity.

[0593] Step 4:

[0594] Data Preprocessing

[0595] Server: Cleans the received data (fills in missing values, handles outliers, etc.) and converts it into a format suitable for generative AI.

[0596] What it does: Normalize the data, sort it by time, and perform feature engineering if necessary, e.g., convert location information into geographic features and add categorical features based on time of day.

[0597] Step 5:

[0598] Generative AI training

[0599] Server: Trains the generative AI using preprocessed data to learn user behavior patterns.

[0600] Specific operation: Preprocessed data is input into the deep learning model, and it learns past behavioral patterns. During the training process, an appropriate learning rate and number of epochs are set, and adjustments are made while evaluating the model's performance.

[0601] Step 6:

[0602] Generate action suggestions

[0603] Server: Using the trained model, predict the user's next action in real time and generate action suggestions.

[0604] Specific Actions: New data points (e.g., current location or recent app usage history) are fed into the model to predict the next action, generating specific suggestions such as "get a coffee at a nearby cafe" or "relax at home."

[0605] Step 7:

[0606] Customize suggestions

[0607] Server: Customizes suggested actions based on the user's preferences and past behavior.

[0608] What it does: It applies personalization algorithms to generate recommendations that take into account the user's preferred activities and dining options. For example, it references a list of restaurants the user has visited in the past and makes new recommendations from there.

[0609] Step 8:

[0610] Submit your proposal

[0611] Server: Sends customized suggestions to the user's smartphone app.

[0612] Specific action: Using the push notification system, the suggested action is sent to the user's device in real time. For example, a notification message saying, "You should go see a movie at this theater right now" is sent.

[0613] Step 9:

[0614] View Suggestions

[0615] On the device: Display received action suggestions visually to the user, for example in the notification area or on an in-app dashboard.

[0616] What it does: Receive a push notification and display the suggestion in the app's UI, such as "Enjoy dinner at a popular restaurant nearby," in an easily accessible format for the user.

[0617] Step 10:

[0618] Presenting options

[0619] Device: Presents specific options for suggested actions (e.g., photos of dishes or a list of activities) and utilizes a camera or touch-enabled interface.

[0620] Specific operation: The camera function is used to scan the surrounding environment, and the presented options are displayed as images. The user taps to confirm the option.

[0621] Step 11:

[0622] Reflecting your selection

[0623] Device: The user's selection is sent to the server and reflected in future behavior predictions.

[0624] Specific actions: The user's selected actions are recorded on the device and sent to the server for the next learning cycle. The server then incorporates the received selections into the training data of the generative AI to improve the model's performance.

[0625] Example 1

[0626] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0627] Many modern users feel fatigued by the numerous choices and decisions they have to make in their daily lives, and are facing what is known as "decision fatigue." This "decision fatigue" reduces the quality of their daily lives and increases stress and dissatisfaction. This has led to a demand for systems that can provide personalized action suggestions and decision support for individual users. However, existing systems are unable to fully learn users' behavioral patterns, making it difficult to provide highly accurate suggestions.

[0628] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0629] In this invention, the server includes means for collecting activity data such as user location information, application usage history, calendar events, and sleep data, means for encrypting the activity data and transmitting it to the server via a secure communication protocol, means for receiving the activity data, storing it in a database for each user, and cleaning the data, means for inputting the activity data into a generative AI model and learning the user's behavioral patterns using a deep learning model, and means for suggesting the next action to be taken based on the learned behavioral patterns, thereby enabling highly accurate action suggestions based on the user's behavioral history.

[0630] "Location information" refers to data that indicates the user's current location and the route they are traveling.

[0631] "Application usage history" refers to data that records usage information of applications used by users on smartphones and other devices.

[0632] "Calendar events" refers to data that records a user's schedules and events on a calendar.

[0633] "Sleep data" refers to data that records the user's sleep time and sleep quality.

[0634] "Activity data" is a general term for data related to a user's daily activities, such as location information, application usage history, calendar events, and sleep data.

[0635] "Encryption" is a technology that converts data to transmit information securely and prevents unauthorized access and data leaks.

[0636] A "secure communication protocol" is a communication protocol for securely sending and receiving data, and a specific example is HTTPS.

[0637] "Data cleaning" is a process of improving data quality by filling in missing values ​​and correcting outliers.

[0638] A "generative AI model" is an artificial intelligence model that generates new information and suggestions based on data.

[0639] A "deep learning model" is a machine learning model that uses a multi-layer neural network to perform advanced learning and prediction.

[0640] A "behavioral pattern" is a series of data that indicates a user's daily behavior and tendencies.

[0641] "Action suggestions" are information or ideas that recommend the next action the user should take.

[0642] "Terminal" refers to a device that is directly operated by a user, such as a smartphone or tablet.

[0643] "Choices" are any list or item that allows a user to choose from multiple suggestions or options.

[0644] Overall system configuration

[0645] The present invention provides an action suggestion system for reducing "decision fatigue" in a user's daily life. This system is implemented by the following components and means.

[0646] Data Collection Module

[0647] Device: Your smartphone collects location information, application usage history, calendar events, sleep data, and other activity data. This is done using the smartphone's GPS, various sensors, and application APIs.

[0648] Example: When a user commutes to work, the GPS function on their smartphone is turned on, and location information is collected every minute. In addition, the launch time and usage time of each application being used are collected through the application API, as are events registered in the calendar.

[0649] Data Transmission Module

[0650] On your device: Collected activity data is encrypted and sent to a server via a secure communication protocol (e.g., HTTPS).

[0651] Example: The collected data is encrypted using the AES encryption algorithm and then sent to the server via HTTPS. The data is uploaded to the server in batches, for example, once an hour.

[0652] Data reception and storage module

[0653] Server: Receives data sent from the device and stores it in a database for each user. It also cleans the data as needed (filling in missing values, correcting outliers, etc.).

[0654] Example: Data received on the server side is organized and saved in a dedicated folder for each user. Before saving to the database, outliers are detected and corrected or deleted, and the data is saved as clean data. For example, the location information data of user A is stored in a dedicated table in the database.

[0655] Data Learning Module

[0656] Server: Inputs the received data into a generative AI model to learn user behavior patterns. This generative AI model is trained using a deep learning model.

[0657] Example: Location data and app usage history from the past month are input into a deep learning model as training data to learn the user's movement patterns and app usage trends. Specifically, CNN (convolutional neural network) and LSTM (long short-term memory) models are used.

[0658] Action proposal generation module

[0659] Server: Based on the learned data, the server proposes the next action to be taken. The proposed actions are customized based on the user's preferences and past behavioral history.

[0660] Example: For example, the server suggests "have dinner at a popular restaurant after work" on Friday evening. This suggestion is generated based on the data of restaurants the user has frequently visited in the past. Also, the server selects vegetarian restaurants as the user's preference.

[0661] Action Suggestion Notification Module

[0662] Server: Sends customized suggestions to the user's smartphone app.

[0663] On the device: The smartphone app opens and notifies the user of the suggested action.

[0664] Example: A smartphone app that receives a suggestion message from a server uses the push notification function to display a message to the user saying, "You can refresh yourself by jogging in a nearby park this Saturday."

[0665] Choice Feedback Module

[0666] On the device: Presents specific options for suggested actions (e.g., photos of recipes or a list of activities) using a camera or touch interface.

[0667] User: Selects from the presented options by tapping, etc. The selected content is sent to the server and reflected in the learning of the generation AI.

[0668] Example: A user selects a suggested action, such as "Go to a popular pasta restaurant," on the camera screen, and this information is sent to the server in real time, allowing the generative AI model to learn new data and provide more personalized recommendations the next time.

[0669] Examples of prompts for this system include "Predict the user's next move based on location data from the past month" and "Suggest activities for this weekend, taking into account the user's preferences."

[0670] Through these modules and processes, the present invention can significantly reduce the burden of decision-making in users' daily lives and effectively prevent "decision fatigue." By providing personalized recommendations, it increases satisfaction and achieves higher accuracy than conventional general recommendation systems.

[0671] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0672] Step 1:

[0673] Data collection

[0674] Device: Collects activity data such as user location, application usage history, calendar events, and sleep data. This data collection is done using the smartphone's GPS function, various sensors, and application APIs.

[0675] Input: Raw data collected from sensors on the user's smartphone and application APIs (location, application usage history, calendar events, sleep data).

[0676] Processing: Collecting data from sensors and APIs and organizing it into a specific format as activity data.

[0677] Output: Organized activity data.

[0678] What it does: When a user commutes to work, the GPS function on their smartphone is turned on, and location information is collected every minute, along with application launch times, usage time, and calendar events.

[0679] Step 2:

[0680] Data transmission

[0681] On your device: Collected activity data is encrypted and sent to a server via a secure communication protocol (e.g., HTTPS).

[0682] Input: Organized activity data.

[0683] Processing: The organized activity data is encrypted using the AES encryption algorithm and sent to the server via HTTPS.

[0684] Output: The encrypted data is sent to the server.

[0685] Specific operation: The collected data is periodically (for example, once an hour) encrypted in batch format and uploaded to the server via HTTPS communication.

[0686] Step 3:

[0687] Data reception and storage

[0688] Server: Receives data sent from the device and stores it in a database for each user. It also cleans the data (fills in missing values, corrects outliers, etc.).

[0689] Input: Encrypted activity data.

[0690] Processing: The encrypted data is decrypted, organized by user, and stored in a database. Data cleaning is performed.

[0691] Output: Cleaned per-user activity data.

[0692] Specific operations: Decrypts encrypted data on the server side and stores it in a folder for each user in the database. Detects and imputes outliers, and fills in missing values ​​to improve data quality.

[0693] Step 4:

[0694] Data Learning

[0695] Server: Inputs the received data into a generative AI model to learn user behavior patterns. This generative AI model is trained using a deep learning model.

[0696] Input: Cleaned per-user activity data.

[0697] Processing: The cleaned data is fed into a deep learning model to learn user behavior patterns.

[0698] Output: Learned behavioral pattern model.

[0699] How it works: The system inputs location data and application usage history from the past month into a deep learning model to learn the user's movement patterns and app usage trends, using deep learning techniques such as CNN and LSTM.

[0700] Step 5:

[0701] Action suggestion generation

[0702] Server: Based on the learned data, the server proposes the next action to be taken. The proposed actions are customized based on the user's preferences and past behavioral history.

[0703] Input: Learned behavioral pattern model.

[0704] Processing: Based on the learned model, next action suggestions are generated, which are customized based on the user's preferences and past behavioral history.

[0705] Output: Customized action suggestions.

[0706] Specific behavior: For example, the server may suggest "have dinner at a popular restaurant after work" on a Friday evening. This suggestion is generated based on data on restaurants the user has frequently visited in the past.

[0707] Step 6:

[0708] Action suggestion notification

[0709] Server: Sends customized suggestions to the user's smartphone app.

[0710] On the device: The smartphone app notifies the user of the suggested action.

[0711] Input: customized action suggestions.

[0712] Processing: Sends the proposed content to the smartphone app and notifies it.

[0713] Output: The user is notified of the suggestion.

[0714] Specific operation: The smartphone app receives a proposal message from the server and uses the push notification function to notify the user of the proposal.

[0715] Step 7:

[0716] Selection Feedback

[0717] On the device: Present specific options for the suggested action (e.g., photos of recipes or a list of activities) using a camera or touch interface.

[0718] User: Selects from the presented options by tapping, etc. The selected content is sent to the server and reflected in the learning of the generation AI.

[0719] Input: The user's selection.

[0720] Processing: The user's selection is sent to the server and reflected in the generative AI model.

[0721] Output: The updated training model.

[0722] What happens: The user taps to select the suggested action "Go to a popular pasta restaurant," which is then sent to the server, allowing the generative AI model to learn new data and provide more personalized recommendations the next time.

[0723] (Application example 1)

[0724] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0725] While modern users have access to a wide range of entertainment content, they often experience "decision fatigue" when trying to choose the best content from the diverse options. Under these circumstances, there is a need for systems that can appropriately suggest the next content to watch based on the user's preferences and behavioral history. Current systems are unable to effectively utilize the content and activity data of users' past viewings, making it difficult to make optimal suggestions for each user.

[0726] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0727] In this invention, the server includes means for collecting activity data such as a user's location information, app usage history, calendar events, and sleep data, means for transmitting the activity data to the server via a secure communication protocol, means for a generation AI to learn the user's behavioral patterns using a deep learning model based on the activity data, means for suggesting entertainment content to be viewed next based on the behavioral patterns, means for notifying the user's smartphone of the suggested content, and means for reflecting the user's selection in the learning, thereby enabling users to easily select entertainment content appropriately customized based on their preferences and behavioral history.

[0728] "Location information" is data that indicates a user's current location and is obtained using GPS or other location information technology.

[0729] "App usage history" is a record of applications used by a user on a smartphone or other device, and includes data such as the time the app was launched and the amount of time it was used.

[0730] "Calendar events" are information about events and schedules that a user registers in a calendar to manage their plans and appointments.

[0731] "Sleep data" refers to data that indicates the user's sleep time and quality, and is collected via smartphones or wearable devices.

[0732] "Activity data" is data that records various actions and conditions of a user's daily life, and includes location information, app usage history, calendar events, sleep data, and more.

[0733] A "secure communication protocol" is a safe means of communication that prevents eavesdropping or tampering by third parties when sending and receiving data, and examples include protocols such as HTTPS.

[0734] "Generative AI" is artificial intelligence that uses machine learning techniques such as deep learning to train user data to perform specific tasks.

[0735] A "deep learning model" is a type of artificial intelligence that uses multi-layered neural networks to learn complex patterns from data.

[0736] A "behavioral pattern" indicates the tendencies and habits of the user's daily behavior, and is determined based on past data.

[0737] "Entertainment content" refers to media such as movies, dramas, music, podcasts, etc. that users consume for entertainment purposes.

[0738] A "smartphone" is a multi-function mobile phone that is a device that can not only make calls but also access the Internet and use applications.

[0739] "Push notification" is a function that notifies information to smartphones and other devices in real time.

[0740] A "selection" refers to the particular item a user chooses from among the options offered.

[0741] "Means for reflecting in learning" refers to a method for feeding back user choices to the generative AI model to improve the accuracy of subsequent suggestions.

[0742] The present invention is an action suggestion system for reducing "decision fatigue" in a user's daily life, and specific implementation means are described below.

[0743] Data Collection Module

[0744] The device used is the user's smartphone, which collects location information, app usage history, calendar events, sleep data, and other activity data. This data is obtained through the smartphone's built-in sensors and application APIs.

[0745] Data Transmission Module

[0746] Activity data collected on the device is sent to a server using a secure communication protocol (e.g., HTTPS). The data is encrypted and uploaded in batches at regular intervals (e.g., once an hour).

[0747] Data reception and storage module

[0748] The server side receives the transmitted data and stores it in a database for each user. Data cleansing (filling in missing values, correcting outliers, etc.) is also performed. For example, SQLite is used as the database.

[0749] Data Learning Module

[0750] Based on the data received on the server, the generated AI uses a deep learning model to learn user behavior patterns. To do this, it uses deep learning libraries such as TensorFlow. It uses past location data and app usage history as training data to learn user patterns and tendencies.

[0751] Action proposal generation module

[0752] The server uses the learned data to predict the next entertainment content to watch and generates action suggestions, which are customized based on the user's preferences and past behavioral history.

[0753] Action Suggestion Notification Module

[0754] The generated action suggestions are sent to the user's smartphone using the push notification function, and the notification content includes specific entertainment content such as "the next movie to watch."

[0755] Choice Feedback Module

[0756] The user taps to select an option from the suggested content. This selection is sent to the server and reflected in the learning of the generative AI. The suggested options can also be visually displayed using the smartphone's camera function.

[0757] Specific examples

[0758] For example, at 8 p.m. on a weekday while a user is relaxing at home, a push notification on their smartphone suggests a movie to watch next based on their past viewing history and activity data. This suggestion specifically suggests movies in a genre that the user previously enjoyed.

[0759] Example prompt sentence:

[0760] "Below is your past viewing history and recent activity data. Based on this data, we'd like to suggest what you should watch next."

[0761] Viewing history: [Movie A, Drama B, Music C...]

[0762] Latest activity data: {Location: Home, Time: 20:00, Date: 2023-10-02}

[0763] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0764] Step 1:

[0765] The device (user's smartphone) collects activity data such as location information, app usage history, calendar events, and sleep data.

[0766] Input: Data obtained through smartphone sensors and APIs.

[0767] Specific operation: Using the smartphone's GPS, it collects location information every minute and also periodically records app usage history and calendar events.

[0768] Output: Collected activity data (location, app usage history, etc.).

[0769] Step 2:

[0770] The device transmits the collected activity data to the server via a secure communication protocol (e.g., HTTPS).

[0771] Input: Collected activity data.

[0772] What it does: It encrypts your activity data and uploads it to a server in batches using the HTTPS protocol, for example, every hour.

[0773] Output: Activity data sent to the server.

[0774] Step 3:

[0775] The server receives the data and stores it in a database. It also performs data cleansing, such as filling in missing values ​​and correcting outliers.

[0776] Input: Submitted activity data.

[0777] Specific operation: The received data is organized and stored in a database for each user, outliers are detected and corrected, and missing values ​​are complemented.

[0778] Output: Saved and cleansed activity data.

[0779] Step 4:

[0780] The server uses generative AI (deep learning model) to learn the user's behavioral patterns based on the stored activity data.

[0781] Input: Saved activity data.

[0782] Specific operation: Using a deep learning library (such as TensorFlow), the model is trained using past location information and app usage history as training data. The following is an example of a prompt sentence to input.

[0783] Example prompt: "Below is the user's past viewing history and latest activity data. Based on this data, please suggest the next content they should watch." Viewing history: [Movie A, Drama B, Music C...] Latest activity data: {Location: Home, Time: 20:00, Date: 2023-10-02}

[0784] Output: An AI model that has learned the user's behavioral patterns.

[0785] Step 5:

[0786] The server uses the learned AI model to predict the next entertainment content to watch and generate action suggestions.

[0787] Input: An AI model that has learned your behavioral patterns, and your latest activity data.

[0788] What it does: It uses AI models to predict the best entertainment content for users based on their current situation and past behavioral patterns.

[0789] Output: Suggestions about what entertainment content to watch next.

[0790] Step 6:

[0791] The server sends the generated action suggestions to the smartphone via push notification.

[0792] Input: Suggestions for entertainment content to watch next.

[0793] Specific operation: Using a push notification library (e.g., Firebase Cloud Messaging), the suggestion content is sent to the user's smartphone.

[0794] Output: Action suggestions displayed on the user's smartphone.

[0795] Step 7:

[0796] Users tap to select from the suggested entertainment content, and the selected content is sent to the server and reflected in the learning of the generative AI.

[0797] Input: User-selected entertainment content.

[0798] Specific operation: Using the smartphone's camera function and touch operation interface, the proposed options are visually displayed and the user's tap selection information is collected. The selection information is then sent back to the server and used to update the generative AI model.

[0799] Output: The user's choices fed back to the generative AI model.

[0800] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0801] The present invention is an action suggestion system for reducing "decision fatigue" in a user's daily life, and further improves the accuracy and satisfaction of the suggestions by incorporating an emotion engine that recognizes and responds to the user's emotional state. The present invention is implemented by the following means.

[0802] Overall system configuration

[0803] 1. Data Collection Module

[0804] Device: Your smartphone collects activity data, such as location, app usage history, calendar events, and sleep data. This data is periodically retrieved from sensors and application APIs.

[0805] Example: A user turns on the GPS on their smartphone during their commute to collect location information, record app usage history, calendar events, and data from a sleep tracker.

[0806] 2. Data transmission module

[0807] On your device: Collected activity data is encrypted and sent to a server via a secure communication protocol.

[0808] Example: The acquired data is uploaded to the server in batches every hour.

[0809] 3. Data receiving and storage module

[0810] Server: Stores the received data in a database for each user, and cleans the data if necessary.

[0811] Example: Organize data by user ID on the server side, process outliers and missing values, and then save it to the database.

[0812] 4. Data Preprocessing Module

[0813] Server: Cleans the received data and converts it into a format suitable for the generative AI.

[0814] Examples: data normalization, timestamp alignment, feature engineering.

[0815] 5. Generative AI Learning Module

[0816] Server: Trains the generative AI using preprocessed data to learn user behavior patterns.

[0817] Example: Using past location information and app usage history as training data to recognize users' travel patterns and app usage tendencies.

[0818] 6. Action Proposal Generation Module

[0819] Server: Uses the trained model to predict the user's next action in real time and generate action suggestions.

[0820] Example: Feed new data points into the model to generate specific suggestions, such as "have a coffee at a nearby cafe."

[0821] 7. Proposal customization module

[0822] Server: Customizes suggested actions based on the user's preferences and past behavior.

[0823] Example: Referencing data on restaurants the user has visited in the past and suggesting "dinner at a popular restaurant after work."

[0824] 8. Action Suggestion Notification Module

[0825] Server: Sends customized suggestions to the user's smartphone app.

[0826] Terminal: Visually notify the user of the proposed action.

[0827] Example: Use a push notification to say, "You can refresh yourself by jogging in a nearby park this Saturday."

[0828] 9. Choice Feedback Module

[0829] Device: Presents specific options for suggested actions using a camera or touch interface.

[0830] User: Tap to select a presented option. The selection is sent to the server and reflected in the learning of the generation AI.

[0831] Example: A user selects the option "Go to a popular pasta restaurant" from a camera screen, and the selection is sent to a server.

[0832] 10. Emotion Engine Module

[0833] Device: Recognizes the user's emotional state by analyzing facial expressions, voice, and text messages.

[0834] Example: Using a smartphone's camera and microphone, it analyzes emotions from the user's facial expressions and tone of voice, as well as the content of text messages.

[0835] 11. Emotion-based behavior suggestion adjustment module

[0836] Server: Adjusts the recommendations based on the results from the emotion engine.

[0837] Example: If the user is feeling stressed, suggest a relaxing activity (e.g., yoga or meditation).

[0838] Through the above-described embodiments, the present invention reduces the burden of decision-making in the user's daily life, and furthermore, by using an emotion engine, it is possible to realize optimal action suggestions that take into account the user's mental state, thereby providing higher accuracy and satisfaction than conventional systems.

[0839] The processing flow will be explained below.

[0840] Step 1:

[0841] Activity Data Collection

[0842] Device: Your smartphone collects activity data, such as location, app usage history, calendar events, and sleep data. This data is periodically retrieved from sensors and application APIs.

[0843] Specific operations: Turn on the GPS function to obtain location information, record the launch time and usage time of each application in use, extract schedule and event information from the calendar API, and obtain sleep data from a sleep tracker app.

[0844] Step 2:

[0845] Sending data to the server

[0846] On your device: We encrypt the activity data we collect and send it to our servers using a secure communication protocol (e.g., HTTPS).

[0847] Specific behavior: Periodically checks the activity data stored in the local database, uploads any unsent data to the server in batches, encrypts the data using an encryption library, and sends an HTTP request.

[0848] Step 3:

[0849] Receiving and storing data

[0850] Server: Receives data sent from the device and stores it in a database for each user.

[0851] Specific operations: Receive data via API endpoint, organize the data by user ID, and save it in a database. Detect outliers and handle missing values ​​to check data integrity.

[0852] Step 4:

[0853] Data Preprocessing

[0854] Server: Cleans the received data (fills in missing values, handles outliers, etc.) and converts it into a format suitable for generative AI.

[0855] What it does: Normalizes the data, aligns it on the time axis, and performs feature engineering as needed, e.g., converting location information into geographic features and adding categorical features based on time of day.

[0856] Step 5:

[0857] Generative AI training

[0858] Server: Trains the generative AI using preprocessed data to learn user behavior patterns.

[0859] Specific operation: Preprocessed data is input into the deep learning model, and it learns past behavioral patterns. During the training process, an appropriate learning rate and number of epochs are set, and adjustments are made while evaluating the model's performance.

[0860] Step 6:

[0861] Generate action suggestions

[0862] Server: Using the trained model, predict the user's next action in real time and generate action suggestions.

[0863] Specific Actions: New data points (e.g., current location or recent app usage history) are fed into the model to predict the next action, generating specific suggestions such as "get a coffee at a nearby cafe" or "relax at home."

[0864] Step 7:

[0865] Customize suggestions

[0866] Server: Customizes suggested actions based on the user's preferences and past behavior.

[0867] What it does: It applies personalization algorithms to generate recommendations that take into account the user's preferred activities and dining options. For example, it references a list of restaurants the user has visited in the past and makes new recommendations from there.

[0868] Step 8:

[0869] Submit your proposal

[0870] Server: Sends customized suggestions to the user's smartphone app.

[0871] Specific action: Using the push notification system, the suggested action is sent to the user's device in real time. For example, a notification message saying, "You should go see a movie at this theater right now" is sent.

[0872] Step 9:

[0873] View Suggestions

[0874] On the device: Display received action suggestions visually to the user, for example in the notification area or on an in-app dashboard.

[0875] What it does: Receive a push notification and display the suggestion in the app's UI, for example, "Enjoy dinner at a popular restaurant nearby," in a format that's easily accessible to the user.

[0876] Step 10:

[0877] Presenting options

[0878] Device: Presents specific options for suggested actions (e.g., photos of dishes or a list of activities) and utilizes a camera or touch-enabled interface.

[0879] Specific operation: The camera function is used to scan the surrounding environment, and the presented options are displayed as images. The user taps to confirm the option.

[0880] Step 11:

[0881] Reflecting your selection

[0882] Device: The user's selection is sent to the server and reflected in future behavior predictions.

[0883] Specific actions: The user's selected actions are recorded on the device and sent to the server for the next learning cycle. The server then incorporates the received selections into the training data of the generative AI to improve the model's performance.

[0884] Step 12:

[0885] Collecting Emotional Data

[0886] Device: Recognizes the user's emotional state by analyzing facial expressions, voice, and text messages.

[0887] Specific operation: Using the smartphone's camera and microphone, the system analyzes the user's facial expressions and tone of voice, and uses natural language processing to analyze the content of text messages to determine their emotional state.

[0888] Step 13:

[0889] Sending emotional data

[0890] Terminal: Sends analyzed emotion data to the server.

[0891] Specific operation: The emotion data output by the emotion recognition engine is encrypted and sent to the server using a secure communication protocol.

[0892] Step 14:

[0893] Tailoring behavioral suggestions based on emotions

[0894] Server: Adjusts the recommendations based on the results from the emotion engine.

[0895] Specific actions: If the user is feeling stressed, it will suggest relaxing actions (e.g., yoga or meditation). Conversely, if the user is feeling excited, it will suggest active actions (e.g., sports or dancing).

[0896] Through the above-described embodiments, the present invention reduces the burden of decision-making in the user's daily life, and furthermore, by using an emotion engine, it is possible to realize optimal action suggestions that take into account the user's mental state, thereby providing higher accuracy and satisfaction than conventional systems.

[0897] Example 2

[0898] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0899] While current technology exists to reduce the burden of decision-making in users' daily lives, these systems are unable to take into account their emotional state when making suggestions, which can result in low satisfaction. To solve this problem, it is necessary to provide a system that recognizes the user's emotional state and suggests optimal actions.

[0900] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting activity data such as the user's location information, app usage history, calendar events, and sleep data, means for encrypting the activity data and transmitting it to the server, means for a generation AI to learn the user's behavioral patterns based on the activity data, means for suggesting a next action to be taken based on the behavioral patterns, means for recognizing the user's emotional state using an emotion engine and adjusting the suggestion content, means for notifying the user's terminal of the suggested action, and means for reflecting the user's selection in the learning. This reduces the burden of decision-making in the user's daily life and enables highly accurate and satisfying action suggestions that take the user's emotional state into consideration.

[0901] "User location information" is geographical data that indicates where the user is currently located.

[0902] "Application usage history" is data that records the names of applications used by a user, the duration of use, frequency of use, and so on.

[0903] A "calendar event" is event information entered into a calendar application that registers a user's plans and schedules.

[0904] "Sleep data" refers to data that records the user's sleep patterns, sleep duration, quality, etc.

[0905] "Activity data" refers to all data related to daily life activities, such as a user's location information, app usage history, calendar events, and sleep data.

[0906] "Encryption" means converting data into a form that is unintelligible to others using a specific algorithm.

[0907] "Generative AI" is an artificial intelligence model that can learn a user's behavioral patterns and suggest the next action to take.

[0908] A "behavioral pattern" refers to a series of actions and habits in a user's daily life.

[0909] The "emotion engine" is an engine that analyzes and identifies the user's emotional state from their facial expressions, voice, and text.

[0910] "Suggestion content" refers to a suggested action that the generation AI generates by taking into account the user's behavioral patterns and emotional state.

[0911] "Notification" refers to displaying the generated action suggestion on the user's terminal.

[0912] "Selection" refers to the item that the user selects from the suggested actions.

[0913] "Reflecting in learning" means incorporating user selections into the generative AI model to improve the accuracy of future action suggestions.

[0914] This invention is a system for reducing the burden of decision-making in a user's daily life and for suggesting optimal actions that take into account the user's emotional state.

[0915] System configuration

[0916] Data Collection Module

[0917] The user's device, specifically their smartphone, is used to collect activity data such as location information, app usage history, calendar events, and sleep data, which are periodically retrieved through the device's sensors and application APIs.

[0918] Example: A user turns on the GPS on their smartphone during their commute to collect location information, record app usage history, calendar events, and data from a sleep tracker.

[0919] Data Transmission Module

[0920] Activity data collected on the device is encrypted and sent to a server via a secure communication protocol (e.g., HTTPS).

[0921] Example: The collected data is AES encrypted in batches every hour and uploaded to the server via HTTPS protocol.

[0922] Data reception and storage module

[0923] The server stores the received data in a database for each user. Before storing, it cleans the data (dealing with outliers and imputing missing values) as necessary.

[0924] Example: The data received by the server is organized into a database by user ID, and outliers and missing values ​​are automatically processed.

[0925] Data Preprocessing Module

[0926] The server cleans the received data and converts it into a format suitable for generative AI, including normalizing the data, aligning timestamps, and extracting features.

[0927] Example: Prepare a training dataset by properly aligning timestamps, normalizing numeric data, and removing unnecessary fields.

[0928] Generative AI learning module

[0929] The server uses the preprocessed data to train a generative AI to learn the user's behavioral patterns. This generative AI is trained using a deep learning framework (e.g., TensorFlow, PyTorch).

[0930] Example: Using past location information and app usage history to learn about users' travel patterns and app usage habits.

[0931] Action proposal generation module

[0932] The server uses a trained generative AI model to predict the user's next action in real time and generate action suggestions.

[0933] Example: Newly collected data points are fed into a generative AI model to generate specific suggestions, such as "It's almost lunchtime, so suggest a nearby cafe."

[0934] Suggestion customization module

[0935] The server customizes the generated suggestions based on the user's preferences and past behavioral history.

[0936] Example: Referencing data on restaurants frequently visited by the user to provide customized suggestions such as "dinner at a popular restaurant after work."

[0937] Action Suggestion Notification Module

[0938] The server sends the customized action suggestions to the user's smartphone app, which then visually notifies the user of the suggested action.

[0939] Example: Use push notifications to notify users of something like, "You can refresh yourself by jogging in a nearby park this Saturday."

[0940] Choice Feedback Module

[0941] The device uses a camera or touch interface to present specific options for the proposed action. The user taps to select the option. The selection is sent to the server and reflected in the learning of the generative AI.

[0942] Example: A user selects the option "Go to a popular pasta restaurant" from a camera screen, and the selection is sent to a server.

[0943] Emotion Engine Module

[0944] The device recognizes the user's emotional state by analyzing their facial expressions, voice, and text messages.

[0945] Example: Using the smartphone camera and microphone, the system analyzes emotions from the user's facial expressions and tone of voice, including text messages.

[0946] Emotion-based behavior suggestion adjustment module

[0947] The server adjusts the suggestions based on the emotional state obtained from the emotion engine.

[0948] Example: If the user is feeling stressed, suggest a relaxing activity (e.g., meditation, yoga).

[0949] Examples of prompts for generative AI models

[0950] "Generate optimal action suggestions based on the user's current location, app usage history, calendar events, and sleep data."

[0951] This invention can reduce the burden of decision-making in the user's daily life and realize highly accurate and satisfying action suggestions that also take into account the user's emotional state.

[0952] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0953] Step 1:

[0954] Data collection

[0955] Input: Activity data about the user's daily life (location, app usage history, calendar events, sleep data).

[0956] Specific operation: The device (smartphone) acquires location information using GPS, records application usage history in the background, obtains the user's schedule from a calendar app, and collects sleep data from a sleep tracker app.

[0957] Output: Collected activity data.

[0958] Step 2:

[0959] Data transmission

[0960] Input: Collected activity data.

[0961] Specific operation: The device encrypts the collected data using AES and sends it to the server using a secure communication protocol (HTTPS).

[0962] Output: The encrypted data is sent to the server.

[0963] Step 3:

[0964] Data reception and storage

[0965] Input: Encrypted data.

[0966] Specific operation: The server receives the data, unlocks it, and then stores it in a database for each user ID. At that time, it cleans the data by removing outliers and filling in missing values.

[0967] Output: An organized database.

[0968] Step 4:

[0969] Data Preprocessing

[0970] Input: A curated database.

[0971] Specific operations: The server normalizes the data, aligns timestamps, removes unnecessary fields, extracts features, and converts it into a format suitable for generative AI.

[0972] Output: The preprocessed dataset.

[0973] Step 5:

[0974] Generative AI training

[0975] Input: The preprocessed dataset.

[0976] How it works: The server trains the generative AI using a deep learning framework (e.g., TensorFlow, PyTorch), learning the dataset over multiple epochs and incorporating user behavior patterns into the model.

[0977] Output: The trained model.

[0978] Step 6:

[0979] Action suggestion generation

[0980] Input: Newly collected data points, trained model.

[0981] What it does: The server uses the trained model to analyze new data points collected and predict the next action to take. It generates suggestions based on prompts (e.g., "Generate optimal action suggestions based on the current user's location, app usage history, calendar events, and sleep data.").

[0982] Output: Proposal.

[0983] Step 7:

[0984] Customize suggestions

[0985] Input: Proposal content, past behavior history.

[0986] Specific operation: The server customizes the generated suggestions based on the user's preferences and past behavior history, taking into account the places visited in the past and preferred activities to provide optimal suggestions.

[0987] Output: A customized proposal.

[0988] Step 8:

[0989] Action suggestion notification

[0990] Input: Your customized suggestion.

[0991] Specific operation: The server uses the push notification API to send customized suggestions to the device, which then visually notifies the user of the suggested action.

[0992] Output: The suggestion that was communicated to the user.

[0993] Step 9:

[0994] Selection Feedback

[0995] Input: User choices, camera and touch interfaces.

[0996] Specific actions: The device visually displays specific options for the proposed action, and the user taps to select it. The user's selection is sent to the server and reflected in the learning of the generative AI.

[0997] Output: The selections sent to the server.

[0998] Step 10:

[0999] Emotion Engine

[1000] Input: User facial expressions, voice, and text messages.

[1001] How it works: The device uses the smartphone's camera and microphone to collect the user's facial expressions and voice, and analyzes emotions from text messages.

[1002] Output: Emotional state data.

[1003] Step 11:

[1004] Emotion-based behavioral adjustment

[1005] Input: Emotional state data, customized suggestions.

[1006] Specific operation: The server further adjusts the suggested actions based on the emotional state obtained from the emotion engine. If the user is feeling stressed, the server changes the suggested actions to suggest relaxing activities.

[1007] Output: Final recommendations based on the user's emotional state.

[1008] (Application example 2)

[1009] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1010] In recent years, the proliferation of options in consumers' daily lives has led to the problem of "decision fatigue." Particularly in brick-and-mortar stores, consumers are forced to make optimal choices from a vast array of products and services, placing a psychological burden on them. Furthermore, because optimal recommendations vary depending on a consumer's emotional state, there is a need for a system that can analyze each individual's emotional state and adjust the recommendations accordingly.

[1011] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting behavioral data such as user location information, app usage history, time information, and sleep data; means for transmitting the behavioral data to the server; means for a generation AI to learn the user's behavioral patterns based on the behavioral data; means for suggesting the next action to be taken based on the behavioral patterns; means for notifying the user's device of the suggested action and making a purchase suggestion; means for reflecting the user's selection in the learning; and means including an emotion engine for collecting the user's emotional state and adjusting the suggestion content. This reduces the decision-making burden in consumers' daily lives and makes it possible to provide personalized suggestions based on each individual's emotional state in real time.

[1012] "User location information" is data that identifies the user's current geographic location.

[1013] "Application usage history" is data on applications that a user has used in the past.

[1014] "Time information" is data relating to the time and date when the user performed an action.

[1015] "Sleep data" is information related to the user's sleep, including the amount of sleep time and the quality of sleep.

[1016] "Behavioral data" is a general term for data including location information, app usage history, time information, sleep data, and the like in a user's daily life.

[1017] "Generative AI" is an artificial intelligence that learns user behavior patterns based on collected data and generates new suggestions.

[1018] "Behavioral patterns" refer to tendencies and habits extracted from a user's past behavior.

[1019] The "means for suggesting an action" is a method for indicating the next action to be taken based on the user's behavioral patterns.

[1020] A "purchase suggestion" is a suggestion that recommends specific products or services related to consumption behavior to a user.

[1021] The "emotion engine" is a function that analyzes the user's emotional state and adjusts the content of suggestions based on that state.

[1022] "Personalized offers" are individual offers tailored to a user's individual interests, preferences, and emotional state.

[1023] "Server" means a computer system that receives, stores, analyzes, and generates recommendations from user-submitted data.

[1024] This invention is an action suggestion system designed to reduce "decision fatigue" in users' daily lives. It also incorporates an emotion engine that recognizes and responds to the user's emotional state, improving the accuracy of the suggestions and satisfaction. This system collects behavioral data, such as the user's location, app usage history, time information, and sleep data, and uses this data to generate AI that learns the user's behavioral patterns. Based on the behavioral patterns, the system then suggests the next action to be taken and notifies the user's device of the suggested action.

[1025] First, user behavior data is collected from devices such as smartphones. Specifically, this includes location information using GPS, application usage history, calendar events, and data from sleep trackers. This data is collected in real time or periodically and sent to a server. The data is encrypted and transmitted via a secure communication protocol.

[1026] The server stores the received behavioral data in a database, cleans the data, and then performs preprocessing to train the generative AI model. Specifically, it normalizes the data, aligns timestamps, and performs feature engineering. The preprocessed data is then used to train the generative AI, which learns the user's behavioral patterns. This AI model uses deep learning technology.

[1027] The action suggestion generation module uses a trained generative AI model to predict the next action a user should take in real time based on their behavioral patterns, generating action suggestions. For example, if the user is in a physical store, suggestions such as "take a break at a coffee shop" or "check for new product sales" will be made. These suggestions are further refined by an emotion engine that analyzes the user's emotional state. Specifically, the system analyzes the user's facial expressions and voice using the smartphone's camera and microphone, as well as the content of text messages.

[1028] The proposed actions, including purchasing suggestions, are sent to the user's device. The user selects from the suggested options using touch operations on their smartphone, and the selection is then sent back to the server. This allows the generative AI model to constantly retrain based on the latest data, improving the accuracy of the suggestions.

[1029] For example, if the system detects that the user is feeling stressed, it will suggest "take a break in the relaxation area." If the user is feeling normal, it will suggest "try your favorite pasta restaurant." In this way, personalized suggestions based on the user's individual emotional state can be made in real time.

[1030] Example prompt sentence:

[1031] The user opens their smartphone and types, "Please suggest what I should do next." This prompt allows the server to analyze their latest behavioral patterns and emotional state and notify the user of the optimal suggested action.

[1032] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1033] Step 1:

[1034] The user's device collects behavioral data such as location information, app usage history, time information, and sleep data. This data is obtained from sensors and devices such as the smartphone's GPS, application APIs, calendar functions, and sleep trackers. Input data includes location coordinates, a list of used apps, calendar events, and sleep duration, and this is output as the initial behavioral data.

[1035] Step 2:

[1036] The behavioral data collected by the device is encrypted and sent to the server via a secure communication protocol. The input data is the behavioral data collected in step 1, and is output as transmission data after going through the encryption process. Specifically, the data is uploaded in batches every hour.

[1037] Step 3:

[1038] The server stores the received data in a database for each user. This is where data cleaning is also performed. The input data is the data sent in step 2, and outliers and missing values ​​are processed before being stored in the database. The cleaned data is output.

[1039] Step 4:

[1040] The server cleans the received data and converts it into a format suitable for generative AI. Specifically, it normalizes the data, aligns timestamps, and performs feature engineering. The input data is data stored in a database, and the output is preprocessed training data.

[1041] Step 5:

[1042] The server uses the preprocessed data to train the generative AI to learn the user's behavioral patterns. A deep learning model is used here. The input data is the training data output in step 4, and the output is a trained model.

[1043] Step 6:

[1044] The action suggestion generation module uses the trained model to predict the next action to be taken in real time based on the user's behavioral patterns and generates action suggestions. New data points (e.g., current location or app usage status) are used as input, and action suggestions are generated as output. For example, a specific suggestion such as "Take a break at a coffee shop" is generated.

[1045] Step 7:

[1046] The server customizes the generated action suggestions and refers to the user's preferences and past behavioral history to provide more personalized suggestions. The input data are the action suggestions generated in step 6 and the user's preference data, and customized suggestions based on them are output.

[1047] Step 8:

[1048] The server notifies the user's device of the customized suggestion. The user's device visually displays the notification and informs the user of the suggested action. The input data is the customized suggestion output in step 7, and the notification data is output. A push notification is used for the specific notification.

[1049] Step 9:

[1050] The user selects a specific option for the proposed action and sends the selection to the server. The input data is the selection from the device, and this is output as the data to be sent to the server. Specifically, the selection is made using touch operations or the camera function.

[1051] Step 10:

[1052] The server uses an emotion engine that collects the user's emotional state and adjusts suggestions based on that state. Input data includes the user's facial expressions, voice, and text messages, and outputs the emotional state and adjustment data based on that state. If the emotional state is "stressed," the system will make suggestions such as "take a break in the relaxation area."

[1053] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1054] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1055] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1056] [Third embodiment]

[1057] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1058] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1059] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1060] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1061] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1062] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1063] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1064] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1065] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1066] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1067] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1068] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1069] The present invention is an action suggestion system for reducing "decision fatigue" in a user's daily life, and is implemented by the following means.

[1070] Overall system configuration

[1071] 1. Data Collection Module

[1072] Device: Your smartphone collects activity data, such as location, app usage, calendar events, and sleep data. This data is obtained from sensors and application APIs.

[1073] Example: Imagine a user commuting to work and turning on the GPS function on their smartphone, which collects location information every minute. It also records the launch time and usage time of each application they are using, as well as calendar event information.

[1074] 2. Data transmission module

[1075] On your device: We encrypt the activity data we collect and send it to our servers via a secure communication protocol (e.g., HTTPS).

[1076] Example: The acquired data is uploaded to a server in batch format at regular intervals (e.g., once an hour).

[1077] 3. Data receiving and storage module

[1078] Server: Receives data sent from the device and stores it in a database for each user. If necessary, it cleans the data (fills in missing values, corrects outliers, etc.).

[1079] Example: Data received on the server side is organized and saved in folders for each user. For example, user A's location data is stored as a dedicated table in the database.

[1080] 4. Data Learning Module

[1081] Server: Inputs the received data into the Generative AI, which learns the user's behavioral patterns. This Generative AI is trained using a deep learning model.

[1082] Example: Using location data and app usage history from the past month as training data, the app learns users' travel patterns and app usage trends.

[1083] 5. Action Proposal Generation Module

[1084] Server: Based on the learned data, predicts the next action to be taken and generates action suggestions. The generated suggestions are customized based on the user's preferences and past behavioral history.

[1085] Example: The server suggests "after-work dinner at a popular restaurant" on Friday evening, based on data about restaurants the user has frequently visited in the past.

[1086] 6. Action Suggestion Notification Module

[1087] Server: Sends customized suggestions to the user's smartphone app.

[1088] On the device: The smartphone app opens and notifies the user of the suggested action.

[1089] Example: Using the smartphone's push notification function, you can send a message to the user saying, "You can refresh yourself by jogging in a nearby park this Saturday."

[1090] 7. Choice Feedback Module

[1091] On the device: Presents specific options for suggested actions (e.g., photos of recipes or a list of activities) using a camera or touch interface.

[1092] User: Selects from the presented options by tapping, etc. The selected content is sent to the server and reflected in the learning of the generation AI.

[1093] Example: A user selects the suggested action "Go to a popular pasta restaurant" from the camera screen, and the selection is sent to the server.

[1094] Through the above-described embodiments, the present invention can reduce the decision-making burden in users' daily lives and prevent "decision fatigue." In particular, suggestions based on learned behavioral patterns are more tailored to individual users and provide higher accuracy and satisfaction than conventional general suggestion systems.

[1095] The processing flow will be explained below.

[1096] Step 1:

[1097] Activity Data Collection

[1098] Device: The user's smartphone collects location information, app usage history, calendar events, and sleep data, which are periodically retrieved from sensors and application APIs.

[1099] Specific operations: Turn on the GPS function to obtain location information, record the launch time and usage time of applications in use, extract schedule and event information from the calendar API, and obtain sleep data from a sleep tracker app.

[1100] Step 2:

[1101] Sending data to the server

[1102] On your device: Collected activity data is encrypted and sent to our servers using a secure communication protocol (e.g., HTTPS).

[1103] Specific behavior: Periodically checks the activity data stored in the local database, uploads any unsent data to the server in batches, encrypts the data using an encryption library, and sends an HTTP request.

[1104] Step 3:

[1105] Receiving and storing data

[1106] Server: Receives data sent from the device and stores it in a database for each user.

[1107] Specific operations: Receive data via API endpoint, organize the data by user ID, and save it in a database. Detect outliers and handle missing values ​​to check data integrity.

[1108] Step 4:

[1109] Data Preprocessing

[1110] Server: Cleans the received data (fills in missing values, handles outliers, etc.) and converts it into a format suitable for generative AI.

[1111] What it does: Normalize the data, sort it by time, and perform feature engineering if necessary, e.g., convert location information into geographic features and add categorical features based on time of day.

[1112] Step 5:

[1113] Generative AI training

[1114] Server: Trains the generative AI using preprocessed data to learn user behavior patterns.

[1115] Specific operation: Preprocessed data is input into the deep learning model, and it learns past behavioral patterns. During the training process, an appropriate learning rate and number of epochs are set, and adjustments are made while evaluating the model's performance.

[1116] Step 6:

[1117] Generate action suggestions

[1118] Server: Using the trained model, predict the user's next action in real time and generate action suggestions.

[1119] Specific Actions: New data points (e.g., current location or recent app usage history) are fed into the model to predict the next action, generating specific suggestions such as "get a coffee at a nearby cafe" or "relax at home."

[1120] Step 7:

[1121] Customize suggestions

[1122] Server: Customizes suggested actions based on the user's preferences and past behavior.

[1123] What it does: It applies personalization algorithms to generate recommendations that take into account the user's preferred activities and dining options. For example, it references a list of restaurants the user has visited in the past and makes new recommendations from there.

[1124] Step 8:

[1125] Submit your proposal

[1126] Server: Sends customized suggestions to the user's smartphone app.

[1127] Specific action: Using the push notification system, the suggested action is sent to the user's device in real time. For example, a notification message saying, "You should go see a movie at this theater right now" is sent.

[1128] Step 9:

[1129] View Suggestions

[1130] On the device: Display received action suggestions visually to the user, for example in the notification area or on an in-app dashboard.

[1131] What it does: Receive a push notification and display the suggestion in the app's UI, such as "Enjoy dinner at a popular restaurant nearby," in an easily accessible format for the user.

[1132] Step 10:

[1133] Presenting options

[1134] Device: Presents specific options for suggested actions (e.g., photos of dishes or a list of activities) and utilizes a camera or touch-enabled interface.

[1135] Specific operation: The camera function is used to scan the surrounding environment, and the presented options are displayed as images. The user taps to confirm the option.

[1136] Step 11:

[1137] Reflecting your selection

[1138] Device: The user's selection is sent to the server and reflected in future behavior predictions.

[1139] Specific actions: The user's selected actions are recorded on the device and sent to the server for the next learning cycle. The server then incorporates the received selections into the training data of the generative AI to improve the model's performance.

[1140] Example 1

[1141] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1142] Many modern users feel fatigued by the numerous choices and decisions they have to make in their daily lives, and are facing what is known as "decision fatigue." This "decision fatigue" reduces the quality of their daily lives and increases stress and dissatisfaction. This has led to a demand for systems that can provide personalized action suggestions and decision support for individual users. However, existing systems are unable to fully learn users' behavioral patterns, making it difficult to provide highly accurate suggestions.

[1143] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1144] In this invention, the server includes means for collecting activity data such as user location information, application usage history, calendar events, and sleep data, means for encrypting the activity data and transmitting it to the server via a secure communication protocol, means for receiving the activity data, storing it in a database for each user, and cleaning the data, means for inputting the activity data into a generative AI model and learning the user's behavioral patterns using a deep learning model, and means for suggesting the next action to be taken based on the learned behavioral patterns, thereby enabling highly accurate action suggestions based on the user's behavioral history.

[1145] "Location information" refers to data that indicates the user's current location and the route they are traveling.

[1146] "Application usage history" refers to data that records usage information of applications used by users on smartphones and other devices.

[1147] "Calendar events" refers to data that records a user's schedules and events on a calendar.

[1148] "Sleep data" refers to data that records the user's sleep time and sleep quality.

[1149] "Activity data" is a general term for data related to a user's daily activities, such as location information, application usage history, calendar events, and sleep data.

[1150] "Encryption" is a technology that converts data to transmit information securely and prevents unauthorized access and data leaks.

[1151] A "secure communication protocol" is a communication protocol for securely sending and receiving data, and a specific example is HTTPS.

[1152] "Data cleaning" is a process of improving data quality by filling in missing values ​​and correcting outliers.

[1153] A "generative AI model" is an artificial intelligence model that generates new information and suggestions based on data.

[1154] A "deep learning model" is a machine learning model that uses a multi-layer neural network to perform advanced learning and prediction.

[1155] A "behavioral pattern" is a series of data that indicates a user's daily behavior and tendencies.

[1156] "Action suggestions" are information or ideas that recommend the next action the user should take.

[1157] "Terminal" refers to a device that is directly operated by a user, such as a smartphone or tablet.

[1158] "Choices" are any list or item that allows a user to choose from multiple suggestions or options.

[1159] Overall system configuration

[1160] The present invention provides an action suggestion system for reducing "decision fatigue" in a user's daily life. This system is implemented by the following components and means.

[1161] Data Collection Module

[1162] Device: Your smartphone collects location information, application usage history, calendar events, sleep data, and other activity data. This is done using the smartphone's GPS, various sensors, and application APIs.

[1163] Example: When a user commutes to work, the GPS function on their smartphone is turned on, and location information is collected every minute. In addition, the launch time and usage time of each application being used are collected through the application API, as are events registered in the calendar.

[1164] Data Transmission Module

[1165] On your device: Collected activity data is encrypted and sent to a server via a secure communication protocol (e.g., HTTPS).

[1166] Example: The collected data is encrypted using the AES encryption algorithm and then sent to the server via HTTPS. The data is uploaded to the server in batches, for example, once an hour.

[1167] Data reception and storage module

[1168] Server: Receives data sent from the device and stores it in a database for each user. It also cleans the data as needed (filling in missing values, correcting outliers, etc.).

[1169] Example: Data received on the server side is organized and saved in a dedicated folder for each user. Before saving to the database, outliers are detected and corrected or deleted, and the data is saved as clean data. For example, the location information data of user A is stored in a dedicated table in the database.

[1170] Data Learning Module

[1171] Server: Inputs the received data into a generative AI model to learn user behavior patterns. This generative AI model is trained using a deep learning model.

[1172] Example: Location data and app usage history from the past month are input into a deep learning model as training data to learn the user's movement patterns and app usage trends. Specifically, CNN (convolutional neural network) and LSTM (long short-term memory) models are used.

[1173] Action proposal generation module

[1174] Server: Based on the learned data, the server proposes the next action to be taken. The proposed actions are customized based on the user's preferences and past behavioral history.

[1175] Example: For example, the server suggests "have dinner at a popular restaurant after work" on Friday evening. This suggestion is generated based on the data of restaurants the user has frequently visited in the past. Also, the server selects vegetarian restaurants as the user's preference.

[1176] Action Suggestion Notification Module

[1177] Server: Sends customized suggestions to the user's smartphone app.

[1178] On the device: The smartphone app opens and notifies the user of the suggested action.

[1179] Example: A smartphone app that receives a suggestion message from a server uses the push notification function to display a message to the user saying, "You can refresh yourself by jogging in a nearby park this Saturday."

[1180] Choice Feedback Module

[1181] On the device: Presents specific options for suggested actions (e.g., photos of recipes or a list of activities) using a camera or touch interface.

[1182] User: Selects from the presented options by tapping, etc. The selected content is sent to the server and reflected in the learning of the generation AI.

[1183] Example: A user selects a suggested action, such as "Go to a popular pasta restaurant," on the camera screen, and this information is sent to the server in real time, allowing the generative AI model to learn new data and provide more personalized recommendations the next time.

[1184] Examples of prompts for this system include "Predict the user's next move based on location data from the past month" and "Suggest activities for this weekend, taking into account the user's preferences."

[1185] Through these modules and processes, the present invention can significantly reduce the burden of decision-making in users' daily lives and effectively prevent "decision fatigue." By providing personalized recommendations, it increases satisfaction and achieves higher accuracy than conventional general recommendation systems.

[1186] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1187] Step 1:

[1188] Data collection

[1189] Device: Collects activity data such as user location, application usage history, calendar events, and sleep data. This data collection is done using the smartphone's GPS function, various sensors, and application APIs.

[1190] Input: Raw data collected from sensors on the user's smartphone and application APIs (location, application usage history, calendar events, sleep data).

[1191] Processing: Collecting data from sensors and APIs and organizing it into a specific format as activity data.

[1192] Output: Organized activity data.

[1193] What it does: When a user commutes to work, the GPS function on their smartphone is turned on, and location information is collected every minute, along with application launch times, usage time, and calendar events.

[1194] Step 2:

[1195] Data transmission

[1196] On your device: Collected activity data is encrypted and sent to a server via a secure communication protocol (e.g., HTTPS).

[1197] Input: Organized activity data.

[1198] Processing: The organized activity data is encrypted using the AES encryption algorithm and sent to the server via HTTPS.

[1199] Output: The encrypted data is sent to the server.

[1200] Specific operation: The collected data is periodically (for example, once an hour) encrypted in batch format and uploaded to the server via HTTPS communication.

[1201] Step 3:

[1202] Data reception and storage

[1203] Server: Receives data sent from the device and stores it in a database for each user. It also cleans the data (fills in missing values, corrects outliers, etc.).

[1204] Input: Encrypted activity data.

[1205] Processing: The encrypted data is decrypted, organized by user, and stored in a database. Data cleaning is performed.

[1206] Output: Cleaned per-user activity data.

[1207] Specific operations: Decrypts encrypted data on the server side and stores it in a folder for each user in the database. Detects and imputes outliers, and fills in missing values ​​to improve data quality.

[1208] Step 4:

[1209] Data Learning

[1210] Server: Inputs the received data into a generative AI model to learn user behavior patterns. This generative AI model is trained using a deep learning model.

[1211] Input: Cleaned per-user activity data.

[1212] Processing: The cleaned data is fed into a deep learning model to learn user behavior patterns.

[1213] Output: Learned behavioral pattern model.

[1214] How it works: The system inputs location data and application usage history from the past month into a deep learning model to learn the user's movement patterns and app usage trends, using deep learning techniques such as CNN and LSTM.

[1215] Step 5:

[1216] Action suggestion generation

[1217] Server: Based on the learned data, the server proposes the next action to be taken. The proposed actions are customized based on the user's preferences and past behavioral history.

[1218] Input: Learned behavioral pattern model.

[1219] Processing: Based on the learned model, next action suggestions are generated, which are customized based on the user's preferences and past behavioral history.

[1220] Output: Customized action suggestions.

[1221] Specific behavior: For example, the server may suggest "have dinner at a popular restaurant after work" on a Friday evening. This suggestion is generated based on data on restaurants the user has frequently visited in the past.

[1222] Step 6:

[1223] Action suggestion notification

[1224] Server: Sends customized suggestions to the user's smartphone app.

[1225] On the device: The smartphone app notifies the user of the suggested action.

[1226] Input: customized action suggestions.

[1227] Processing: Sends the proposed content to the smartphone app and notifies it.

[1228] Output: The user is notified of the suggestion.

[1229] Specific operation: The smartphone app receives a proposal message from the server and uses the push notification function to notify the user of the proposal.

[1230] Step 7:

[1231] Selection Feedback

[1232] On the device: Present specific options for the suggested action (e.g., photos of recipes or a list of activities) using a camera or touch interface.

[1233] User: Selects from the presented options by tapping, etc. The selected content is sent to the server and reflected in the learning of the generation AI.

[1234] Input: The user's selection.

[1235] Processing: The user's selection is sent to the server and reflected in the generative AI model.

[1236] Output: The updated training model.

[1237] What happens: The user taps to select the suggested action "Go to a popular pasta restaurant," which is then sent to the server, allowing the generative AI model to learn new data and provide more personalized recommendations the next time.

[1238] (Application example 1)

[1239] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1240] While modern users have access to a wide range of entertainment content, they often experience "decision fatigue" when trying to choose the best content from the diverse options. Under these circumstances, there is a need for systems that can appropriately suggest the next content to watch based on the user's preferences and behavioral history. Current systems are unable to effectively utilize the content and activity data of users' past viewings, making it difficult to make optimal suggestions for each user.

[1241] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1242] In this invention, the server includes means for collecting activity data such as a user's location information, app usage history, calendar events, and sleep data, means for transmitting the activity data to the server via a secure communication protocol, means for a generation AI to learn the user's behavioral patterns using a deep learning model based on the activity data, means for suggesting entertainment content to be viewed next based on the behavioral patterns, means for notifying the user's smartphone of the suggested content, and means for reflecting the user's selection in the learning, thereby enabling users to easily select entertainment content appropriately customized based on their preferences and behavioral history.

[1243] "Location information" is data that indicates a user's current location and is obtained using GPS or other location information technology.

[1244] "App usage history" is a record of applications used by a user on a smartphone or other device, and includes data such as the time the app was launched and the amount of time it was used.

[1245] "Calendar events" are information about events and schedules that a user registers in a calendar to manage their plans and appointments.

[1246] "Sleep data" refers to data that indicates the user's sleep time and quality, and is collected via smartphones or wearable devices.

[1247] "Activity data" is data that records various actions and conditions of a user's daily life, and includes location information, app usage history, calendar events, sleep data, and more.

[1248] A "secure communication protocol" is a safe means of communication that prevents eavesdropping or tampering by third parties when sending and receiving data, and examples include protocols such as HTTPS.

[1249] "Generative AI" is artificial intelligence that uses machine learning techniques such as deep learning to train user data to perform specific tasks.

[1250] A "deep learning model" is a type of artificial intelligence that uses multi-layered neural networks to learn complex patterns from data.

[1251] A "behavioral pattern" indicates the tendencies and habits of the user's daily behavior, and is determined based on past data.

[1252] "Entertainment content" refers to media such as movies, dramas, music, podcasts, etc. that users consume for entertainment purposes.

[1253] A "smartphone" is a multi-function mobile phone that is a device that can not only make calls but also access the Internet and use applications.

[1254] "Push notification" is a function that notifies information to smartphones and other devices in real time.

[1255] A "selection" refers to the particular item a user chooses from among the options offered.

[1256] "Means for reflecting in learning" refers to a method for feeding back user choices to the generative AI model to improve the accuracy of subsequent suggestions.

[1257] The present invention is an action suggestion system for reducing "decision fatigue" in a user's daily life, and specific implementation means are described below.

[1258] Data Collection Module

[1259] The device used is the user's smartphone, which collects location information, app usage history, calendar events, sleep data, and other activity data. This data is obtained through the smartphone's built-in sensors and application APIs.

[1260] Data Transmission Module

[1261] Activity data collected on the device is sent to a server using a secure communication protocol (e.g., HTTPS). The data is encrypted and uploaded in batches at regular intervals (e.g., once an hour).

[1262] Data reception and storage module

[1263] The server side receives the transmitted data and stores it in a database for each user. Data cleansing (filling in missing values, correcting outliers, etc.) is also performed. For example, SQLite is used as the database.

[1264] Data Learning Module

[1265] Based on the data received on the server, the generated AI uses a deep learning model to learn user behavior patterns. To do this, it uses deep learning libraries such as TensorFlow. It uses past location data and app usage history as training data to learn user patterns and tendencies.

[1266] Action proposal generation module

[1267] The server uses the learned data to predict the next entertainment content to watch and generates action suggestions, which are customized based on the user's preferences and past behavioral history.

[1268] Action Suggestion Notification Module

[1269] The generated action suggestions are sent to the user's smartphone using the push notification function, and the notification content includes specific entertainment content such as "the next movie to watch."

[1270] Choice Feedback Module

[1271] The user taps to select an option from the suggested content. This selection is sent to the server and reflected in the learning of the generative AI. The suggested options can also be visually displayed using the smartphone's camera function.

[1272] Specific examples

[1273] For example, at 8 p.m. on a weekday while a user is relaxing at home, a push notification on their smartphone suggests a movie to watch next based on their past viewing history and activity data. This suggestion specifically suggests movies in a genre that the user previously enjoyed.

[1274] Example prompt sentence:

[1275] "Below is your past viewing history and recent activity data. Based on this data, we'd like to suggest what you should watch next."

[1276] Viewing history: [Movie A, Drama B, Music C...]

[1277] Latest activity data: {Location: Home, Time: 20:00, Date: 2023-10-02}

[1278] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1279] Step 1:

[1280] The device (user's smartphone) collects activity data such as location information, app usage history, calendar events, and sleep data.

[1281] Input: Data obtained through smartphone sensors and APIs.

[1282] Specific operation: Using the smartphone's GPS, it collects location information every minute and also periodically records app usage history and calendar events.

[1283] Output: Collected activity data (location, app usage history, etc.).

[1284] Step 2:

[1285] The device transmits the collected activity data to the server via a secure communication protocol (e.g., HTTPS).

[1286] Input: Collected activity data.

[1287] What it does: It encrypts your activity data and uploads it to a server in batches using the HTTPS protocol, for example, every hour.

[1288] Output: Activity data sent to the server.

[1289] Step 3:

[1290] The server receives the data and stores it in a database. It also performs data cleansing, such as filling in missing values ​​and correcting outliers.

[1291] Input: Submitted activity data.

[1292] Specific operation: The received data is organized and stored in a database for each user, outliers are detected and corrected, and missing values ​​are complemented.

[1293] Output: Saved and cleansed activity data.

[1294] Step 4:

[1295] The server uses generative AI (deep learning model) to learn the user's behavioral patterns based on the stored activity data.

[1296] Input: Saved activity data.

[1297] Specific operation: Using a deep learning library (such as TensorFlow), the model is trained using past location information and app usage history as training data. The following is an example of a prompt sentence to input.

[1298] Example prompt: "Below is the user's past viewing history and latest activity data. Based on this data, please suggest the next content they should watch." Viewing history: [Movie A, Drama B, Music C...] Latest activity data: {Location: Home, Time: 20:00, Date: 2023-10-02}

[1299] Output: An AI model that has learned the user's behavioral patterns.

[1300] Step 5:

[1301] The server uses the learned AI model to predict the next entertainment content to watch and generate action suggestions.

[1302] Input: An AI model that has learned your behavioral patterns, and your latest activity data.

[1303] What it does: It uses AI models to predict the best entertainment content for users based on their current situation and past behavioral patterns.

[1304] Output: Suggestions about what entertainment content to watch next.

[1305] Step 6:

[1306] The server sends the generated action suggestions to the smartphone via push notification.

[1307] Input: Suggestions for entertainment content to watch next.

[1308] Specific operation: Using a push notification library (e.g., Firebase Cloud Messaging), the suggestion content is sent to the user's smartphone.

[1309] Output: Action suggestions displayed on the user's smartphone.

[1310] Step 7:

[1311] Users tap to select from the suggested entertainment content, and the selected content is sent to the server and reflected in the learning of the generative AI.

[1312] Input: User-selected entertainment content.

[1313] Specific operation: Using the smartphone's camera function and touch operation interface, the proposed options are visually displayed and the user's tap selection information is collected. The selection information is then sent back to the server and used to update the generative AI model.

[1314] Output: The user's choices fed back to the generative AI model.

[1315] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1316] The present invention is an action suggestion system for reducing "decision fatigue" in a user's daily life, and further improves the accuracy and satisfaction of the suggestions by incorporating an emotion engine that recognizes and responds to the user's emotional state. The present invention is implemented by the following means.

[1317] Overall system configuration

[1318] 1. Data Collection Module

[1319] Device: Your smartphone collects activity data, such as location, app usage history, calendar events, and sleep data. This data is periodically retrieved from sensors and application APIs.

[1320] Example: A user turns on the GPS on their smartphone during their commute to collect location information, record app usage history, calendar events, and data from a sleep tracker.

[1321] 2. Data transmission module

[1322] On your device: Collected activity data is encrypted and sent to a server via a secure communication protocol.

[1323] Example: The acquired data is uploaded to the server in batches every hour.

[1324] 3. Data receiving and storage module

[1325] Server: Stores the received data in a database for each user, and cleans the data if necessary.

[1326] Example: Organize data by user ID on the server side, process outliers and missing values, and then save it to the database.

[1327] 4. Data Preprocessing Module

[1328] Server: Cleans the received data and converts it into a format suitable for the generative AI.

[1329] Examples: data normalization, timestamp alignment, feature engineering.

[1330] 5. Generative AI Learning Module

[1331] Server: Trains the generative AI using preprocessed data to learn user behavior patterns.

[1332] Example: Using past location information and app usage history as training data to recognize users' travel patterns and app usage tendencies.

[1333] 6. Action Proposal Generation Module

[1334] Server: Uses the trained model to predict the user's next action in real time and generate action suggestions.

[1335] Example: Feed new data points into the model to generate specific suggestions, such as "have a coffee at a nearby cafe."

[1336] 7. Proposal customization module

[1337] Server: Customizes suggested actions based on the user's preferences and past behavior.

[1338] Example: Referencing data on restaurants the user has visited in the past and suggesting "dinner at a popular restaurant after work."

[1339] 8. Action Suggestion Notification Module

[1340] Server: Sends customized suggestions to the user's smartphone app.

[1341] Terminal: Visually notify the user of the proposed action.

[1342] Example: Use a push notification to say, "You can refresh yourself by jogging in a nearby park this Saturday."

[1343] 9. Choice Feedback Module

[1344] Device: Presents specific options for suggested actions using a camera or touch interface.

[1345] User: Tap to select a presented option. The selection is sent to the server and reflected in the learning of the generation AI.

[1346] Example: A user selects the option "Go to a popular pasta restaurant" from a camera screen, and the selection is sent to a server.

[1347] 10. Emotion Engine Module

[1348] Device: Recognizes the user's emotional state by analyzing facial expressions, voice, and text messages.

[1349] Example: Using a smartphone's camera and microphone, it analyzes emotions from the user's facial expressions and tone of voice, as well as the content of text messages.

[1350] 11. Emotion-based behavior suggestion adjustment module

[1351] Server: Adjusts the recommendations based on the results from the emotion engine.

[1352] Example: If the user is feeling stressed, suggest a relaxing activity (e.g., yoga or meditation).

[1353] Through the above-described embodiments, the present invention reduces the burden of decision-making in the user's daily life, and furthermore, by using an emotion engine, it is possible to realize optimal action suggestions that take into account the user's mental state, thereby providing higher accuracy and satisfaction than conventional systems.

[1354] The processing flow will be explained below.

[1355] Step 1:

[1356] Activity Data Collection

[1357] Device: Your smartphone collects activity data, such as location, app usage history, calendar events, and sleep data. This data is periodically retrieved from sensors and application APIs.

[1358] Specific operations: Turn on the GPS function to obtain location information, record the launch time and usage time of each application in use, extract schedule and event information from the calendar API, and obtain sleep data from a sleep tracker app.

[1359] Step 2:

[1360] Sending data to the server

[1361] On your device: We encrypt the activity data we collect and send it to our servers using a secure communication protocol (e.g., HTTPS).

[1362] Specific behavior: Periodically checks the activity data stored in the local database, uploads any unsent data to the server in batches, encrypts the data using an encryption library, and sends an HTTP request.

[1363] Step 3:

[1364] Receiving and storing data

[1365] Server: Receives data sent from the device and stores it in a database for each user.

[1366] Specific operations: Receive data via API endpoint, organize the data by user ID, and save it in a database. Detect outliers and handle missing values ​​to check data integrity.

[1367] Step 4:

[1368] Data Preprocessing

[1369] Server: Cleans the received data (fills in missing values, handles outliers, etc.) and converts it into a format suitable for generative AI.

[1370] What it does: Normalizes the data, aligns it on the time axis, and performs feature engineering as needed, e.g., converting location information into geographic features and adding categorical features based on time of day.

[1371] Step 5:

[1372] Generative AI training

[1373] Server: Trains the generative AI using preprocessed data to learn user behavior patterns.

[1374] Specific operation: Preprocessed data is input into the deep learning model, and it learns past behavioral patterns. During the training process, an appropriate learning rate and number of epochs are set, and adjustments are made while evaluating the model's performance.

[1375] Step 6:

[1376] Generate action suggestions

[1377] Server: Using the trained model, predict the user's next action in real time and generate action suggestions.

[1378] Specific Actions: New data points (e.g., current location or recent app usage history) are fed into the model to predict the next action, generating specific suggestions such as "get a coffee at a nearby cafe" or "relax at home."

[1379] Step 7:

[1380] Customize suggestions

[1381] Server: Customizes suggested actions based on the user's preferences and past behavior.

[1382] What it does: It applies personalization algorithms to generate recommendations that take into account the user's preferred activities and dining options. For example, it references a list of restaurants the user has visited in the past and makes new recommendations from there.

[1383] Step 8:

[1384] Submit your proposal

[1385] Server: Sends customized suggestions to the user's smartphone app.

[1386] Specific action: Using the push notification system, the suggested action is sent to the user's device in real time. For example, a notification message saying, "You should go see a movie at this theater right now" is sent.

[1387] Step 9:

[1388] View Suggestions

[1389] On the device: Display received action suggestions visually to the user, for example in the notification area or on an in-app dashboard.

[1390] What it does: Receive a push notification and display the suggestion in the app's UI, for example, "Enjoy dinner at a popular restaurant nearby," in a format that's easily accessible to the user.

[1391] Step 10:

[1392] Presenting options

[1393] Device: Presents specific options for suggested actions (e.g., photos of dishes or a list of activities) and utilizes a camera or touch-enabled interface.

[1394] Specific operation: The camera function is used to scan the surrounding environment, and the presented options are displayed as images. The user taps to confirm the option.

[1395] Step 11:

[1396] Reflecting your selection

[1397] Device: The user's selection is sent to the server and reflected in future behavior predictions.

[1398] Specific actions: The user's selected actions are recorded on the device and sent to the server for the next learning cycle. The server then incorporates the received selections into the training data of the generative AI to improve the model's performance.

[1399] Step 12:

[1400] Collecting Emotional Data

[1401] Device: Recognizes the user's emotional state by analyzing facial expressions, voice, and text messages.

[1402] Specific operation: Using the smartphone's camera and microphone, the system analyzes the user's facial expressions and tone of voice, and uses natural language processing to analyze the content of text messages to determine their emotional state.

[1403] Step 13:

[1404] Sending emotional data

[1405] Terminal: Sends analyzed emotion data to the server.

[1406] Specific operation: The emotion data output by the emotion recognition engine is encrypted and sent to the server using a secure communication protocol.

[1407] Step 14:

[1408] Tailoring behavioral suggestions based on emotions

[1409] Server: Adjusts the recommendations based on the results from the emotion engine.

[1410] Specific actions: If the user is feeling stressed, it will suggest relaxing actions (e.g., yoga or meditation). Conversely, if the user is feeling excited, it will suggest active actions (e.g., sports or dancing).

[1411] Through the above-described embodiments, the present invention reduces the burden of decision-making in the user's daily life, and furthermore, by using an emotion engine, it is possible to realize optimal action suggestions that take into account the user's mental state, thereby providing higher accuracy and satisfaction than conventional systems.

[1412] Example 2

[1413] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1414] While current technology exists to reduce the burden of decision-making in users' daily lives, these systems are unable to take into account their emotional state when making suggestions, which can result in low satisfaction. To solve this problem, it is necessary to provide a system that recognizes the user's emotional state and suggests optimal actions.

[1415] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting activity data such as the user's location information, app usage history, calendar events, and sleep data, means for encrypting the activity data and transmitting it to the server, means for a generation AI to learn the user's behavioral patterns based on the activity data, means for suggesting a next action to be taken based on the behavioral patterns, means for recognizing the user's emotional state using an emotion engine and adjusting the suggestion content, means for notifying the user's terminal of the suggested action, and means for reflecting the user's selection in the learning. This reduces the burden of decision-making in the user's daily life and enables highly accurate and satisfying action suggestions that take the user's emotional state into consideration.

[1416] "User location information" is geographical data that indicates where the user is currently located.

[1417] "Application usage history" is data that records the names of applications used by a user, the duration of use, frequency of use, and so on.

[1418] A "calendar event" is event information entered into a calendar application that registers a user's plans and schedules.

[1419] "Sleep data" refers to data that records the user's sleep patterns, sleep duration, quality, etc.

[1420] "Activity data" refers to all data related to daily life activities, such as a user's location information, app usage history, calendar events, and sleep data.

[1421] "Encryption" means converting data into a form that is unintelligible to others using a specific algorithm.

[1422] "Generative AI" is an artificial intelligence model that can learn a user's behavioral patterns and suggest the next action to take.

[1423] A "behavioral pattern" refers to a series of actions and habits in a user's daily life.

[1424] The "emotion engine" is an engine that analyzes and identifies the user's emotional state from their facial expressions, voice, and text.

[1425] "Suggestion content" refers to a suggested action that the generation AI generates by taking into account the user's behavioral patterns and emotional state.

[1426] "Notification" refers to displaying the generated action suggestion on the user's terminal.

[1427] "Selection" refers to the item that the user selects from the suggested actions.

[1428] "Reflecting in learning" means incorporating user selections into the generative AI model to improve the accuracy of future action suggestions.

[1429] This invention is a system for reducing the burden of decision-making in a user's daily life and for suggesting optimal actions that take into account the user's emotional state.

[1430] System configuration

[1431] Data Collection Module

[1432] The user's device, specifically their smartphone, is used to collect activity data such as location information, app usage history, calendar events, and sleep data, which are periodically retrieved through the device's sensors and application APIs.

[1433] Example: A user turns on the GPS on their smartphone during their commute to collect location information, record app usage history, calendar events, and data from a sleep tracker.

[1434] Data Transmission Module

[1435] Activity data collected on the device is encrypted and sent to a server via a secure communication protocol (e.g., HTTPS).

[1436] Example: The collected data is AES encrypted in batches every hour and uploaded to the server via HTTPS protocol.

[1437] Data reception and storage module

[1438] The server stores the received data in a database for each user. Before storing, it cleans the data (dealing with outliers and imputing missing values) as necessary.

[1439] Example: The data received by the server is organized into a database by user ID, and outliers and missing values ​​are automatically processed.

[1440] Data Preprocessing Module

[1441] The server cleans the received data and converts it into a format suitable for generative AI, including normalizing the data, aligning timestamps, and extracting features.

[1442] Example: Prepare a training dataset by properly aligning timestamps, normalizing numeric data, and removing unnecessary fields.

[1443] Generative AI learning module

[1444] The server uses the preprocessed data to train a generative AI to learn the user's behavioral patterns. This generative AI is trained using a deep learning framework (e.g., TensorFlow, PyTorch).

[1445] Example: Using past location information and app usage history to learn about users' travel patterns and app usage habits.

[1446] Action proposal generation module

[1447] The server uses a trained generative AI model to predict the user's next action in real time and generate action suggestions.

[1448] Example: Newly collected data points are fed into a generative AI model to generate specific suggestions, such as "It's almost lunchtime, so suggest a nearby cafe."

[1449] Suggestion customization module

[1450] The server customizes the generated suggestions based on the user's preferences and past behavioral history.

[1451] Example: Referencing data on restaurants frequently visited by the user to provide customized suggestions such as "dinner at a popular restaurant after work."

[1452] Action Suggestion Notification Module

[1453] The server sends the customized action suggestions to the user's smartphone app, which then visually notifies the user of the suggested action.

[1454] Example: Use push notifications to notify users of something like, "You can refresh yourself by jogging in a nearby park this Saturday."

[1455] Choice Feedback Module

[1456] The device uses a camera or touch interface to present specific options for the proposed action. The user taps to select the option. The selection is sent to the server and reflected in the learning of the generative AI.

[1457] Example: A user selects the option "Go to a popular pasta restaurant" from a camera screen, and the selection is sent to a server.

[1458] Emotion Engine Module

[1459] The device recognizes the user's emotional state by analyzing their facial expressions, voice, and text messages.

[1460] Example: Using the smartphone camera and microphone, the system analyzes emotions from the user's facial expressions and tone of voice, including text messages.

[1461] Emotion-based behavior suggestion adjustment module

[1462] The server adjusts the suggestions based on the emotional state obtained from the emotion engine.

[1463] Example: If the user is feeling stressed, suggest a relaxing activity (e.g., meditation, yoga).

[1464] Examples of prompts for generative AI models

[1465] "Generate optimal action suggestions based on the user's current location, app usage history, calendar events, and sleep data."

[1466] This invention can reduce the burden of decision-making in the user's daily life and realize highly accurate and satisfying action suggestions that also take into account the user's emotional state.

[1467] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1468] Step 1:

[1469] Data collection

[1470] Input: Activity data about the user's daily life (location, app usage history, calendar events, sleep data).

[1471] Specific operation: The device (smartphone) acquires location information using GPS, records application usage history in the background, obtains the user's schedule from a calendar app, and collects sleep data from a sleep tracker app.

[1472] Output: Collected activity data.

[1473] Step 2:

[1474] Data transmission

[1475] Input: Collected activity data.

[1476] Specific operation: The device encrypts the collected data using AES and sends it to the server using a secure communication protocol (HTTPS).

[1477] Output: The encrypted data is sent to the server.

[1478] Step 3:

[1479] Data reception and storage

[1480] Input: Encrypted data.

[1481] Specific operation: The server receives the data, unlocks it, and then stores it in a database for each user ID. At that time, it cleans the data by removing outliers and filling in missing values.

[1482] Output: An organized database.

[1483] Step 4:

[1484] Data Preprocessing

[1485] Input: A curated database.

[1486] Specific operations: The server normalizes the data, aligns timestamps, removes unnecessary fields, extracts features, and converts it into a format suitable for generative AI.

[1487] Output: The preprocessed dataset.

[1488] Step 5:

[1489] Generative AI training

[1490] Input: The preprocessed dataset.

[1491] How it works: The server trains the generative AI using a deep learning framework (e.g., TensorFlow, PyTorch), learning the dataset over multiple epochs and incorporating user behavior patterns into the model.

[1492] Output: The trained model.

[1493] Step 6:

[1494] Action suggestion generation

[1495] Input: Newly collected data points, trained model.

[1496] What it does: The server uses the trained model to analyze new data points collected and predict the next action to take. It generates suggestions based on prompts (e.g., "Generate optimal action suggestions based on the current user's location, app usage history, calendar events, and sleep data.").

[1497] Output: Proposal.

[1498] Step 7:

[1499] Customize suggestions

[1500] Input: Proposal content, past behavior history.

[1501] Specific operation: The server customizes the generated suggestions based on the user's preferences and past behavior history, taking into account the places visited in the past and preferred activities to provide optimal suggestions.

[1502] Output: A customized proposal.

[1503] Step 8:

[1504] Action suggestion notification

[1505] Input: Your customized suggestion.

[1506] Specific operation: The server uses the push notification API to send customized suggestions to the device, which then visually notifies the user of the suggested action.

[1507] Output: The suggestion that was communicated to the user.

[1508] Step 9:

[1509] Selection Feedback

[1510] Input: User choices, camera and touch interfaces.

[1511] Specific actions: The device visually displays specific options for the proposed action, and the user taps to select it. The user's selection is sent to the server and reflected in the learning of the generative AI.

[1512] Output: The selections sent to the server.

[1513] Step 10:

[1514] Emotion Engine

[1515] Input: User facial expressions, voice, and text messages.

[1516] How it works: The device uses the smartphone's camera and microphone to collect the user's facial expressions and voice, and analyzes emotions from text messages.

[1517] Output: Emotional state data.

[1518] Step 11:

[1519] Emotion-based behavioral adjustment

[1520] Input: Emotional state data, customized suggestions.

[1521] Specific operation: The server further adjusts the suggested actions based on the emotional state obtained from the emotion engine. If the user is feeling stressed, the server changes the suggested actions to suggest relaxing activities.

[1522] Output: Final recommendations based on the user's emotional state.

[1523] (Application example 2)

[1524] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1525] In recent years, the proliferation of options in consumers' daily lives has led to the problem of "decision fatigue." Particularly in brick-and-mortar stores, consumers are forced to make optimal choices from a vast array of products and services, placing a psychological burden on them. Furthermore, because optimal recommendations vary depending on a consumer's emotional state, there is a need for a system that can analyze each individual's emotional state and adjust the recommendations accordingly.

[1526] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting behavioral data such as user location information, app usage history, time information, and sleep data; means for transmitting the behavioral data to the server; means for a generation AI to learn the user's behavioral patterns based on the behavioral data; means for suggesting the next action to be taken based on the behavioral patterns; means for notifying the user's device of the suggested action and making a purchase suggestion; means for reflecting the user's selection in the learning; and means including an emotion engine for collecting the user's emotional state and adjusting the suggestion content. This reduces the decision-making burden in consumers' daily lives and makes it possible to provide personalized suggestions based on each individual's emotional state in real time.

[1527] "User location information" is data that identifies the user's current geographic location.

[1528] "Application usage history" is data on applications that a user has used in the past.

[1529] "Time information" is data relating to the time and date when the user performed an action.

[1530] "Sleep data" is information related to the user's sleep, including the amount of sleep time and the quality of sleep.

[1531] "Behavioral data" is a general term for data including location information, app usage history, time information, sleep data, and the like in a user's daily life.

[1532] "Generative AI" is an artificial intelligence that learns user behavior patterns based on collected data and generates new suggestions.

[1533] "Behavioral patterns" refer to tendencies and habits extracted from a user's past behavior.

[1534] The "means for suggesting an action" is a method for indicating the next action to be taken based on the user's behavioral patterns.

[1535] A "purchase suggestion" is a suggestion that recommends specific products or services related to consumption behavior to a user.

[1536] The "emotion engine" is a function that analyzes the user's emotional state and adjusts the content of suggestions based on that state.

[1537] "Personalized offers" are individual offers tailored to a user's individual interests, preferences, and emotional state.

[1538] "Server" means a computer system that receives, stores, analyzes, and generates recommendations from user-submitted data.

[1539] This invention is an action suggestion system designed to reduce "decision fatigue" in users' daily lives. It also incorporates an emotion engine that recognizes and responds to the user's emotional state, improving the accuracy of the suggestions and satisfaction. This system collects behavioral data, such as the user's location, app usage history, time information, and sleep data, and uses this data to generate AI that learns the user's behavioral patterns. Based on the behavioral patterns, the system then suggests the next action to be taken and notifies the user's device of the suggested action.

[1540] First, user behavior data is collected from devices such as smartphones. Specifically, this includes location information using GPS, application usage history, calendar events, and data from sleep trackers. This data is collected in real time or periodically and sent to a server. The data is encrypted and transmitted via a secure communication protocol.

[1541] The server stores the received behavioral data in a database, cleans the data, and then performs preprocessing to train the generative AI model. Specifically, it normalizes the data, aligns timestamps, and performs feature engineering. The preprocessed data is then used to train the generative AI, which learns the user's behavioral patterns. This AI model uses deep learning technology.

[1542] The action suggestion generation module uses a trained generative AI model to predict the next action a user should take in real time based on their behavioral patterns, generating action suggestions. For example, if the user is in a physical store, suggestions such as "take a break at a coffee shop" or "check for new product sales" will be made. These suggestions are further refined by an emotion engine that analyzes the user's emotional state. Specifically, the system analyzes the user's facial expressions and voice using the smartphone's camera and microphone, as well as the content of text messages.

[1543] The proposed actions, including purchasing suggestions, are sent to the user's device. The user selects from the suggested options using touch operations on their smartphone, and the selection is then sent back to the server. This allows the generative AI model to constantly retrain based on the latest data, improving the accuracy of the suggestions.

[1544] For example, if the system detects that the user is feeling stressed, it will suggest "take a break in the relaxation area." If the user is feeling normal, it will suggest "try your favorite pasta restaurant." In this way, personalized suggestions based on the user's individual emotional state can be made in real time.

[1545] Example prompt sentence:

[1546] The user opens their smartphone and types, "Please suggest what I should do next." This prompt allows the server to analyze their latest behavioral patterns and emotional state and notify the user of the optimal suggested action.

[1547] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1548] Step 1:

[1549] The user's device collects behavioral data such as location information, app usage history, time information, and sleep data. This data is obtained from sensors and devices such as the smartphone's GPS, application APIs, calendar functions, and sleep trackers. Input data includes location coordinates, a list of used apps, calendar events, and sleep duration, and this is output as the initial behavioral data.

[1550] Step 2:

[1551] The behavioral data collected by the device is encrypted and sent to the server via a secure communication protocol. The input data is the behavioral data collected in step 1, and is output as transmission data after going through the encryption process. Specifically, the data is uploaded in batches every hour.

[1552] Step 3:

[1553] The server stores the received data in a database for each user. This is where data cleaning is also performed. The input data is the data sent in step 2, and outliers and missing values ​​are processed before being stored in the database. The cleaned data is output.

[1554] Step 4:

[1555] The server cleans the received data and converts it into a format suitable for generative AI. Specifically, it normalizes the data, aligns timestamps, and performs feature engineering. The input data is data stored in a database, and the output is preprocessed training data.

[1556] Step 5:

[1557] The server uses the preprocessed data to train the generative AI to learn the user's behavioral patterns. A deep learning model is used here. The input data is the training data output in step 4, and the output is a trained model.

[1558] Step 6:

[1559] The action suggestion generation module uses the trained model to predict the next action to be taken in real time based on the user's behavioral patterns and generates action suggestions. New data points (e.g., current location or app usage status) are used as input, and action suggestions are generated as output. For example, a specific suggestion such as "Take a break at a coffee shop" is generated.

[1560] Step 7:

[1561] The server customizes the generated action suggestions and refers to the user's preferences and past behavioral history to provide more personalized suggestions. The input data are the action suggestions generated in step 6 and the user's preference data, and customized suggestions based on them are output.

[1562] Step 8:

[1563] The server notifies the user's device of the customized suggestion. The user's device visually displays the notification and informs the user of the suggested action. The input data is the customized suggestion output in step 7, and the notification data is output. A push notification is used for the specific notification.

[1564] Step 9:

[1565] The user selects a specific option for the proposed action and sends the selection to the server. The input data is the selection from the device, and this is output as the data to be sent to the server. Specifically, the selection is made using touch operations or the camera function.

[1566] Step 10:

[1567] The server uses an emotion engine that collects the user's emotional state and adjusts suggestions based on that state. Input data includes the user's facial expressions, voice, and text messages, and outputs the emotional state and adjustment data based on that state. If the emotional state is "stressed," the system will make suggestions such as "take a break in the relaxation area."

[1568] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1569] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1570] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1571] [Fourth embodiment]

[1572] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1573] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1574] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1575] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1576] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1577] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1578] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1579] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1580] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1581] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1582] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1583] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1584] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1585] The present invention is an action suggestion system for reducing "decision fatigue" in a user's daily life, and is implemented by the following means.

[1586] Overall system configuration

[1587] 1. Data Collection Module

[1588] Device: Your smartphone collects activity data, such as location, app usage, calendar events, and sleep data. This data is obtained from sensors and application APIs.

[1589] Example: Imagine a user commuting to work and turning on the GPS function on their smartphone, which collects location information every minute. It also records the launch time and usage time of each application they are using, as well as calendar event information.

[1590] 2. Data transmission module

[1591] On your device: We encrypt the activity data we collect and send it to our servers via a secure communication protocol (e.g., HTTPS).

[1592] Example: The acquired data is uploaded to a server in batch format at regular intervals (e.g., once an hour).

[1593] 3. Data receiving and storage module

[1594] Server: Receives data sent from the device and stores it in a database for each user. If necessary, it cleans the data (fills in missing values, corrects outliers, etc.).

[1595] Example: Data received on the server side is organized and saved in folders for each user. For example, user A's location data is stored as a dedicated table in the database.

[1596] 4. Data Learning Module

[1597] Server: Inputs the received data into the Generative AI, which learns the user's behavioral patterns. This Generative AI is trained using a deep learning model.

[1598] Example: Using location data and app usage history from the past month as training data, the app learns users' travel patterns and app usage trends.

[1599] 5. Action Proposal Generation Module

[1600] Server: Based on the learned data, predicts the next action to be taken and generates action suggestions. The generated suggestions are customized based on the user's preferences and past behavioral history.

[1601] Example: The server suggests "after-work dinner at a popular restaurant" on Friday evening, based on data about restaurants the user has frequently visited in the past.

[1602] 6. Action Suggestion Notification Module

[1603] Server: Sends customized suggestions to the user's smartphone app.

[1604] On the device: The smartphone app opens and notifies the user of the suggested action.

[1605] Example: Using the smartphone's push notification function, you can send a message to the user saying, "You can refresh yourself by jogging in a nearby park this Saturday."

[1606] 7. Choice Feedback Module

[1607] On the device: Presents specific options for suggested actions (e.g., photos of recipes or a list of activities) using a camera or touch interface.

[1608] User: Selects from the presented options by tapping, etc. The selected content is sent to the server and reflected in the learning of the generation AI.

[1609] Example: A user selects the suggested action "Go to a popular pasta restaurant" from the camera screen, and the selection is sent to the server.

[1610] Through the above-described embodiments, the present invention can reduce the decision-making burden in users' daily lives and prevent "decision fatigue." In particular, suggestions based on learned behavioral patterns are more tailored to individual users and provide higher accuracy and satisfaction than conventional general suggestion systems.

[1611] The processing flow will be explained below.

[1612] Step 1:

[1613] Activity Data Collection

[1614] Device: The user's smartphone collects location information, app usage history, calendar events, and sleep data, which are periodically retrieved from sensors and application APIs.

[1615] Specific operations: Turn on the GPS function to obtain location information, record the launch time and usage time of applications in use, extract schedule and event information from the calendar API, and obtain sleep data from a sleep tracker app.

[1616] Step 2:

[1617] Sending data to the server

[1618] On your device: Collected activity data is encrypted and sent to our servers using a secure communication protocol (e.g., HTTPS).

[1619] Specific behavior: Periodically checks the activity data stored in the local database, uploads any unsent data to the server in batches, encrypts the data using an encryption library, and sends an HTTP request.

[1620] Step 3:

[1621] Receiving and storing data

[1622] Server: Receives data sent from the device and stores it in a database for each user.

[1623] Specific operations: Receive data via API endpoint, organize the data by user ID, and save it in a database. Detect outliers and handle missing values ​​to check data integrity.

[1624] Step 4:

[1625] Data Preprocessing

[1626] Server: Cleans the received data (fills in missing values, handles outliers, etc.) and converts it into a format suitable for generative AI.

[1627] What it does: Normalize the data, sort it by time, and perform feature engineering if necessary, e.g., convert location information into geographic features and add categorical features based on time of day.

[1628] Step 5:

[1629] Generative AI training

[1630] Server: Trains the generative AI using preprocessed data to learn user behavior patterns.

[1631] Specific operation: Preprocessed data is input into the deep learning model, and it learns past behavioral patterns. During the training process, an appropriate learning rate and number of epochs are set, and adjustments are made while evaluating the model's performance.

[1632] Step 6:

[1633] Generate action suggestions

[1634] Server: Using the trained model, predict the user's next action in real time and generate action suggestions.

[1635] Specific Actions: New data points (e.g., current location or recent app usage history) are fed into the model to predict the next action, generating specific suggestions such as "get a coffee at a nearby cafe" or "relax at home."

[1636] Step 7:

[1637] Customize suggestions

[1638] Server: Customizes suggested actions based on the user's preferences and past behavior.

[1639] What it does: It applies personalization algorithms to generate recommendations that take into account the user's preferred activities and dining options. For example, it references a list of restaurants the user has visited in the past and makes new recommendations from there.

[1640] Step 8:

[1641] Submit your proposal

[1642] Server: Sends customized suggestions to the user's smartphone app.

[1643] Specific action: Using the push notification system, the suggested action is sent to the user's device in real time. For example, a notification message saying, "You should go see a movie at this theater right now" is sent.

[1644] Step 9:

[1645] View Suggestions

[1646] On the device: Display received action suggestions visually to the user, for example in the notification area or on an in-app dashboard.

[1647] What it does: Receive a push notification and display the suggestion in the app's UI, such as "Enjoy dinner at a popular restaurant nearby," in an easily accessible format for the user.

[1648] Step 10:

[1649] Presenting options

[1650] Device: Presents specific options for suggested actions (e.g., photos of dishes or a list of activities) and utilizes a camera or touch-enabled interface.

[1651] Specific operation: The camera function is used to scan the surrounding environment, and the presented options are displayed as images. The user taps to confirm the option.

[1652] Step 11:

[1653] Reflecting your selection

[1654] Device: The user's selection is sent to the server and reflected in future behavior predictions.

[1655] Specific actions: The user's selected actions are recorded on the device and sent to the server for the next learning cycle. The server then incorporates the received selections into the training data of the generative AI to improve the model's performance.

[1656] Example 1

[1657] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1658] Many modern users feel fatigued by the numerous choices and decisions they have to make in their daily lives, and are facing what is known as "decision fatigue." This "decision fatigue" reduces the quality of their daily lives and increases stress and dissatisfaction. This has led to a demand for systems that can provide personalized action suggestions and decision support for individual users. However, existing systems are unable to fully learn users' behavioral patterns, making it difficult to provide highly accurate suggestions.

[1659] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1660] In this invention, the server includes means for collecting activity data such as user location information, application usage history, calendar events, and sleep data, means for encrypting the activity data and transmitting it to the server via a secure communication protocol, means for receiving the activity data, storing it in a database for each user, and cleaning the data, means for inputting the activity data into a generative AI model and learning the user's behavioral patterns using a deep learning model, and means for suggesting the next action to be taken based on the learned behavioral patterns, thereby enabling highly accurate action suggestions based on the user's behavioral history.

[1661] "Location information" refers to data that indicates the user's current location and the route they are traveling.

[1662] "Application usage history" refers to data that records usage information of applications used by users on smartphones and other devices.

[1663] "Calendar events" refers to data that records a user's schedules and events on a calendar.

[1664] "Sleep data" refers to data that records the user's sleep time and sleep quality.

[1665] "Activity data" is a general term for data related to a user's daily activities, such as location information, application usage history, calendar events, and sleep data.

[1666] "Encryption" is a technology that converts data to transmit information securely and prevents unauthorized access and data leaks.

[1667] A "secure communication protocol" is a communication protocol for securely sending and receiving data, and a specific example is HTTPS.

[1668] "Data cleaning" is a process of improving data quality by filling in missing values ​​and correcting outliers.

[1669] A "generative AI model" is an artificial intelligence model that generates new information and suggestions based on data.

[1670] A "deep learning model" is a machine learning model that uses a multi-layer neural network to perform advanced learning and prediction.

[1671] A "behavioral pattern" is a series of data that indicates a user's daily behavior and tendencies.

[1672] "Action suggestions" are information or ideas that recommend the next action the user should take.

[1673] "Terminal" refers to a device that is directly operated by a user, such as a smartphone or tablet.

[1674] "Choices" are any list or item that allows a user to choose from multiple suggestions or options.

[1675] Overall system configuration

[1676] The present invention provides an action suggestion system for reducing "decision fatigue" in a user's daily life. This system is implemented by the following components and means.

[1677] Data Collection Module

[1678] Device: Your smartphone collects location information, application usage history, calendar events, sleep data, and other activity data. This is done using the smartphone's GPS, various sensors, and application APIs.

[1679] Example: When a user commutes to work, the GPS function on their smartphone is turned on, and location information is collected every minute. In addition, the launch time and usage time of each application being used are collected through the application API, as are events registered in the calendar.

[1680] Data Transmission Module

[1681] On your device: Collected activity data is encrypted and sent to a server via a secure communication protocol (e.g., HTTPS).

[1682] Example: The collected data is encrypted using the AES encryption algorithm and then sent to the server via HTTPS. The data is uploaded to the server in batches, for example, once an hour.

[1683] Data reception and storage module

[1684] Server: Receives data sent from the device and stores it in a database for each user. It also cleans the data as needed (filling in missing values, correcting outliers, etc.).

[1685] Example: Data received on the server side is organized and saved in a dedicated folder for each user. Before saving to the database, outliers are detected and corrected or deleted, and the data is saved as clean data. For example, the location information data of user A is stored in a dedicated table in the database.

[1686] Data Learning Module

[1687] Server: Inputs the received data into a generative AI model to learn user behavior patterns. This generative AI model is trained using a deep learning model.

[1688] Example: Location data and app usage history from the past month are input into a deep learning model as training data to learn the user's movement patterns and app usage trends. Specifically, CNN (convolutional neural network) and LSTM (long short-term memory) models are used.

[1689] Action proposal generation module

[1690] Server: Based on the learned data, the server proposes the next action to be taken. The proposed actions are customized based on the user's preferences and past behavioral history.

[1691] Example: For example, the server suggests "have dinner at a popular restaurant after work" on Friday evening. This suggestion is generated based on the data of restaurants the user has frequently visited in the past. Also, the server selects vegetarian restaurants as the user's preference.

[1692] Action Suggestion Notification Module

[1693] Server: Sends customized suggestions to the user's smartphone app.

[1694] On the device: The smartphone app opens and notifies the user of the suggested action.

[1695] Example: A smartphone app that receives a suggestion message from a server uses the push notification function to display a message to the user saying, "You can refresh yourself by jogging in a nearby park this Saturday."

[1696] Choice Feedback Module

[1697] On the device: Presents specific options for suggested actions (e.g., photos of recipes or a list of activities) using a camera or touch interface.

[1698] User: Selects from the presented options by tapping, etc. The selected content is sent to the server and reflected in the learning of the generation AI.

[1699] Example: A user selects a suggested action, such as "Go to a popular pasta restaurant," on the camera screen, and this information is sent to the server in real time, allowing the generative AI model to learn new data and provide more personalized recommendations the next time.

[1700] Examples of prompts for this system include "Predict the user's next move based on location data from the past month" and "Suggest activities for this weekend, taking into account the user's preferences."

[1701] Through these modules and processes, the present invention can significantly reduce the burden of decision-making in users' daily lives and effectively prevent "decision fatigue." By providing personalized recommendations, it increases satisfaction and achieves higher accuracy than conventional general recommendation systems.

[1702] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1703] Step 1:

[1704] Data collection

[1705] Device: Collects activity data such as user location, application usage history, calendar events, and sleep data. This data collection is done using the smartphone's GPS function, various sensors, and application APIs.

[1706] Input: Raw data collected from sensors on the user's smartphone and application APIs (location, application usage history, calendar events, sleep data).

[1707] Processing: Collecting data from sensors and APIs and organizing it into a specific format as activity data.

[1708] Output: Organized activity data.

[1709] What it does: When a user commutes to work, the GPS function on their smartphone is turned on, and location information is collected every minute, along with application launch times, usage time, and calendar events.

[1710] Step 2:

[1711] Data transmission

[1712] On your device: Collected activity data is encrypted and sent to a server via a secure communication protocol (e.g., HTTPS).

[1713] Input: Organized activity data.

[1714] Processing: The organized activity data is encrypted using the AES encryption algorithm and sent to the server via HTTPS.

[1715] Output: The encrypted data is sent to the server.

[1716] Specific operation: The collected data is periodically (for example, once an hour) encrypted in batch format and uploaded to the server via HTTPS communication.

[1717] Step 3:

[1718] Data reception and storage

[1719] Server: Receives data sent from the device and stores it in a database for each user. It also cleans the data (fills in missing values, corrects outliers, etc.).

[1720] Input: Encrypted activity data.

[1721] Processing: The encrypted data is decrypted, organized by user, and stored in a database. Data cleaning is performed.

[1722] Output: Cleaned per-user activity data.

[1723] Specific operations: Decrypts encrypted data on the server side and stores it in a folder for each user in the database. Detects and imputes outliers, and fills in missing values ​​to improve data quality.

[1724] Step 4:

[1725] Data Learning

[1726] Server: Inputs the received data into a generative AI model to learn user behavior patterns. This generative AI model is trained using a deep learning model.

[1727] Input: Cleaned per-user activity data.

[1728] Processing: The cleaned data is fed into a deep learning model to learn user behavior patterns.

[1729] Output: Learned behavioral pattern model.

[1730] How it works: The system inputs location data and application usage history from the past month into a deep learning model to learn the user's movement patterns and app usage trends, using deep learning techniques such as CNN and LSTM.

[1731] Step 5:

[1732] Action suggestion generation

[1733] Server: Based on the learned data, the server proposes the next action to be taken. The proposed actions are customized based on the user's preferences and past behavioral history.

[1734] Input: Learned behavioral pattern model.

[1735] Processing: Based on the learned model, next action suggestions are generated, which are customized based on the user's preferences and past behavioral history.

[1736] Output: Customized action suggestions.

[1737] Specific behavior: For example, the server may suggest "have dinner at a popular restaurant after work" on a Friday evening. This suggestion is generated based on data on restaurants the user has frequently visited in the past.

[1738] Step 6:

[1739] Action suggestion notification

[1740] Server: Sends customized suggestions to the user's smartphone app.

[1741] On the device: The smartphone app notifies the user of the suggested action.

[1742] Input: customized action suggestions.

[1743] Processing: Sends the proposed content to the smartphone app and notifies it.

[1744] Output: The user is notified of the suggestion.

[1745] Specific operation: The smartphone app receives a proposal message from the server and uses the push notification function to notify the user of the proposal.

[1746] Step 7:

[1747] Selection Feedback

[1748] On the device: Present specific options for the suggested action (e.g., photos of recipes or a list of activities) using a camera or touch interface.

[1749] User: Selects from the presented options by tapping, etc. The selected content is sent to the server and reflected in the learning of the generation AI.

[1750] Input: The user's selection.

[1751] Processing: The user's selection is sent to the server and reflected in the generative AI model.

[1752] Output: The updated training model.

[1753] What happens: The user taps to select the suggested action "Go to a popular pasta restaurant," which is then sent to the server, allowing the generative AI model to learn new data and provide more personalized recommendations the next time.

[1754] (Application example 1)

[1755] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1756] While modern users have access to a wide range of entertainment content, they often experience "decision fatigue" when trying to choose the best content from the diverse options. Under these circumstances, there is a need for systems that can appropriately suggest the next content to watch based on the user's preferences and behavioral history. Current systems are unable to effectively utilize the content and activity data of users' past viewings, making it difficult to make optimal suggestions for each user.

[1757] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1758] In this invention, the server includes means for collecting activity data such as a user's location information, app usage history, calendar events, and sleep data, means for transmitting the activity data to the server via a secure communication protocol, means for a generation AI to learn the user's behavioral patterns using a deep learning model based on the activity data, means for suggesting entertainment content to be viewed next based on the behavioral patterns, means for notifying the user's smartphone of the suggested content, and means for reflecting the user's selection in the learning, thereby enabling users to easily select entertainment content appropriately customized based on their preferences and behavioral history.

[1759] "Location information" is data that indicates a user's current location and is obtained using GPS or other location information technology.

[1760] "App usage history" is a record of applications used by a user on a smartphone or other device, and includes data such as the time the app was launched and the amount of time it was used.

[1761] "Calendar events" are information about events and schedules that a user registers in a calendar to manage their plans and appointments.

[1762] "Sleep data" refers to data that indicates the user's sleep time and quality, and is collected via smartphones or wearable devices.

[1763] "Activity data" is data that records various actions and conditions of a user's daily life, and includes location information, app usage history, calendar events, sleep data, and more.

[1764] A "secure communication protocol" is a safe means of communication that prevents eavesdropping or tampering by third parties when sending and receiving data, and examples include protocols such as HTTPS.

[1765] "Generative AI" is artificial intelligence that uses machine learning techniques such as deep learning to train user data to perform specific tasks.

[1766] A "deep learning model" is a type of artificial intelligence that uses multi-layered neural networks to learn complex patterns from data.

[1767] A "behavioral pattern" indicates the tendencies and habits of the user's daily behavior, and is determined based on past data.

[1768] "Entertainment content" refers to media such as movies, dramas, music, podcasts, etc. that users consume for entertainment purposes.

[1769] A "smartphone" is a multi-function mobile phone that is a device that can not only make calls but also access the Internet and use applications.

[1770] "Push notification" is a function that notifies information to smartphones and other devices in real time.

[1771] A "selection" refers to the particular item a user chooses from among the options offered.

[1772] "Means for reflecting in learning" refers to a method for feeding back user choices to the generative AI model to improve the accuracy of subsequent suggestions.

[1773] The present invention is an action suggestion system for reducing "decision fatigue" in a user's daily life, and specific implementation means are described below.

[1774] Data Collection Module

[1775] The device used is the user's smartphone, which collects location information, app usage history, calendar events, sleep data, and other activity data. This data is obtained through the smartphone's built-in sensors and application APIs.

[1776] Data Transmission Module

[1777] Activity data collected on the device is sent to a server using a secure communication protocol (e.g., HTTPS). The data is encrypted and uploaded in batches at regular intervals (e.g., once an hour).

[1778] Data reception and storage module

[1779] The server side receives the transmitted data and stores it in a database for each user. Data cleansing (filling in missing values, correcting outliers, etc.) is also performed. For example, SQLite is used as the database.

[1780] Data Learning Module

[1781] Based on the data received on the server, the generated AI uses a deep learning model to learn user behavior patterns. To do this, it uses deep learning libraries such as TensorFlow. It uses past location data and app usage history as training data to learn user patterns and tendencies.

[1782] Action proposal generation module

[1783] The server uses the learned data to predict the next entertainment content to watch and generates action suggestions, which are customized based on the user's preferences and past behavioral history.

[1784] Action Suggestion Notification Module

[1785] The generated action suggestions are sent to the user's smartphone using the push notification function, and the notification content includes specific entertainment content such as "the next movie to watch."

[1786] Choice Feedback Module

[1787] The user taps to select an option from the suggested content. This selection is sent to the server and reflected in the learning of the generative AI. The suggested options can also be visually displayed using the smartphone's camera function.

[1788] Specific examples

[1789] For example, at 8 p.m. on a weekday while a user is relaxing at home, a push notification on their smartphone suggests a movie to watch next based on their past viewing history and activity data. This suggestion specifically suggests movies in a genre that the user previously enjoyed.

[1790] Example prompt sentence:

[1791] "Below is your past viewing history and recent activity data. Based on this data, we'd like to suggest what you should watch next."

[1792] Viewing history: [Movie A, Drama B, Music C...]

[1793] Latest activity data: {Location: Home, Time: 20:00, Date: 2023-10-02}

[1794] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1795] Step 1:

[1796] The device (user's smartphone) collects activity data such as location information, app usage history, calendar events, and sleep data.

[1797] Input: Data obtained through smartphone sensors and APIs.

[1798] Specific operation: Using the smartphone's GPS, it collects location information every minute and also periodically records app usage history and calendar events.

[1799] Output: Collected activity data (location, app usage history, etc.).

[1800] Step 2:

[1801] The device transmits the collected activity data to the server via a secure communication protocol (e.g., HTTPS).

[1802] Input: Collected activity data.

[1803] What it does: It encrypts your activity data and uploads it to a server in batches using the HTTPS protocol, for example, every hour.

[1804] Output: Activity data sent to the server.

[1805] Step 3:

[1806] The server receives the data and stores it in a database. It also performs data cleansing, such as filling in missing values ​​and correcting outliers.

[1807] Input: Submitted activity data.

[1808] Specific operation: The received data is organized and stored in a database for each user, outliers are detected and corrected, and missing values ​​are complemented.

[1809] Output: Saved and cleansed activity data.

[1810] Step 4:

[1811] The server uses generative AI (deep learning model) to learn the user's behavioral patterns based on the stored activity data.

[1812] Input: Saved activity data.

[1813] Specific operation: Using a deep learning library (such as TensorFlow), the model is trained using past location information and app usage history as training data. The following is an example of a prompt sentence to input.

[1814] Example prompt: "Below is the user's past viewing history and latest activity data. Based on this data, please suggest the next content they should watch." Viewing history: [Movie A, Drama B, Music C...] Latest activity data: {Location: Home, Time: 20:00, Date: 2023-10-02}

[1815] Output: An AI model that has learned the user's behavioral patterns.

[1816] Step 5:

[1817] The server uses the learned AI model to predict the next entertainment content to watch and generate action suggestions.

[1818] Input: An AI model that has learned your behavioral patterns, and your latest activity data.

[1819] What it does: It uses AI models to predict the best entertainment content for users based on their current situation and past behavioral patterns.

[1820] Output: Suggestions about what entertainment content to watch next.

[1821] Step 6:

[1822] The server sends the generated action suggestions to the smartphone via push notification.

[1823] Input: Suggestions for entertainment content to watch next.

[1824] Specific operation: Using a push notification library (e.g., Firebase Cloud Messaging), the suggestion content is sent to the user's smartphone.

[1825] Output: Action suggestions displayed on the user's smartphone.

[1826] Step 7:

[1827] Users tap to select from the suggested entertainment content, and the selected content is sent to the server and reflected in the learning of the generative AI.

[1828] Input: User-selected entertainment content.

[1829] Specific operation: Using the smartphone's camera function and touch operation interface, the proposed options are visually displayed and the user's tap selection information is collected. The selection information is then sent back to the server and used to update the generative AI model.

[1830] Output: The user's choices fed back to the generative AI model.

[1831] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1832] The present invention is an action suggestion system for reducing "decision fatigue" in a user's daily life, and further improves the accuracy and satisfaction of the suggestions by incorporating an emotion engine that recognizes and responds to the user's emotional state. The present invention is implemented by the following means.

[1833] Overall system configuration

[1834] 1. Data Collection Module

[1835] Device: Your smartphone collects activity data, such as location, app usage history, calendar events, and sleep data. This data is periodically retrieved from sensors and application APIs.

[1836] Example: A user turns on the GPS on their smartphone during their commute to collect location information, record app usage history, calendar events, and data from a sleep tracker.

[1837] 2. Data transmission module

[1838] On your device: Collected activity data is encrypted and sent to a server via a secure communication protocol.

[1839] Example: The acquired data is uploaded to the server in batches every hour.

[1840] 3. Data receiving and storage module

[1841] Server: Stores the received data in a database for each user, and cleans the data if necessary.

[1842] Example: Organize data by user ID on the server side, process outliers and missing values, and then save it to the database.

[1843] 4. Data Preprocessing Module

[1844] Server: Cleans the received data and converts it into a format suitable for the generative AI.

[1845] Examples: data normalization, timestamp alignment, feature engineering.

[1846] 5. Generative AI Learning Module

[1847] Server: Trains the generative AI using preprocessed data to learn user behavior patterns.

[1848] Example: Using past location information and app usage history as training data to recognize users' travel patterns and app usage tendencies.

[1849] 6. Action Proposal Generation Module

[1850] Server: Uses the trained model to predict the user's next action in real time and generate action suggestions.

[1851] Example: Feed new data points into the model to generate specific suggestions, such as "have a coffee at a nearby cafe."

[1852] 7. Proposal customization module

[1853] Server: Customizes suggested actions based on the user's preferences and past behavior.

[1854] Example: Referencing data on restaurants the user has visited in the past and suggesting "dinner at a popular restaurant after work."

[1855] 8. Action Suggestion Notification Module

[1856] Server: Sends customized suggestions to the user's smartphone app.

[1857] Terminal: Visually notify the user of the proposed action.

[1858] Example: Use a push notification to say, "You can refresh yourself by jogging in a nearby park this Saturday."

[1859] 9. Choice Feedback Module

[1860] Device: Presents specific options for suggested actions using a camera or touch interface.

[1861] User: Tap to select a presented option. The selection is sent to the server and reflected in the learning of the generation AI.

[1862] Example: A user selects the option "Go to a popular pasta restaurant" from a camera screen, and the selection is sent to a server.

[1863] 10. Emotion Engine Module

[1864] Device: Recognizes the user's emotional state by analyzing facial expressions, voice, and text messages.

[1865] Example: Using a smartphone's camera and microphone, it analyzes emotions from the user's facial expressions and tone of voice, as well as the content of text messages.

[1866] 11. Emotion-based behavior suggestion adjustment module

[1867] Server: Adjusts the recommendations based on the results from the emotion engine.

[1868] Example: If the user is feeling stressed, suggest a relaxing activity (e.g., yoga or meditation).

[1869] Through the above-described embodiments, the present invention reduces the burden of decision-making in the user's daily life, and furthermore, by using an emotion engine, it is possible to realize optimal action suggestions that take into account the user's mental state, thereby providing higher accuracy and satisfaction than conventional systems.

[1870] The processing flow will be explained below.

[1871] Step 1:

[1872] Activity Data Collection

[1873] Device: Your smartphone collects activity data, such as location, app usage history, calendar events, and sleep data. This data is periodically retrieved from sensors and application APIs.

[1874] Specific operations: Turn on the GPS function to obtain location information, record the launch time and usage time of each application in use, extract schedule and event information from the calendar API, and obtain sleep data from a sleep tracker app.

[1875] Step 2:

[1876] Sending data to the server

[1877] On your device: We encrypt the activity data we collect and send it to our servers using a secure communication protocol (e.g., HTTPS).

[1878] Specific behavior: Periodically checks the activity data stored in the local database, uploads any unsent data to the server in batches, encrypts the data using an encryption library, and sends an HTTP request.

[1879] Step 3:

[1880] Receiving and storing data

[1881] Server: Receives data sent from the device and stores it in a database for each user.

[1882] Specific operations: Receive data via API endpoint, organize the data by user ID, and save it in a database. Detect outliers and handle missing values ​​to check data integrity.

[1883] Step 4:

[1884] Data Preprocessing

[1885] Server: Cleans the received data (fills in missing values, handles outliers, etc.) and converts it into a format suitable for generative AI.

[1886] What it does: Normalizes the data, aligns it on the time axis, and performs feature engineering as needed, e.g., converting location information into geographic features and adding categorical features based on time of day.

[1887] Step 5:

[1888] Generative AI training

[1889] Server: Trains the generative AI using preprocessed data to learn user behavior patterns.

[1890] Specific operation: Preprocessed data is input into the deep learning model, and it learns past behavioral patterns. During the training process, an appropriate learning rate and number of epochs are set, and adjustments are made while evaluating the model's performance.

[1891] Step 6:

[1892] Generate action suggestions

[1893] Server: Using the trained model, predict the user's next action in real time and generate action suggestions.

[1894] Specific Actions: New data points (e.g., current location or recent app usage history) are fed into the model to predict the next action, generating specific suggestions such as "get a coffee at a nearby cafe" or "relax at home."

[1895] Step 7:

[1896] Customize suggestions

[1897] Server: Customizes suggested actions based on the user's preferences and past behavior.

[1898] What it does: It applies personalization algorithms to generate recommendations that take into account the user's preferred activities and dining options. For example, it references a list of restaurants the user has visited in the past and makes new recommendations from there.

[1899] Step 8:

[1900] Submit your proposal

[1901] Server: Sends customized suggestions to the user's smartphone app.

[1902] Specific action: Using the push notification system, the suggested action is sent to the user's device in real time. For example, a notification message saying, "You should go see a movie at this theater right now" is sent.

[1903] Step 9:

[1904] View Suggestions

[1905] On the device: Display received action suggestions visually to the user, for example in the notification area or on an in-app dashboard.

[1906] What it does: Receive a push notification and display the suggestion in the app's UI, for example, "Enjoy dinner at a popular restaurant nearby," in a format that's easily accessible to the user.

[1907] Step 10:

[1908] Presenting options

[1909] Device: Presents specific options for suggested actions (e.g., photos of dishes or a list of activities) and utilizes a camera or touch-enabled interface.

[1910] Specific operation: The camera function is used to scan the surrounding environment, and the presented options are displayed as images. The user taps to confirm the option.

[1911] Step 11:

[1912] Reflecting your selection

[1913] Device: The user's selection is sent to the server and reflected in future behavior predictions.

[1914] Specific actions: The user's selected actions are recorded on the device and sent to the server for the next learning cycle. The server then incorporates the received selections into the training data of the generative AI to improve the model's performance.

[1915] Step 12:

[1916] Collecting Emotional Data

[1917] Device: Recognizes the user's emotional state by analyzing facial expressions, voice, and text messages.

[1918] Specific operation: Using the smartphone's camera and microphone, the system analyzes the user's facial expressions and tone of voice, and uses natural language processing to analyze the content of text messages to determine their emotional state.

[1919] Step 13:

[1920] Sending emotional data

[1921] Terminal: Sends analyzed emotion data to the server.

[1922] Specific operation: The emotion data output by the emotion recognition engine is encrypted and sent to the server using a secure communication protocol.

[1923] Step 14:

[1924] Tailoring behavioral suggestions based on emotions

[1925] Server: Adjusts the recommendations based on the results from the emotion engine.

[1926] Specific actions: If the user is feeling stressed, it will suggest relaxing actions (e.g., yoga or meditation). Conversely, if the user is feeling excited, it will suggest active actions (e.g., sports or dancing).

[1927] Through the above-described embodiments, the present invention reduces the burden of decision-making in the user's daily life, and furthermore, by using an emotion engine, it is possible to realize optimal action suggestions that take into account the user's mental state, thereby providing higher accuracy and satisfaction than conventional systems.

[1928] Example 2

[1929] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1930] While current technology exists to reduce the burden of decision-making in users' daily lives, these systems are unable to take into account their emotional state when making suggestions, which can result in low satisfaction. To solve this problem, it is necessary to provide a system that recognizes the user's emotional state and suggests optimal actions.

[1931] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting activity data such as the user's location information, app usage history, calendar events, and sleep data, means for encrypting the activity data and transmitting it to the server, means for a generation AI to learn the user's behavioral patterns based on the activity data, means for suggesting a next action to be taken based on the behavioral patterns, means for recognizing the user's emotional state using an emotion engine and adjusting the suggestion content, means for notifying the user's terminal of the suggested action, and means for reflecting the user's selection in the learning. This reduces the burden of decision-making in the user's daily life and enables highly accurate and satisfying action suggestions that take the user's emotional state into consideration.

[1932] "User location information" is geographical data that indicates where the user is currently located.

[1933] "Application usage history" is data that records the names of applications used by a user, the duration of use, frequency of use, and so on.

[1934] A "calendar event" is event information entered into a calendar application that registers a user's plans and schedules.

[1935] "Sleep data" refers to data that records the user's sleep patterns, sleep duration, quality, etc.

[1936] "Activity data" refers to all data related to daily life activities, such as a user's location information, app usage history, calendar events, and sleep data.

[1937] "Encryption" means converting data into a form that is unintelligible to others using a specific algorithm.

[1938] "Generative AI" is an artificial intelligence model that can learn a user's behavioral patterns and suggest the next action to take.

[1939] A "behavioral pattern" refers to a series of actions and habits in a user's daily life.

[1940] The "emotion engine" is an engine that analyzes and identifies the user's emotional state from their facial expressions, voice, and text.

[1941] "Suggestion content" refers to a suggested action that the generation AI generates by taking into account the user's behavioral patterns and emotional state.

[1942] "Notification" refers to displaying the generated action suggestion on the user's terminal.

[1943] "Selection" refers to the item that the user selects from the suggested actions.

[1944] "Reflecting in learning" means incorporating user selections into the generative AI model to improve the accuracy of future action suggestions.

[1945] This invention is a system for reducing the burden of decision-making in a user's daily life and for suggesting optimal actions that take into account the user's emotional state.

[1946] System configuration

[1947] Data Collection Module

[1948] The user's device, specifically their smartphone, is used to collect activity data such as location information, app usage history, calendar events, and sleep data, which are periodically retrieved through the device's sensors and application APIs.

[1949] Example: A user turns on the GPS on their smartphone during their commute to collect location information, record app usage history, calendar events, and data from a sleep tracker.

[1950] Data Transmission Module

[1951] Activity data collected on the device is encrypted and sent to a server via a secure communication protocol (e.g., HTTPS).

[1952] Example: The collected data is AES encrypted in batches every hour and uploaded to the server via HTTPS protocol.

[1953] Data reception and storage module

[1954] The server stores the received data in a database for each user. Before storing, it cleans the data (dealing with outliers and imputing missing values) as necessary.

[1955] Example: The data received by the server is organized into a database by user ID, and outliers and missing values ​​are automatically processed.

[1956] Data Preprocessing Module

[1957] The server cleans the received data and converts it into a format suitable for generative AI, including normalizing the data, aligning timestamps, and extracting features.

[1958] Example: Prepare a training dataset by properly aligning timestamps, normalizing numeric data, and removing unnecessary fields.

[1959] Generative AI learning module

[1960] The server uses the preprocessed data to train a generative AI to learn the user's behavioral patterns. This generative AI is trained using a deep learning framework (e.g., TensorFlow, PyTorch).

[1961] Example: Using past location information and app usage history to learn about users' travel patterns and app usage habits.

[1962] Action proposal generation module

[1963] The server uses a trained generative AI model to predict the user's next action in real time and generate action suggestions.

[1964] Example: Newly collected data points are fed into a generative AI model to generate specific suggestions, such as "It's almost lunchtime, so suggest a nearby cafe."

[1965] Suggestion customization module

[1966] The server customizes the generated suggestions based on the user's preferences and past behavioral history.

[1967] Example: Referencing data on restaurants frequently visited by the user to provide customized suggestions such as "dinner at a popular restaurant after work."

[1968] Action Suggestion Notification Module

[1969] The server sends the customized action suggestions to the user's smartphone app, which then visually notifies the user of the suggested action.

[1970] Example: Use push notifications to notify users of something like, "You can refresh yourself by jogging in a nearby park this Saturday."

[1971] Choice Feedback Module

[1972] The device uses a camera or touch interface to present specific options for the proposed action. The user taps to select the option. The selection is sent to the server and reflected in the learning of the generative AI.

[1973] Example: A user selects the option "Go to a popular pasta restaurant" from a camera screen, and the selection is sent to a server.

[1974] Emotion Engine Module

[1975] The device recognizes the user's emotional state by analyzing their facial expressions, voice, and text messages.

[1976] Example: Using the smartphone camera and microphone, the system analyzes emotions from the user's facial expressions and tone of voice, including text messages.

[1977] Emotion-based behavior suggestion adjustment module

[1978] The server adjusts the suggestions based on the emotional state obtained from the emotion engine.

[1979] Example: If the user is feeling stressed, suggest a relaxing activity (e.g., meditation, yoga).

[1980] Examples of prompts for generative AI models

[1981] "Generate optimal action suggestions based on the user's current location, app usage history, calendar events, and sleep data."

[1982] This invention can reduce the burden of decision-making in the user's daily life and realize highly accurate and satisfying action suggestions that also take into account the user's emotional state.

[1983] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1984] Step 1:

[1985] Data collection

[1986] Input: Activity data about the user's daily life (location, app usage history, calendar events, sleep data).

[1987] Specific operation: The device (smartphone) acquires location information using GPS, records application usage history in the background, obtains the user's schedule from a calendar app, and collects sleep data from a sleep tracker app.

[1988] Output: Collected activity data.

[1989] Step 2:

[1990] Data transmission

[1991] Input: Collected activity data.

[1992] Specific operation: The device encrypts the collected data using AES and sends it to the server using a secure communication protocol (HTTPS).

[1993] Output: The encrypted data is sent to the server.

[1994] Step 3:

[1995] Data reception and storage

[1996] Input: Encrypted data.

[1997] Specific operation: The server receives the data, unlocks it, and then stores it in a database for each user ID. At that time, it cleans the data by removing outliers and filling in missing values.

[1998] Output: An organized database.

[1999] Step 4:

[2000] Data Preprocessing

[2001] Input: A curated database.

[2002] Specific operations: The server normalizes the data, aligns timestamps, removes unnecessary fields, extracts features, and converts it into a format suitable for generative AI.

[2003] Output: The preprocessed dataset.

[2004] Step 5:

[2005] Generative AI training

[2006] Input: The preprocessed dataset.

[2007] How it works: The server trains the generative AI using a deep learning framework (e.g., TensorFlow, PyTorch), learning the dataset over multiple epochs and incorporating user behavior patterns into the model.

[2008] Output: The trained model.

[2009] Step 6:

[2010] Action suggestion generation

[2011] Input: Newly collected data points, trained model.

[2012] What it does: The server uses the trained model to analyze new data points collected and predict the next action to take. It generates suggestions based on prompts (e.g., "Generate optimal action suggestions based on the current user's location, app usage history, calendar events, and sleep data.").

[2013] Output: Proposal.

[2014] Step 7:

[2015] Customize suggestions

[2016] Input: Proposal content, past behavior history.

[2017] Specific operation: The server customizes the generated suggestions based on the user's preferences and past behavior history, taking into account the places visited in the past and preferred activities to provide optimal suggestions.

[2018] Output: A customized proposal.

[2019] Step 8:

[2020] Action suggestion notification

[2021] Input: Your customized suggestion.

[2022] Specific operation: The server uses the push notification API to send customized suggestions to the device, which then visually notifies the user of the suggested action.

[2023] Output: The suggestion that was communicated to the user.

[2024] Step 9:

[2025] Selection Feedback

[2026] Input: User choices, camera and touch interfaces.

[2027] Specific actions: The device visually displays specific options for the proposed action, and the user taps to select it. The user's selection is sent to the server and reflected in the learning of the generative AI.

[2028] Output: The selections sent to the server.

[2029] Step 10:

[2030] Emotion Engine

[2031] Input: User facial expressions, voice, and text messages.

[2032] How it works: The device uses the smartphone's camera and microphone to collect the user's facial expressions and voice, and analyzes emotions from text messages.

[2033] Output: Emotional state data.

[2034] Step 11:

[2035] Emotion-based behavioral adjustment

[2036] Input: Emotional state data, customized suggestions.

[2037] Specific operation: The server further adjusts the suggested actions based on the emotional state obtained from the emotion engine. If the user is feeling stressed, the server changes the suggested actions to suggest relaxing activities.

[2038] Output: Final recommendations based on the user's emotional state.

[2039] (Application example 2)

[2040] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2041] In recent years, the proliferation of options in consumers' daily lives has led to the problem of "decision fatigue." Particularly in brick-and-mortar stores, consumers are forced to make optimal choices from a vast array of products and services, placing a psychological burden on them. Furthermore, because optimal recommendations vary depending on a consumer's emotional state, there is a need for a system that can analyze each individual's emotional state and adjust the recommendations accordingly.

[2042] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting behavioral data such as user location information, app usage history, time information, and sleep data; means for transmitting the behavioral data to the server; means for a generation AI to learn the user's behavioral patterns based on the behavioral data; means for suggesting the next action to be taken based on the behavioral patterns; means for notifying the user's device of the suggested action and making a purchase suggestion; means for reflecting the user's selection in the learning; and means including an emotion engine for collecting the user's emotional state and adjusting the suggestion content. This reduces the decision-making burden in consumers' daily lives and makes it possible to provide personalized suggestions based on each individual's emotional state in real time.

[2043] "User location information" is data that identifies the user's current geographic location.

[2044] "Application usage history" is data on applications that a user has used in the past.

[2045] "Time information" is data relating to the time and date when the user performed an action.

[2046] "Sleep data" is information related to the user's sleep, including the amount of sleep time and the quality of sleep.

[2047] "Behavioral data" is a general term for data including location information, app usage history, time information, sleep data, and the like in a user's daily life.

[2048] "Generative AI" is an artificial intelligence that learns user behavior patterns based on collected data and generates new suggestions.

[2049] "Behavioral patterns" refer to tendencies and habits extracted from a user's past behavior.

[2050] The "means for suggesting an action" is a method for indicating the next action to be taken based on the user's behavioral patterns.

[2051] A "purchase suggestion" is a suggestion that recommends specific products or services related to consumption behavior to a user.

[2052] The "emotion engine" is a function that analyzes the user's emotional state and adjusts the content of suggestions based on that state.

[2053] "Personalized offers" are individual offers tailored to a user's individual interests, preferences, and emotional state.

[2054] "Server" means a computer system that receives, stores, analyzes, and generates recommendations from user-submitted data.

[2055] This invention is an action suggestion system designed to reduce "decision fatigue" in users' daily lives. It also incorporates an emotion engine that recognizes and responds to the user's emotional state, improving the accuracy of the suggestions and satisfaction. This system collects behavioral data, such as the user's location, app usage history, time information, and sleep data, and uses this data to generate AI that learns the user's behavioral patterns. Based on the behavioral patterns, the system then suggests the next action to be taken and notifies the user's device of the suggested action.

[2056] First, user behavior data is collected from devices such as smartphones. Specifically, this includes location information using GPS, application usage history, calendar events, and data from sleep trackers. This data is collected in real time or periodically and sent to a server. The data is encrypted and transmitted via a secure communication protocol.

[2057] The server stores the received behavioral data in a database, cleans the data, and then performs preprocessing to train the generative AI model. Specifically, it normalizes the data, aligns timestamps, and performs feature engineering. The preprocessed data is then used to train the generative AI, which learns the user's behavioral patterns. This AI model uses deep learning technology.

[2058] The action suggestion generation module uses a trained generative AI model to predict the next action a user should take in real time based on their behavioral patterns, generating action suggestions. For example, if the user is in a physical store, suggestions such as "take a break at a coffee shop" or "check for new product sales" will be made. These suggestions are further refined by an emotion engine that analyzes the user's emotional state. Specifically, the system analyzes the user's facial expressions and voice using the smartphone's camera and microphone, as well as the content of text messages.

[2059] The proposed actions, including purchasing suggestions, are sent to the user's device. The user selects from the suggested options using touch operations on their smartphone, and the selection is then sent back to the server. This allows the generative AI model to constantly retrain based on the latest data, improving the accuracy of the suggestions.

[2060] For example, if the system detects that the user is feeling stressed, it will suggest "take a break in the relaxation area." If the user is feeling normal, it will suggest "try your favorite pasta restaurant." In this way, personalized suggestions based on the user's individual emotional state can be made in real time.

[2061] Example prompt sentence:

[2062] The user opens their smartphone and types, "Please suggest what I should do next." This prompt allows the server to analyze their latest behavioral patterns and emotional state and notify the user of the optimal suggested action.

[2063] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2064] Step 1:

[2065] The user's device collects behavioral data such as location information, app usage history, time information, and sleep data. This data is obtained from sensors and devices such as the smartphone's GPS, application APIs, calendar functions, and sleep trackers. Input data includes location coordinates, a list of used apps, calendar events, and sleep duration, and this is output as the initial behavioral data.

[2066] Step 2:

[2067] The behavioral data collected by the device is encrypted and sent to the server via a secure communication protocol. The input data is the behavioral data collected in step 1, and is output as transmission data after going through the encryption process. Specifically, the data is uploaded in batches every hour.

[2068] Step 3:

[2069] The server stores the received data in a database for each user. This is where data cleaning is also performed. The input data is the data sent in step 2, and outliers and missing values ​​are processed before being stored in the database. The cleaned data is output.

[2070] Step 4:

[2071] The server cleans the received data and converts it into a format suitable for generative AI. Specifically, it normalizes the data, aligns timestamps, and performs feature engineering. The input data is data stored in a database, and the output is preprocessed training data.

[2072] Step 5:

[2073] The server uses the preprocessed data to train the generative AI to learn the user's behavioral patterns. A deep learning model is used here. The input data is the training data output in step 4, and the output is a trained model.

[2074] Step 6:

[2075] The action suggestion generation module uses the trained model to predict the next action to be taken in real time based on the user's behavioral patterns and generates action suggestions. New data points (e.g., current location or app usage status) are used as input, and action suggestions are generated as output. For example, a specific suggestion such as "Take a break at a coffee shop" is generated.

[2076] Step 7:

[2077] The server customizes the generated action suggestions and refers to the user's preferences and past behavioral history to provide more personalized suggestions. The input data are the action suggestions generated in step 6 and the user's preference data, and customized suggestions based on them are output.

[2078] Step 8:

[2079] The server notifies the user's device of the customized suggestion. The user's device visually displays the notification and informs the user of the suggested action. The input data is the customized suggestion output in step 7, and the notification data is output. A push notification is used for the specific notification.

[2080] Step 9:

[2081] The user selects a specific option for the proposed action and sends the selection to the server. The input data is the selection from the device, and this is output as the data to be sent to the server. Specifically, the selection is made using touch operations or the camera function.

[2082] Step 10:

[2083] The server uses an emotion engine that collects the user's emotional state and adjusts suggestions based on that state. Input data includes the user's facial expressions, voice, and text messages, and outputs the emotional state and adjustment data based on that state. If the emotional state is "stressed," the system will make suggestions such as "take a break in the relaxation area."

[2084] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2085] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2086] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2087] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2088] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2089] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2090] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2091] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the e...

Claims

1. A means of collecting activity data, such as user location, app usage history, calendar events, and sleep data; means for transmitting the activity data to a server; A means for the generation AI to learn the user's behavioral patterns based on the activity data; means for suggesting a next action to be taken based on the behavior pattern; means for notifying a user terminal of the proposed action; means for reflecting the user's selection in the learning; A system including:

2. The system of claim 1 , wherein the generative AI is trained using a deep learning model.

3. The system according to claim 1, wherein the user's device uses a camera function to visually display the proposed options, allowing the user to select one by tapping.

Images