system

Abstract

We provide the system. [Solution] A means for aggregating and storing individual behavioral history data, A means of analyzing stored behavioral history data to predict specific behaviors of individuals, A means of generating suggestions for the next necessary tasks or purchases based on predicted actions, A means of notifying individuals of the generated proposals through a communication application, A means of collecting individual responses to the proposed content and using it as data to improve prediction accuracy, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] With the aging of the population and the diversification of work styles, there is a demand for preventing mistakes and improving efficiency in daily work. Also, in daily life, forgetting to buy things frequently occurs during busyness, resulting in waste of time and labor. These problems arise because there is no system that can accurately predict individual behavior and make proposals at appropriate timings.

Means for Solving the Problems

[0005] This invention provides a system that aggregates individual behavioral history data and analyzes behavioral patterns using machine learning to predict the next necessary tasks and purchases. This system includes a means of notifying individuals of the generated suggestions via a communication application, allowing users to optimize their actions in a timely manner. Furthermore, by collecting user feedback and updating the prediction model, the system can further improve accuracy. The aim is to improve work efficiency, prevent forgotten purchases, and enhance the quality of life.

[0006] "Behavioral history data" refers to information that shows a record of an individual's past actions, including work performance and purchase history.

[0007] "Analysis" refers to the process of extracting specific regularities or patterns from collected data and generating predictive models.

[0008] "Prediction" refers to the process of estimating and proposing future actions and needs by analyzing past data.

[0009] "Communication applications" are a general term for software and platforms that enable the sending and receiving of information over the internet and facilitate communication between individuals.

[0010] "Feedback" refers to data collected from individuals regarding their responses to suggestions and the results of their implementation, which is then used to improve the system and facilitate learning.

[0011] A "machine learning model" is a computational model created using algorithms that learn patterns based on past data, and it contributes to improving the accuracy of predictions and suggestions. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, a RAM (Random Access Memory) with a reference number is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, a storage with a reference number is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, a communication I / F (Interface) with a reference number is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] The system according to the present invention efficiently manages an individual's daily work and lifestyle activities, providing necessary action suggestions and notifications to prevent forgotten purchases. This system exchanges data between the server, terminal, and user, aiming for operation optimized for the individual.

[0034] The server first collects individual behavioral history data from the terminal. This behavioral history data includes the content of the tasks performed by the individual, the time they were performed, and the history of products purchased in the past. The server analyzes this data to identify each individual's behavioral patterns. In doing so, it uses machine learning algorithms to perform more sophisticated pattern recognition and predict the user's next actions and necessary purchases.

[0035] Based on the predictions obtained through analysis, the server generates personalized suggestions for the user's next actions and purchases. These suggestions include information such as what tasks the user should perform next and when would be the best time to perform them. For purchases, it also suggests the optimal time to buy and includes links to online stores where the items can be purchased.

[0036] The generated proposals are sent to the terminal via a communication application. The terminal then notifies the user of the proposals received from the server in an easy-to-understand format. This notification can be sent via a messaging application such as LINE, allowing the user to immediately review the notification and decide on an action based on the proposals.

[0037] Users check notifications received on their devices and perform suggested tasks or click purchase links to buy products. During this process, the user's actions and selected responses are recorded on the device, and this feedback information is sent to the server. The server uses this feedback to update its machine learning model and improve the accuracy of its suggestions.

[0038] For example, if the server analyzes a user's monthly spending patterns and discovers a tendency to purchase certain consumables at the end of the month, the server will suggest repurchasing those consumables near the end of the month and provide the user with a link to an online store where those consumables can be purchased. In this way, the present invention enables improvements in work efficiency and quality of life by providing personalized suggestions to individual users.

[0039] The following describes the processing flow.

[0040] Step 1:

[0041] The server receives user activity history data from the terminal and stores it in a database. This data includes information such as past work activities, purchase history, and products used.

[0042] Step 2:

[0043] The server analyzes the stored data using machine learning algorithms to identify user behavior patterns. This allows it to extract actions that are repeated on specific days of the week or at specific times, and predict future actions.

[0044] Step 3:

[0045] Based on the analysis results, the server generates a list of recommended tasks and purchases for the user. The generated list clearly specifies the next tasks to be performed and the items to be purchased.

[0046] Step 4:

[0047] The server creates a notification message containing details of the proposed tasks and purchases, and sends that message to the user's device.

[0048] Step 5:

[0049] The device displays notifications received from the server on the user interface. Users can check these notifications within applications such as LINE.

[0050] Step 6:

[0051] Users review the suggestion notifications received on their devices and, if necessary, take action or purchase products using the provided links.

[0052] Step 7:

[0053] Feedback regarding user actions and choices is recorded on the device. The device then sends this feedback information to the server.

[0054] Step 8:

[0055] The server receives feedback from users and updates its learning model, allowing for continuous improvement in the accuracy of its suggestions.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] Providing appropriate behavioral suggestions based on individuals' diverse daily activities and purchasing behaviors is challenging. Furthermore, effectively utilizing individual feedback to improve the accuracy of these suggestions is a major challenge. To meet this need for personalized optimization, there is a demand for technologies that can provide efficient and highly accurate suggestions.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for collecting and storing an individual's behavioral history information, means for analyzing the stored behavioral history information to identify the individual's behavioral patterns, and means for generating suggestions for necessary tasks and purchases based on predicted behavior using a machine learning algorithm. This enables highly accurate suggestion generation tailored to the individual's behavioral patterns and efficient action decisions based on those suggestions.

[0061] "Personal behavioral history information" refers to information about an individual's past activities and actions, including their frequency and content. This information is used to identify specific behavioral patterns of an individual.

[0062] A "machine learning algorithm" refers to a computational method that allows computers to learn patterns and rules from large amounts of data to make future predictions and decisions. This algorithm can accumulate experience from data and apply that knowledge to new data.

[0063] "Behavioral patterns" refer to an individual's specific habitual actions, tendencies, and recurring behaviors. By identifying these patterns, it becomes possible to predict future actions and necessary suggestions in detail.

[0064] "Proposal accuracy" refers to the degree to which the proposed solution is appropriate and useful to the individual. This accuracy is an indicator of how well the proposal matches the individual's actual behavior and needs.

[0065] "Communication methods" refer to the technologies, devices, and protocols used to send and receive information. These methods make it possible to deliver generated proposals to individuals and collect their feedback.

[0066] "Responsive design" refers to a web design technique that automatically optimizes the displayed content according to the user's device and screen size. This allows for consistent usability across various devices.

[0067] In order to implement this invention, it is necessary for stakeholders and devices such as servers, terminals, and users to work together in coordination. The server first collects personal activity history information from the terminal. The terminal records data about the user's daily activities and purchasing behavior and sends it to the server. The server uses a database to receive and store this information. Specifically, database management systems such as MySQL® or PostgreSQL can be used.

[0068] The server can use Python, the R language, and machine learning libraries such as TENSORFLOW® and SciKit-Learn for data analysis. These help identify user behavior patterns from the dataset and build predictive models using machine learning algorithms. The server then generates appropriate action and purchase suggestions for individual users.

[0069] The generated proposals are sent from the server to the terminal, which then notifies the user via a messaging app. This notification technology can utilize general-purpose protocols such as WebSocket and HTTP / 2. The terminal employs responsive design to provide a user-friendly and easy-to-use interface.

[0070] Users review suggestions through their devices and decide on actions based on their content. Feedback on the user's choices and actions is recorded on the device and sent back to the server. The server processes this feedback as new data and uses it to update the machine learning model and improve the accuracy of the suggestions.

[0071] For example, if the server analyzes a user's spending patterns and discovers that they purchase a specific product every month, it will suggest the next purchase date and provide a link to purchase that product. This enables timely and appropriate suggestions tailored to the user. An example of a prompt to input into the generating AI model is, "Create a repurchase suggestion for a specific consumable based on the user's purchase history over the past three months."

[0072] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0073] Step 1:

[0074] The server collects personal activity history information from the terminal. The terminal periodically records user activity data and purchasing behavior and sends it to the server. This input data includes date and time, location, and information about purchased items. After receiving this data, the server performs a format conversion for storage in the database. For example, the data might be converted from JSON format to SQL table format. As output, a structured database entry is generated.

[0075] Step 2:

[0076] The server analyzes stored behavioral history information to identify user behavior patterns. In this step, database information is retrieved as input for analysis, and machine learning algorithms are used for the analysis. Specifically, the user's behavioral history is treated as time-series data, and pattern recognition is performed. Algorithms such as TensorFlow and SciKit-Learn are used to cluster behavioral patterns and extract features, and predictions of the next action are generated as output.

[0077] Step 3:

[0078] The server generates individual suggestions based on the analysis results. The prediction results generated in step 2 are used as input. Based on this information, it creates suggestions for the user's next actions and purchases. For example, if it suggests purchasing a specific product on a specific day, this suggestion will include a purchase link and the optimal time to do so. As output, it generates a data object of the suggested content.

[0079] Step 4:

[0080] The server sends the generated proposal to the terminal. The proposal data created in step 3 is used as input. This data is transferred to the terminal using a communication method. Specifically, data transfer is performed using a RESTful API or WebSocket. The terminal returns a confirmation of receipt as output.

[0081] Step 5:

[0082] The device notifies the user of suggestions received from the server. The input is suggestion information received from the server. This suggestion is sent to the user via a messaging app such as LINE. Specifically, a notification message is created and displayed according to the device's screen size using responsive design technology. The output is a visual notification to the user.

[0083] Step 6:

[0084] The user checks the notification and takes action based on the suggestion. The input is a suggestion notification from the device. If the user selects an action, the details are recorded on the device. For example, this could be an action such as clicking a purchase link to buy a product. As output, record data of the action is generated.

[0085] Step 7:

[0086] The terminal sends user behavior data to the server. The input is the behavior data recorded in step 6. By sending this to the server, it is saved again in the database and used as information for further analysis. Specifically, the data is encrypted and sent securely. The output is the behavior data saved in the database.

[0087] Step 8:

[0088] The server updates the machine learning model based on the collected feedback information. The input is the behavioral data received in step 7. The server uses this data to retrain the model and improve the accuracy of suggestions for subsequent steps. Specifically, it performs batch learning and fine-tunes the model with new data. The output is the updated machine learning model.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] Individuals often forget to purchase necessary items or miss the optimal timing for purchases in their daily work and lives. Furthermore, users struggle to make informed decisions about what products to buy and what actions to take, making it difficult to maintain an efficient lifestyle. This highlights the growing need for effective suggestion systems to improve quality of life.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes means for aggregating and storing individual behavioral history data, means for analyzing the stored behavioral history data and predicting specific actions of the individual, and means for providing consumables and recommended products at specific times based on the individual's past purchase history. This enables users to purchase necessary products at the optimal time, streamlining daily management and improving their quality of life.

[0094] "Activity history data" refers to information such as the content of activities and tasks performed by an individual on a daily basis, and the time spent performing them.

[0095] "Analysis" refers to the process of using stored data to identify regularities and patterns and predict individual behavior.

[0096] "Prediction" refers to the act of inferring the next tasks or purchases a user will need based on their past behavioral history.

[0097] "Suggestions" refer to recommending the best next course of action or items to purchase to a user based on their predicted behavior.

[0098] A "communication application" refers to software used to transmit information and notify users of proposed content.

[0099] "Individual responses" refer to users' responses and actions to suggestions, and recording them improves the accuracy of the data.

[0100] "Consumable goods" refer to items used in daily life that need to be repurchased over time.

[0101] A "recommended product" refers to an item that is suggested as the ideal next purchase based on the user's purchase history and behavioral patterns.

[0102] "Promotional and special offer information" refers to information about discounts or special treatment offered for specific products.

[0103] The system implementing this invention consists of a server, a user's terminal, and a communication application. The server implements a machine learning algorithm using Python and has an API using the Flask framework. MySQL is used as the database for aggregating, storing, and analyzing behavioral history data. The main processing performed by the server is to analyze the user's past behavioral history and identify individual behavioral patterns. This makes it possible to predict the user's next actions and the consumables they should purchase.

[0104] The user's device is assumed to be a smartphone, and an application developed using React Native will be installed. This application has the function of receiving notifications from the server in real time and providing information to the user through an intuitive interface. For communication, a general messaging software is used to ensure smooth communication with the user.

[0105] For example, if the server analyzes user behavior and determines that consumable item A is purchased in the third week of each month, a notification is sent to the device at the end of the second week stating, "We recommend checking your inventory and purchasing it this weekend if necessary." In this way, users can purchase necessary products at the optimal time. Furthermore, real-time promotional information is also provided, making purchasing behavior more efficient.

[0106] An example of a prompt given to a generative AI model is, "Design an algorithm that predicts a user's next purchase based on their past purchase history and provides notifications and recommended products at the appropriate time." Based on this prompt, an algorithm is generated that delivers highly accurate suggestions. This approach can provide personalized support to individual users and improve the efficiency of their lives.

[0107] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0108] Step 1:

[0109] The server receives activity history data from the user's device. This data includes the user's activities, time spent, and past purchase history. The received data is stored in a database. The input is the user's activity history data, and the output is a record of the stored data.

[0110] Step 2:

[0111] The server analyzes stored behavioral history data. Using a machine learning algorithm implemented in Python, it performs pattern recognition to identify each user's behavioral patterns. Based on the analysis results, the server predicts the consumables and tasks the user will likely need next. The input is behavioral history data, and the output is behavioral patterns and prediction results.

[0112] Step 3:

[0113] The server generates specific suggestions for the user based on predictions. These suggestions include information on necessary tasks, products to purchase, and links to relevant online stores and special offers. The input is the prediction result, and the output is the content of the suggestions.

[0114] Step 4:

[0115] The server sends the generated proposal to the terminal. The terminal immediately notifies the user via a communication application and displays the proposal content. The input is the proposal content, and the output is the notification provided to the user.

[0116] Step 5:

[0117] The user receives a notification and decides whether to take the suggested action. If they click a purchase link or buy a recommended product, the result and selection information are recorded as feedback on the device.

[0118] Step 6:

[0119] The device sends feedback information to the server. The server uses this information to update its machine learning model and improve the accuracy of the next prediction. The input is the user's feedback information, and the output is the updated machine learning model.

[0120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0121] The system according to the present invention has the function of providing optimized task suggestions and purchase suggestions based on an individual's behavior and emotional state. By combining a server, terminal, and emotion engine, this system achieves efficient user behavior management and personalized experiences based on emotions.

[0122] The server first receives user behavior history data and emotional state data collected by the emotion engine from the terminal and stores them in a database. The emotional state data includes information such as whether the user is stressed or relaxed.

[0123] Next, the server analyzes behavioral history data and emotional state data using machine learning algorithms to identify the user's behavioral patterns and adjust suggestions in response to changes in their emotions. As a result, the system predicts the next tasks and purchases that should be made, tailored to the user's mental and emotional state. The system anticipates emotional changes before they occur and suggests appropriate actions to the user, thereby encouraging better behavior.

[0124] Based on these analysis results, the server generates a list of appropriate tasks and purchases for the user. These suggestions include recommendations on when a user should perform a task, given their emotional state, and how they should choose their purchases.

[0125] The generated suggestions are sent to the terminal via a communication application. The terminal displays the suggestions received from the server in a format that is easy for the user to understand. For example, if the user is tired, it may suggest purchasing items to create a relaxing environment, providing detailed advice based on emotions.

[0126] Users check notifications on their devices, perform suggested tasks, or purchase products based on emotionally related suggestions. During this process, the user's actions and responses to emotional changes are recorded on the device and sent to the server via the emotion engine. The server uses this feedback to further update its machine learning model, continuously improving the quality and accuracy of its suggestions.

[0127] For example, if a user frequently experiences anxiety, the server can recommend consuming relaxing music or entertainment and encourage the purchase of related products (such as aromatherapy products or relaxation chairs). In this way, the goal is to provide optimal suggestions tailored to the user's emotional state, supporting not only daily work efficiency but also mental well-being.

[0128] The following describes the processing flow.

[0129] Step 1:

[0130] The device collects user behavior history data and emotional state data obtained by the emotion engine. The emotional state data includes information about the user's emotions obtained through technologies such as facial recognition and voice tone analysis.

[0131] Step 2:

[0132] The device sends collected behavioral history data and emotional state data to the server. The server stores this data in a database and manages it as an individual data stream.

[0133] Step 3:

[0134] The server uses machine learning algorithms to process the stored data and analyze user behavior patterns and emotional changes. The analysis identifies the user's current emotional state and tendencies.

[0135] Step 4:

[0136] Based on the analysis results, the server generates recommendations and purchase suggestions that take into account the user's current emotional state. This may include, for example, suggesting items that promote rest and enhance feelings of comfort when the user is feeling tired.

[0137] Step 5:

[0138] The server sends the generated proposals to the terminal via a communication application. The terminal displays the received proposals in its user interface for immediate review.

[0139] Step 6:

[0140] Users can review the displayed suggestions, perform the suggested tasks as needed, or click the purchase link to buy products online.

[0141] Step 7:

[0142] User reactions and selected actions are recorded on the device and sent back to the server as feedback. This is used to gain a more detailed understanding of user behavioral patterns and emotional changes.

[0143] Step 8:

[0144] The server receives feedback and updates its learning model to improve the accuracy of its suggestions. This allows it to continuously learn so that future suggestions become even more appropriate.

[0145] (Example 2)

[0146] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0147] In individual activities, there is a need for more effective and personalized suggestions based on behavioral history and emotional state, but conventional systems have been unable to meet this need. Furthermore, when making purchase suggestions, there has been no system that can accurately predict changes in emotions.

[0148] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0149] In this invention, the server includes means for aggregating and storing an individual's behavioral history data and emotional state data; means for analyzing the stored behavioral history data and emotional state data to optimize suggestions based on the individual's specific behaviors and emotional states; and means for generating suggestions for necessary tasks and purchases using an AI model generated from the analyzed data. This makes it possible to provide efficient and personalized suggestions while taking into account the individual's emotional state, thereby improving the accuracy of actions and the purchase of related products.

[0150] "Behavioral history data" refers to recorded information about an individual's daily activities and actions.

[0151] "Emotional state data" refers to information about the emotions and moods an individual is experiencing at a specific point in time.

[0152] "Aggregation" refers to the process of consolidating and managing multiple data sets in one place.

[0153] "Storage" refers to the act of permanently recording data so that it can be retrieved as needed.

[0154] "Analysis" refers to the process of identifying trends and patterns in collected data and using that information to gain specific insights.

[0155] A "generative AI model" refers to an artificial intelligence model that creates new information and suggestions based on a large amount of data.

[0156] A "proposal" refers to information that outlines actions or options to consider in order to achieve a specific objective.

[0157] "Communication technology" refers to the technical means and protocols used to send and receive data and information.

[0158] "Feedback" refers to an individual's response or result to a suggestion made by a system.

[0159] This invention is implemented as a system that utilizes individual behavioral history data and emotional state data to provide personalized suggestions to users. Specifically, a server, terminal, and emotion engine work together to perform these functions.

[0160] The server aggregates individual behavioral history data and emotional state data and stores it in a database system (e.g., Apache® Cassandra or MySQL) for centralized data management. It also uses open-source machine learning libraries such as Scikit-learn and TensorFlow as an analysis platform for executing machine learning algorithms. This analysis helps understand behavioral patterns and emotional changes, generating personalized suggestions.

[0161] The device plays the initial role of collecting user behavioral history data and emotional state data. This data is acquired in real time by mobile devices and sensors. This enables suggestions that reflect the user's emotional state. The device receives suggestions from the server and provides a user interface that displays them to the user in an easy-to-understand visual way. For example, if the user wants to relax, it can suggest music or relaxation products.

[0162] Users take action or make purchases based on suggestions displayed on their devices. During this process, the user's reactions and operation history are recorded on the device as feedback data. This feedback is sent to a server and used to optimize future suggestions.

[0163] For example, if a user frequently experiences anxiety, the server might suggest purchasing relaxing music or aromatherapy products. A possible prompt could be input to the generative AI model in the form of, "What emotional state is the user currently in? How can we make appropriate suggestions based on that?"

[0164] This system makes it possible to leverage data on users' emotions and behavior to continuously provide personalized suggestions for improving their lives.

[0165] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0166] Step 1:

[0167] The device collects user behavior history data and emotional state data. Inputs include sensor information from the mobile device (GPS location data, app usage history, heart rate, etc.) and a user interface for evaluating emotional state. Based on this data, the device records the user's daily activities and their emotional state at the time. Outputs include behavior history data and emotional state data in a format that can be stored in a database.

[0168] Step 2:

[0169] The terminal sends the collected data to the server. The inputs used are the behavioral history data and emotional state data obtained in step 1. The output is encrypted data, which is sent to the server via a secure communication protocol. The server receives this data and stores it in its internal database.

[0170] Step 3:

[0171] The server analyzes the received data. It uses behavioral history data and emotional state data obtained from a database as input. To analyze the data, it uses a Python machine learning library to identify behavioral patterns and changes in emotional state. The output provides personalized behavioral patterns and emotional fluctuation information for the user.

[0172] Step 4:

[0173] The server uses a generative AI model to create suggestions. It uses the analysis results and prompt text obtained from step 3 (e.g., "What emotional state is the user currently in? How can we make appropriate suggestions based on that?") as input. Based on this data, it generates suggestions using natural language generation technology. The output is text suggestions regarding the next task or purchase.

[0174] Step 5:

[0175] The server sends the generated proposal to the terminal. The proposal text generated in step 4 is used as input. As output, the proposal is sent to the terminal using communication technology and displayed on the terminal. The user can visually confirm this.

[0176] Step 6:

[0177] The user reviews the suggestions on the device and takes the necessary actions. For example, they might access an online store to purchase a recommended relaxation item. The results of the user's actions, and the resulting emotional changes, are recorded again on the device as feedback data.

[0178] Step 7:

[0179] The device collects user feedback and sends it to the server. The input uses the behavioral results and emotional changes recorded in step 6. The output is encrypted feedback data, which is then sent to the server. The server uses this data to update its machine learning model and improve the accuracy of future suggestions.

[0180] (Application Example 2)

[0181] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0182] Conventional behavior management systems offer suggestions based on user behavior history, but they are insufficient in suggesting appropriate content that takes emotional states into account in real time. In particular, recommending content tailored to emotions is crucial for supporting users' mental health, but no system effectively addresses this. Therefore, the challenge is to suggest optimal content in real time based on the user's emotional state, thereby improving mental health and enabling efficient behavior management.

[0183] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0184] In this invention, the server includes means for collecting and storing an individual's behavioral history information, means for storing and analyzing emotional state information obtained from an emotion recognition device, and means for analyzing the stored behavioral history information and emotional state information to predict the individual's specific actions. This makes it possible to suggest optimal tasks, purchases, and content appropriate to the user's emotional state in real time.

[0185] "Personal behavioral history information" refers to information about specific actions and choices that a user has made in the past.

[0186] An "emotion recognition device" is a technological device that analyzes images, sounds, etc., to detect the user's emotional state.

[0187] "Emotional state information" refers to information that indicates the user's psychological state (e.g., stress, relaxation, happiness, etc.).

[0188] "Communication technology" refers to the technical means used to transmit data to users in remote locations.

[0189] "Proposal accuracy" is an indicator that shows how well the generated proposals match the user's current state and needs.

[0190] A "machine learning model" is an algorithm that learns patterns from past data and makes predictions based on new data.

[0191] "Feedback information" refers to the results of collecting user responses and opinions to suggestions.

[0192] A "link for online purchase" is a URL provided to directly purchase a product when buying it via the internet.

[0193] This system enables content delivery using user behavior history and emotional state information by having the user's device, including their smartphone, communicate with a server. The server uses data collected from the smartphone and other sensors (e.g., camera, microphone, heart rate sensor). The emotion recognition device utilizes emotion analysis libraries (e.g., Google® Cloud Vision API, AWS® Rekognition) to analyze image and audio data and generate information about the user's emotional state.

[0194] The server analyzes collected behavioral history and emotional state information using machine learning algorithms. Based on the analysis results, it predicts and generates content suitable for the user's behavioral patterns and current emotions. It also uses communication technology to send the generated suggestions to the user's device, making it easy for the user to receive the suggested content.

[0195] For example, if the system detects that a user is experiencing stress, it will suggest music or video content that is expected to have a relaxing effect on the device. This allows the user to improve their mental health and maintain a state in which they can perform their daily tasks efficiently.

[0196] An example of a prompt to input into the generating AI model is: "Analyze the user's facial expression detection data and recommend appropriate relaxation content when the emotional state is determined to be 'stressed'."

[0197] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0198] Step 1:

[0199] The device collects user activity history information from sensors (e.g., smartphone accelerometer and GPS) and application logs. This information is stored in a database as the user's movement path and application usage history. The input is raw sensor data and log data, and the output is activity history information.

[0200] Step 2:

[0201] The device detects the user's emotional state using its built-in camera and microphone. The acquired image and audio data are analyzed using an emotion analysis library (e.g., AWS Rekognition) to identify the user's emotional state (e.g., happy, stressed, relaxed). The input is image and audio data, and the output is emotional state information.

[0202] Step 3:

[0203] The server receives the collected behavioral history and emotional state information and stores it in a database. This process uses the received behavioral history and emotional state information as input and transforms it into an appropriate data structure for storage. The output is an organized database entry.

[0204] Step 4:

[0205] The server uses machine learning algorithms to analyze stored behavioral history and emotional state information. This analysis makes estimations based on the user's behavioral patterns and emotions, and identifies the content that should be suggested next. The input is stored data, and the output is a list of suggested content.

[0206] Step 5:

[0207] The server notifies the terminal of the generated content suggestions. Here, the content list is sent via a communication protocol (e.g., HTTP / HTTPS) and displayed in a format visible to the user. The input is the generated content list, and the output is the notification displayed on the user's terminal screen.

[0208] Step 6:

[0209] The user reviews the suggestions notified from their device and then views or purchases the selected content. These user actions are recorded by the device as feedback information, which is then sent back to the server. The input is the suggested content, and the output is the feedback information.

[0210] Step 7:

[0211] The server updates the machine learning model based on the feedback information to improve the proposal accuracy. In this step, the latest feedback information is used as input, and the model's learning parameters are updated. The output is the new machine learning model with improved proposal accuracy.

[0212] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0213] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0214] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0215] [Second Embodiment]

[0216] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0217] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0218] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0219] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0220] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0221] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0222] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0223] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0224] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0225] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0226] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0227] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0228] The system according to the present invention efficiently manages an individual's daily work and lifestyle activities, providing necessary action suggestions and notifications to prevent forgotten purchases. This system exchanges data between the server, terminal, and user, aiming for operation optimized for the individual.

[0229] The server first collects individual behavioral history data from the terminal. This behavioral history data includes the content of the tasks performed by the individual, the time they were performed, and the history of products purchased in the past. The server analyzes this data to identify each individual's behavioral patterns. In doing so, it uses machine learning algorithms to perform more sophisticated pattern recognition and predict the user's next actions and necessary purchases.

[0230] Based on the predictions obtained through analysis, the server generates personalized suggestions for the user's next actions and purchases. These suggestions include information such as what tasks the user should perform next and when would be the best time to perform them. For purchases, it also suggests the optimal time to buy and includes links to online stores where the items can be purchased.

[0231] The generated proposals are sent to the terminal via a communication application. The terminal then notifies the user of the proposals received from the server in an easy-to-understand format. This notification can be sent via a messaging application such as LINE, allowing the user to immediately review the notification and decide on an action based on the proposals.

[0232] Users check notifications received on their devices and perform suggested tasks or click purchase links to buy products. During this process, the user's actions and selected responses are recorded on the device, and this feedback information is sent to the server. The server uses this feedback to update its machine learning model and improve the accuracy of its suggestions.

[0233] For example, if the server analyzes a user's monthly spending patterns and discovers a tendency to purchase certain consumables at the end of the month, the server will suggest repurchasing those consumables near the end of the month and provide the user with a link to an online store where those consumables can be purchased. In this way, the present invention enables improvements in work efficiency and quality of life by providing personalized suggestions to individual users.

[0234] The following describes the processing flow.

[0235] Step 1:

[0236] The server receives user activity history data from the terminal and stores it in a database. This data includes information such as past work activities, purchase history, and products used.

[0237] Step 2:

[0238] The server analyzes the stored data using machine learning algorithms to identify user behavior patterns. This allows it to extract actions that are repeated on specific days of the week or at specific times, and predict future actions.

[0239] Step 3:

[0240] Based on the analysis results, the server generates a list of recommended tasks and purchases for the user. The generated list clearly specifies the next tasks to be performed and the items to be purchased.

[0241] Step 4:

[0242] The server creates a notification message containing details of the proposed tasks and purchases, and sends that message to the user's device.

[0243] Step 5:

[0244] The device displays notifications received from the server on the user interface. Users can check these notifications within applications such as LINE.

[0245] Step 6:

[0246] Users review the suggestion notifications received on their devices and, if necessary, take action or purchase products using the provided links.

[0247] Step 7:

[0248] Feedback regarding user actions and choices is recorded on the device. The device then sends this feedback information to the server.

[0249] Step 8:

[0250] The server receives feedback from users and updates its learning model, allowing for continuous improvement in the accuracy of its suggestions.

[0251] (Example 1)

[0252] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0253] Providing appropriate behavioral suggestions based on individuals' diverse daily activities and purchasing behaviors is challenging. Furthermore, effectively utilizing individual feedback to improve the accuracy of these suggestions is a major challenge. To meet this need for personalized optimization, there is a demand for technologies that can provide efficient and highly accurate suggestions.

[0254] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0255] In this invention, the server includes means for collecting and storing an individual's behavioral history information, means for analyzing the stored behavioral history information to identify the individual's behavioral patterns, and means for generating suggestions for necessary tasks and purchases based on predicted behavior using a machine learning algorithm. This enables highly accurate suggestion generation tailored to the individual's behavioral patterns and efficient action decisions based on those suggestions.

[0256] "Personal behavioral history information" refers to information about an individual's past activities and actions, including their frequency and content. This information is used to identify specific behavioral patterns of an individual.

[0257] A "machine learning algorithm" refers to a computational method that allows computers to learn patterns and rules from large amounts of data to make future predictions and decisions. This algorithm can accumulate experience from data and apply that knowledge to new data.

[0258] "Behavioral patterns" refer to an individual's specific habitual actions, tendencies, and recurring behaviors. By identifying these patterns, it becomes possible to predict future actions and necessary suggestions in detail.

[0259] "Proposal accuracy" refers to the degree to which the proposed solution is appropriate and useful to the individual. This accuracy is an indicator of how well the proposal matches the individual's actual behavior and needs.

[0260] "Communication methods" refer to the technologies, devices, and protocols used to send and receive information. These methods make it possible to deliver generated proposals to individuals and collect their feedback.

[0261] "Responsive design" refers to a web design technique that automatically optimizes the displayed content according to the user's device and screen size. This allows for consistent usability across various devices.

[0262] In order to implement this invention, it is necessary for stakeholders and devices such as servers, terminals, and users to work in coordination. The server first collects personal activity history information from the terminal. The terminal records data about the user's daily activities and purchasing behavior and sends it to the server. The server uses a database to receive and store this information. Specifically, database management systems such as MySQL and PostgreSQL can be used.

[0263] The server can use Python, the R language, and machine learning libraries such as TensorFlow and SciKit-Learn for data analysis. These help identify user behavior patterns from the dataset and build predictive models using machine learning algorithms. The server then generates appropriate action and purchase suggestions for individual users.

[0264] The generated proposals are sent from the server to the terminal, which then notifies the user via a messaging app. This notification technology can utilize general-purpose protocols such as WebSocket and HTTP / 2. The terminal employs responsive design to provide a user-friendly and easy-to-use interface.

[0265] Users review suggestions through their devices and decide on actions based on their content. Feedback on the user's choices and actions is recorded on the device and sent back to the server. The server processes this feedback as new data and uses it to update the machine learning model and improve the accuracy of the suggestions.

[0266] For example, if the server analyzes a user's spending patterns and discovers that they purchase a specific product every month, it will suggest the next purchase date and provide a link to purchase that product. This enables timely and appropriate suggestions tailored to the user. An example of a prompt to input into the generating AI model is, "Create a repurchase suggestion for a specific consumable based on the user's purchase history over the past three months."

[0267] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0268] Step 1:

[0269] The server collects personal activity history information from the terminal. The terminal periodically records user activity data and purchasing behavior and sends it to the server. This input data includes date and time, location, and information about purchased items. After receiving this data, the server performs a format conversion for storage in the database. For example, the data might be converted from JSON format to SQL table format. As output, a structured database entry is generated.

[0270] Step 2:

[0271] The server analyzes stored behavioral history information to identify user behavior patterns. In this step, database information is retrieved as input for analysis, and machine learning algorithms are used for the analysis. Specifically, the user's behavioral history is treated as time-series data, and pattern recognition is performed. Algorithms such as TensorFlow and SciKit-Learn are used to cluster behavioral patterns and extract features, and predictions of the next action are generated as output.

[0272] Step 3:

[0273] The server generates individual suggestions based on the analysis results. The prediction results generated in step 2 are used as input. Based on this information, it creates suggestions for the user's next actions and purchases. For example, if it suggests purchasing a specific product on a specific day, this suggestion will include a purchase link and the optimal time to do so. As output, it generates a data object of the suggested content.

[0274] Step 4:

[0275] The server sends the generated proposal to the terminal. The proposal data created in step 3 is used as input. This data is transferred to the terminal using a communication method. Specifically, data transfer is performed using a RESTful API or WebSocket. The terminal returns a confirmation of receipt as output.

[0276] Step 5:

[0277] The terminal notifies the user of the proposal received from the server. As input, there is proposal information received from the server. This proposal is sent to the user through a messaging app such as LINE. Specifically, a notification message is created and displayed according to the screen size of the device using responsive design technology. As output, a visual notification to the user is made.

[0278] Step 6:

[0279] The user checks the notification and performs an action based on the proposal. As input, there is a proposal notification from the terminal. When the user selects an action, the content is recorded on the terminal. For example, actions such as clicking on a purchase link to purchase a product can be considered. As output, action record data is generated.

[0280] Step 7:

[0281] The terminal sends the user's action data to the server. As input, there is the action data recorded in Step 6. By sending this to the server, it is saved in the database again and used as information for the next analysis. Specifically, the data is encrypted and securely transmitted. As output, the action data is saved in the database.

[0282] Step 8:

[0283] The server updates the machine learning model based on the accumulated feedback information. As input, there is the action data received in Step 7. The server uses these data to perform re-learning of the model and improve the proposal accuracy for subsequent times. As a specific process, batch learning is performed and the model is fine-tuned with new data. As output, an updated machine learning model is generated.

[0284] (Application Example 1)

[0285] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0286] In an individual's daily work and life, problems occur such as forgetting to purchase necessary items or missing the optimal timing for purchase. Also, it is difficult for users to appropriately judge the products or actions they should purchase, making it difficult to maintain an efficient life. As a result, the need for an effective proposal system to improve the quality of life is increasing.

[0287] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0288] In this invention, the server includes means for aggregating and storing an individual's action history data, means for analyzing the stored action history data and predicting an individual's specific actions, and means for providing consumables and recommended products at specific timings based on an individual's past purchase history. As a result, it becomes possible for the user to purchase necessary products at the optimal timing, streamline daily management, and improve the quality of life.

[0289] "Action history data" refers to information such as the content and implementation time of activities and tasks that an individual performs daily.

[0290] "Analysis" refers to a process of finding regularities and patterns using the stored data to predict an individual's actions.

[0291] "Prediction" refers to an act of speculating about the next necessary work or purchases of a user based on past action history.

[0292] "Proposal" refers to recommending the optimal actions that a user should take next or the items that should be purchased based on the predicted actions.

[0293] "Communication application" refers to software for information transmission used to notify a user of the proposed content.

[0294] "Individual responses" refer to users' responses and actions to suggestions, and recording them improves the accuracy of the data.

[0295] "Consumable goods" refer to items used in daily life that need to be repurchased over time.

[0296] A "recommended product" refers to an item that is suggested as the ideal next purchase based on the user's purchase history and behavioral patterns.

[0297] "Promotional and special offer information" refers to information about discounts or special treatment offered for specific products.

[0298] The system implementing this invention consists of a server, a user's terminal, and a communication application. The server implements a machine learning algorithm using Python and has an API using the Flask framework. MySQL is used as the database for aggregating, storing, and analyzing behavioral history data. The main processing performed by the server is to analyze the user's past behavioral history and identify individual behavioral patterns. This makes it possible to predict the user's next actions and the consumables they should purchase.

[0299] The user's device is assumed to be a smartphone, and an application developed using React Native will be installed. This application has the function of receiving notifications from the server in real time and providing information to the user through an intuitive interface. For communication, a general messaging software is used to ensure smooth communication with the user.

[0300] As a specific example, when the server analyzes the user's behavior and determines that consumable A is being purchased in the third week of each month, a notification saying "Check the inventory and if necessary, it is recommended to purchase this weekend" is sent to the terminal at the end of the second week. In this way, the user can purchase the necessary products at the optimal timing. Also, since promotional information is provided in real time, the purchasing behavior becomes more efficient.

[0301] As an example of the prompt text input into the generative AI model, there is something like "Based on the user's past purchase history, predict the next purchase and design an algorithm to provide notifications and recommended products at appropriate times." Based on this prompt, an algorithm that realizes highly accurate proposals is generated. By this approach, it is possible to provide support optimized for individual users and improve the efficiency of life.

[0302] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0303] Step 1:

[0304] The server receives action history data from the user's terminal. This data includes the activities the user has done, the time consumed, and the past purchase history. The received data is saved in the database. The input is the user's action history data, and the output is the record of the saved data.

[0305] Step 2:

[0306] The server analyzes the saved action history data. Pattern recognition is performed using a machine learning algorithm implemented in Python to identify the action pattern for each user. At this time, the server predicts the consumables and operations that the user will need next as the analysis result. The input is the action history data, and the output is the action pattern and the prediction result.

[0307] Step 3:

[0308] The server generates specific suggestions for the user based on predictions. These suggestions include information on necessary tasks, products to purchase, and links to relevant online stores and special offers. The input is the prediction result, and the output is the content of the suggestions.

[0309] Step 4:

[0310] The server sends the generated proposal to the terminal. The terminal immediately notifies the user via a communication application and displays the proposal content. The input is the proposal content, and the output is the notification provided to the user.

[0311] Step 5:

[0312] The user receives a notification and decides whether to take the suggested action. If they click a purchase link or buy a recommended product, the result and selection information are recorded as feedback on the device.

[0313] Step 6:

[0314] The device sends feedback information to the server. The server uses this information to update its machine learning model and improve the accuracy of the next prediction. The input is the user's feedback information, and the output is the updated machine learning model.

[0315] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0316] The system according to the present invention has the function of providing optimized task suggestions and purchase suggestions based on an individual's behavior and emotional state. By combining a server, terminal, and emotion engine, this system achieves efficient user behavior management and personalized experiences based on emotions.

[0317] The server first receives user behavior history data and emotional state data collected by the emotion engine from the terminal and stores them in a database. The emotional state data includes information such as whether the user is stressed or relaxed.

[0318] Next, the server analyzes behavioral history data and emotional state data using machine learning algorithms to identify the user's behavioral patterns and adjust suggestions in response to changes in their emotions. As a result, the system predicts the next tasks and purchases that should be made, tailored to the user's mental and emotional state. The system anticipates emotional changes before they occur and suggests appropriate actions to the user, thereby encouraging better behavior.

[0319] Based on these analysis results, the server generates a list of appropriate tasks and purchases for the user. These suggestions include recommendations on when a user should perform a task, given their emotional state, and how they should choose their purchases.

[0320] The generated suggestions are sent to the terminal via a communication application. The terminal displays the suggestions received from the server in a format that is easy for the user to understand. For example, if the user is tired, it may suggest purchasing items to create a relaxing environment, providing detailed advice based on emotions.

[0321] Users check notifications on their devices, perform suggested tasks, or purchase products based on emotionally related suggestions. During this process, the user's actions and responses to emotional changes are recorded on the device and sent to the server via the emotion engine. The server uses this feedback to further update its machine learning model, continuously improving the quality and accuracy of its suggestions.

[0322] For example, if a user frequently experiences anxiety, the server can recommend consuming relaxing music or entertainment and encourage the purchase of related products (such as aromatherapy products or relaxation chairs). In this way, the goal is to provide optimal suggestions tailored to the user's emotional state, supporting not only daily work efficiency but also mental well-being.

[0323] The following describes the processing flow.

[0324] Step 1:

[0325] The device collects user behavior history data and emotional state data obtained by the emotion engine. The emotional state data includes information about the user's emotions obtained through technologies such as facial recognition and voice tone analysis.

[0326] Step 2:

[0327] The device sends collected behavioral history data and emotional state data to the server. The server stores this data in a database and manages it as an individual data stream.

[0328] Step 3:

[0329] The server uses machine learning algorithms to process the stored data and analyze user behavior patterns and emotional changes. The analysis identifies the user's current emotional state and tendencies.

[0330] Step 4:

[0331] Based on the analysis results, the server generates recommendations and purchase suggestions that take into account the user's current emotional state. This may include, for example, suggesting items that promote rest and enhance feelings of comfort when the user is feeling tired.

[0332] Step 5:

[0333] The server sends the generated proposals to the terminal via a communication application. The terminal displays the received proposals in its user interface for immediate review.

[0334] Step 6:

[0335] Users can review the displayed suggestions, perform the suggested tasks as needed, or click the purchase link to buy products online.

[0336] Step 7:

[0337] User reactions and selected actions are recorded on the device and sent back to the server as feedback. This is used to gain a more detailed understanding of user behavioral patterns and emotional changes.

[0338] Step 8:

[0339] The server receives feedback and updates its learning model to improve the accuracy of its suggestions. This allows it to continuously learn so that future suggestions become even more appropriate.

[0340] (Example 2)

[0341] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0342] In individual activities, there is a need for more effective and personalized suggestions based on behavioral history and emotional state, but conventional systems have been unable to meet this need. Furthermore, when making purchase suggestions, there has been no system that can accurately predict changes in emotions.

[0343] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0344] In this invention, the server includes means for aggregating and storing an individual's behavioral history data and emotional state data; means for analyzing the stored behavioral history data and emotional state data to optimize suggestions based on the individual's specific behaviors and emotional states; and means for generating suggestions for necessary tasks and purchases using an AI model generated from the analyzed data. This makes it possible to provide efficient and personalized suggestions while taking into account the individual's emotional state, thereby improving the accuracy of actions and the purchase of related products.

[0345] "Behavioral history data" refers to recorded information about an individual's daily activities and actions.

[0346] "Emotional state data" refers to information about the emotions and moods an individual is experiencing at a specific point in time.

[0347] "Aggregation" refers to the process of consolidating and managing multiple data sets in one place.

[0348] "Storage" refers to the act of permanently recording data so that it can be retrieved as needed.

[0349] "Analysis" refers to the process of identifying trends and patterns in collected data and using that information to gain specific insights.

[0350] A "generative AI model" refers to an artificial intelligence model that creates new information and suggestions based on a large amount of data.

[0351] A "proposal" refers to information that outlines actions or options to consider in order to achieve a specific objective.

[0352] "Communication technology" refers to the technical means and protocols used to send and receive data and information.

[0353] "Feedback" refers to an individual's response or result to a suggestion made by a system.

[0354] This invention is implemented as a system that utilizes individual behavioral history data and emotional state data to provide personalized suggestions to users. Specifically, a server, terminal, and emotion engine work together to perform these functions.

[0355] The server aggregates individual behavioral history data and emotional state data and stores it in a database system (e.g., Apache Cassandra or MySQL) for centralized data management. It also uses open-source machine learning libraries such as Scikit-learn and TensorFlow as an analysis platform for executing machine learning algorithms. This analysis helps understand behavioral patterns and emotional changes, generating personalized suggestions.

[0356] The device plays the initial role of collecting user behavioral history data and emotional state data. This data is acquired in real time by mobile devices and sensors. This enables suggestions that reflect the user's emotional state. The device receives suggestions from the server and provides a user interface that displays them to the user in an easy-to-understand visual way. For example, if the user wants to relax, it can suggest music or relaxation products.

[0357] Users take action or make purchases based on suggestions displayed on their devices. During this process, the user's reactions and operation history are recorded on the device as feedback data. This feedback is sent to a server and used to optimize future suggestions.

[0358] For example, if a user frequently experiences anxiety, the server might suggest purchasing relaxing music or aromatherapy products. A possible prompt could be input to the generative AI model in the form of, "What emotional state is the user currently in? How can we make appropriate suggestions based on that?"

[0359] This system makes it possible to leverage data on users' emotions and behavior to continuously provide personalized suggestions for improving their lives.

[0360] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0361] Step 1:

[0362] The device collects user behavior history data and emotional state data. Inputs include sensor information from the mobile device (GPS location data, app usage history, heart rate, etc.) and a user interface for evaluating emotional state. Based on this data, the device records the user's daily activities and their emotional state at the time. Outputs include behavior history data and emotional state data in a format that can be stored in a database.

[0363] Step 2:

[0364] The terminal sends the collected data to the server. The inputs used are the behavioral history data and emotional state data obtained in step 1. The output is encrypted data, which is sent to the server via a secure communication protocol. The server receives this data and stores it in its internal database.

[0365] Step 3:

[0366] The server analyzes the received data. It uses behavioral history data and emotional state data obtained from a database as input. To analyze the data, it uses a Python machine learning library to identify behavioral patterns and changes in emotional state. The output provides personalized behavioral patterns and emotional fluctuation information for the user.

[0367] Step 4:

[0368] The server uses a generative AI model to create suggestions. It uses the analysis results and prompt text obtained from step 3 (e.g., "What emotional state is the user currently in? How can we make appropriate suggestions based on that?") as input. Based on this data, it generates suggestions using natural language generation technology. The output is text suggestions regarding the next task or purchase.

[0369] Step 5:

[0370] The server sends the generated proposal to the terminal. The proposal text generated in step 4 is used as input. As output, the proposal is sent to the terminal using communication technology and displayed on the terminal. The user can visually confirm this.

[0371] Step 6:

[0372] The user reviews the suggestions on the device and takes the necessary actions. For example, they might access an online store to purchase a recommended relaxation item. The results of the user's actions, and the resulting emotional changes, are recorded again on the device as feedback data.

[0373] Step 7:

[0374] The device collects user feedback and sends it to the server. The input uses the behavioral results and emotional changes recorded in step 6. The output is encrypted feedback data, which is then sent to the server. The server uses this data to update its machine learning model and improve the accuracy of future suggestions.

[0375] (Application Example 2)

[0376] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0377] Conventional behavior management systems offer suggestions based on user behavior history, but they are insufficient in suggesting appropriate content that takes emotional states into account in real time. In particular, recommending content tailored to emotions is crucial for supporting users' mental health, but no system effectively addresses this. Therefore, the challenge is to suggest optimal content in real time based on the user's emotional state, thereby improving mental health and enabling efficient behavior management.

[0378] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0379] In this invention, the server includes means for collecting and storing an individual's behavioral history information, means for storing and analyzing emotional state information obtained from an emotion recognition device, and means for analyzing the stored behavioral history information and emotional state information to predict the individual's specific actions. This makes it possible to suggest optimal tasks, purchases, and content appropriate to the user's emotional state in real time.

[0380] "Personal behavioral history information" refers to information about specific actions and choices that a user has made in the past.

[0381] An "emotion recognition device" is a technological device that analyzes images, sounds, etc., to detect the user's emotional state.

[0382] "Emotional state information" refers to information that indicates the user's psychological state (e.g., stress, relaxation, happiness, etc.).

[0383] "Communication technology" refers to the technical means used to transmit data to users in remote locations.

[0384] "Proposal accuracy" is an indicator that shows how well the generated proposals match the user's current state and needs.

[0385] A "machine learning model" is an algorithm that learns patterns from past data and makes predictions based on new data.

[0386] "Feedback information" refers to the results of collecting user responses and opinions to suggestions.

[0387] A "link for online purchase" is a URL provided to directly purchase a product when buying it via the internet.

[0388] This system enables content delivery using user behavior history and emotional state information by having the user's device, including their smartphone, communicate with a server. The server uses data collected from the smartphone and other sensors (e.g., camera, microphone, heart rate sensor). The emotion recognition device utilizes emotion analysis libraries (e.g., Google Cloud Vision API, AWS Rekognition) to analyze image and audio data and generate information about the user's emotional state.

[0389] The server analyzes collected behavioral history and emotional state information using machine learning algorithms. Based on the analysis results, it predicts and generates content suitable for the user's behavioral patterns and current emotions. It also uses communication technology to send the generated suggestions to the user's device, making it easy for the user to receive the suggested content.

[0390] For example, if the system detects that a user is experiencing stress, it will suggest music or video content that is expected to have a relaxing effect on the device. This allows the user to improve their mental health and maintain a state in which they can perform their daily tasks efficiently.

[0391] An example of a prompt to input into the generating AI model is: "Analyze the user's facial expression detection data and recommend appropriate relaxation content when the emotional state is determined to be 'stressed'."

[0392] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0393] Step 1:

[0394] The device collects user activity history information from sensors (e.g., smartphone accelerometer and GPS) and application logs. This information is stored in a database as the user's movement path and application usage history. The input is raw sensor data and log data, and the output is activity history information.

[0395] Step 2:

[0396] The device detects the user's emotional state using its built-in camera and microphone. The acquired image and audio data are analyzed using an emotion analysis library (e.g., AWS Rekognition) to identify the user's emotional state (e.g., happy, stressed, relaxed). The input is image and audio data, and the output is emotional state information.

[0397] Step 3:

[0398] The server receives the collected behavioral history and emotional state information and stores it in a database. This process uses the received behavioral history and emotional state information as input and transforms it into an appropriate data structure for storage. The output is an organized database entry.

[0399] Step 4:

[0400] The server uses machine learning algorithms to analyze stored behavioral history and emotional state information. This analysis makes estimations based on the user's behavioral patterns and emotions, and identifies the content that should be suggested next. The input is stored data, and the output is a list of suggested content.

[0401] Step 5:

[0402] The server notifies the terminal of the generated content suggestions. Here, the content list is sent via a communication protocol (e.g., HTTP / HTTPS) and displayed in a format visible to the user. The input is the generated content list, and the output is the notification displayed on the user's terminal screen.

[0403] Step 6:

[0404] The user reviews the suggestions notified from their device and then views or purchases the selected content. These user actions are recorded by the device as feedback information, which is then sent back to the server. The input is the suggested content, and the output is the feedback information.

[0405] Step 7:

[0406] The server updates the machine learning model based on the feedback information to improve the proposal accuracy. In this step, the latest feedback information is used as input, and the model's learning parameters are updated. The output is the new machine learning model with improved proposal accuracy.

[0407] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0408] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0409] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0410] [Third Embodiment]

[0411] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0412] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0413] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0414] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0415] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0416] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0417] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0418] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0419] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0420] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0421] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0422] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0423] The system according to the present invention efficiently manages an individual's daily work and lifestyle activities, providing necessary action suggestions and notifications to prevent forgotten purchases. This system exchanges data between the server, terminal, and user, aiming for operation optimized for the individual.

[0424] The server first collects individual behavioral history data from the terminal. This behavioral history data includes the content of the tasks performed by the individual, the time they were performed, and the history of products purchased in the past. The server analyzes this data to identify each individual's behavioral patterns. In doing so, it uses machine learning algorithms to perform more sophisticated pattern recognition and predict the user's next actions and necessary purchases.

[0425] Based on the predictions obtained through analysis, the server generates personalized suggestions for the user's next actions and purchases. These suggestions include information such as what tasks the user should perform next and when would be the best time to perform them. For purchases, it also suggests the optimal time to buy and includes links to online stores where the items can be purchased.

[0426] The generated proposals are sent to the terminal via a communication application. The terminal then notifies the user of the proposals received from the server in an easy-to-understand format. This notification can be sent via a messaging application such as LINE, allowing the user to immediately review the notification and decide on an action based on the proposals.

[0427] Users check notifications received on their devices and perform suggested tasks or click purchase links to buy products. During this process, the user's actions and selected responses are recorded on the device, and this feedback information is sent to the server. The server uses this feedback to update its machine learning model and improve the accuracy of its suggestions.

[0428] For example, if the server analyzes a user's monthly spending patterns and discovers a tendency to purchase certain consumables at the end of the month, the server will suggest repurchasing those consumables near the end of the month and provide the user with a link to an online store where those consumables can be purchased. In this way, the present invention enables improvements in work efficiency and quality of life by providing personalized suggestions to individual users.

[0429] The following describes the processing flow.

[0430] Step 1:

[0431] The server receives user activity history data from the terminal and stores it in a database. This data includes information such as past work activities, purchase history, and products used.

[0432] Step 2:

[0433] The server analyzes the stored data using machine learning algorithms to identify user behavior patterns. This allows it to extract actions that are repeated on specific days of the week or at specific times, and predict future actions.

[0434] Step 3:

[0435] Based on the analysis results, the server generates a list of recommended tasks and purchases for the user. The generated list clearly specifies the next tasks to be performed and the items to be purchased.

[0436] Step 4:

[0437] The server creates a notification message containing details of the proposed tasks and purchases, and sends that message to the user's device.

[0438] Step 5:

[0439] The device displays notifications received from the server on the user interface. Users can check these notifications within applications such as LINE.

[0440] Step 6:

[0441] Users review the suggestion notifications received on their devices and, if necessary, take action or purchase products using the provided links.

[0442] Step 7:

[0443] Feedback regarding user actions and choices is recorded on the device. The device then sends this feedback information to the server.

[0444] Step 8:

[0445] The server receives feedback from users and updates its learning model, allowing for continuous improvement in the accuracy of its suggestions.

[0446] (Example 1)

[0447] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0448] Providing appropriate behavioral suggestions based on individuals' diverse daily activities and purchasing behaviors is challenging. Furthermore, effectively utilizing individual feedback to improve the accuracy of these suggestions is a major challenge. To meet this need for personalized optimization, there is a demand for technologies that can provide efficient and highly accurate suggestions.

[0449] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0450] In this invention, the server includes means for collecting and storing an individual's behavioral history information, means for analyzing the stored behavioral history information to identify the individual's behavioral patterns, and means for generating suggestions for necessary tasks and purchases based on predicted behavior using a machine learning algorithm. This enables highly accurate suggestion generation tailored to the individual's behavioral patterns and efficient action decisions based on those suggestions.

[0451] "Personal behavioral history information" refers to information about an individual's past activities and actions, including their frequency and content. This information is used to identify specific behavioral patterns of an individual.

[0452] A "machine learning algorithm" refers to a computational method that allows computers to learn patterns and rules from large amounts of data to make future predictions and decisions. This algorithm can accumulate experience from data and apply that knowledge to new data.

[0453] "Behavioral patterns" refer to an individual's specific habitual actions, tendencies, and recurring behaviors. By identifying these patterns, it becomes possible to predict future actions and necessary suggestions in detail.

[0454] "Proposal accuracy" refers to the degree to which the proposed solution is appropriate and useful to the individual. This accuracy is an indicator of how well the proposal matches the individual's actual behavior and needs.

[0455] "Communication methods" refer to the technologies, devices, and protocols used to send and receive information. These methods make it possible to deliver generated proposals to individuals and collect their feedback.

[0456] "Responsive design" refers to a web design technique that automatically optimizes the displayed content according to the user's device and screen size. This allows for consistent usability across various devices.

[0457] In order to implement this invention, it is necessary for stakeholders and devices such as servers, terminals, and users to work in coordination. The server first collects personal activity history information from the terminal. The terminal records data about the user's daily activities and purchasing behavior and sends it to the server. The server uses a database to receive and store this information. Specifically, database management systems such as MySQL and PostgreSQL can be used.

[0458] The server can use Python, the R language, and machine learning libraries such as TensorFlow and SciKit-Learn for data analysis. These help identify user behavior patterns from the dataset and build predictive models using machine learning algorithms. The server then generates appropriate action and purchase suggestions for individual users.

[0459] The generated proposals are sent from the server to the terminal, which then notifies the user via a messaging app. This notification technology can utilize general-purpose protocols such as WebSocket and HTTP / 2. The terminal employs responsive design to provide a user-friendly and easy-to-use interface.

[0460] Users review suggestions through their devices and decide on actions based on their content. Feedback on the user's choices and actions is recorded on the device and sent back to the server. The server processes this feedback as new data and uses it to update the machine learning model and improve the accuracy of the suggestions.

[0461] For example, if the server analyzes a user's spending patterns and discovers that they purchase a specific product every month, it will suggest the next purchase date and provide a link to purchase that product. This enables timely and appropriate suggestions tailored to the user. An example of a prompt to input into the generating AI model is, "Create a repurchase suggestion for a specific consumable based on the user's purchase history over the past three months."

[0462] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0463] Step 1:

[0464] The server collects personal activity history information from the terminal. The terminal periodically records user activity data and purchasing behavior and sends it to the server. This input data includes date and time, location, and information about purchased items. After receiving this data, the server performs a format conversion for storage in the database. For example, the data might be converted from JSON format to SQL table format. As output, a structured database entry is generated.

[0465] Step 2:

[0466] The server analyzes stored behavioral history information to identify user behavior patterns. In this step, database information is retrieved as input for analysis, and machine learning algorithms are used for the analysis. Specifically, the user's behavioral history is treated as time-series data, and pattern recognition is performed. Algorithms such as TensorFlow and SciKit-Learn are used to cluster behavioral patterns and extract features, and predictions of the next action are generated as output.

[0467] Step 3:

[0468] The server generates individual suggestions based on the analysis results. The prediction results generated in step 2 are used as input. Based on this information, it creates suggestions for the user's next actions and purchases. For example, if it suggests purchasing a specific product on a specific day, this suggestion will include a purchase link and the optimal time to do so. As output, it generates a data object of the suggested content.

[0469] Step 4:

[0470] The server sends the generated proposal to the terminal. The proposal data created in step 3 is used as input. This data is transferred to the terminal using a communication method. Specifically, data transfer is performed using a RESTful API or WebSocket. The terminal returns a confirmation of receipt as output.

[0471] Step 5:

[0472] The device notifies the user of suggestions received from the server. The input is suggestion information received from the server. This suggestion is sent to the user via a messaging app such as LINE. Specifically, a notification message is created and displayed according to the device's screen size using responsive design technology. The output is a visual notification to the user.

[0473] Step 6:

[0474] The user checks the notification and takes action based on the suggestion. The input is a suggestion notification from the device. If the user selects an action, the details are recorded on the device. For example, this could be an action such as clicking a purchase link to buy a product. As output, record data of the action is generated.

[0475] Step 7:

[0476] The terminal sends user behavior data to the server. The input is the behavior data recorded in step 6. By sending this to the server, it is saved again in the database and used as information for further analysis. Specifically, the data is encrypted and sent securely. The output is the behavior data saved in the database.

[0477] Step 8:

[0478] The server updates the machine learning model based on the collected feedback information. The input is the behavioral data received in step 7. The server uses this data to retrain the model and improve the accuracy of suggestions for subsequent steps. Specifically, it performs batch learning and fine-tunes the model with new data. The output is the updated machine learning model.

[0479] (Application Example 1)

[0480] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0481] Individuals often forget to purchase necessary items or miss the optimal timing for purchases in their daily work and lives. Furthermore, users struggle to make informed decisions about what products to buy and what actions to take, making it difficult to maintain an efficient lifestyle. This highlights the growing need for effective suggestion systems to improve quality of life.

[0482] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0483] In this invention, the server includes means for aggregating and storing individual behavioral history data, means for analyzing the stored behavioral history data and predicting specific actions of the individual, and means for providing consumables and recommended products at specific times based on the individual's past purchase history. This enables users to purchase necessary products at the optimal time, streamlining daily management and improving their quality of life.

[0484] "Activity history data" refers to information such as the content of activities and tasks performed by an individual on a daily basis, and the time spent performing them.

[0485] "Analysis" refers to the process of using stored data to identify regularities and patterns and predict individual behavior.

[0486] "Prediction" refers to the act of inferring the next tasks or purchases a user will need based on their past behavioral history.

[0487] "Suggestions" refer to recommending the best next course of action or items to purchase to a user based on their predicted behavior.

[0488] A "communication application" refers to software used to transmit information and notify users of proposed content.

[0489] "Individual responses" refer to users' responses and actions to suggestions, and recording them improves the accuracy of the data.

[0490] "Consumable goods" refer to items used in daily life that need to be repurchased over time.

[0491] A "recommended product" refers to an item that is suggested as the ideal next purchase based on the user's purchase history and behavioral patterns.

[0492] "Promotional and special offer information" refers to information about discounts or special treatment offered for specific products.

[0493] The system implementing this invention consists of a server, a user's terminal, and a communication application. The server implements a machine learning algorithm using Python and has an API using the Flask framework. MySQL is used as the database for aggregating, storing, and analyzing behavioral history data. The main processing performed by the server is to analyze the user's past behavioral history and identify individual behavioral patterns. This makes it possible to predict the user's next actions and the consumables they should purchase.

[0494] The user's device is assumed to be a smartphone, and an application developed using React Native will be installed. This application has the function of receiving notifications from the server in real time and providing information to the user through an intuitive interface. For communication, a general messaging software is used to ensure smooth communication with the user.

[0495] For example, if the server analyzes user behavior and determines that consumable item A is purchased in the third week of each month, a notification is sent to the device at the end of the second week stating, "We recommend checking your inventory and purchasing it this weekend if necessary." In this way, users can purchase necessary products at the optimal time. Furthermore, real-time promotional information is also provided, making purchasing behavior more efficient.

[0496] An example of a prompt given to a generative AI model is, "Design an algorithm that predicts a user's next purchase based on their past purchase history and provides notifications and recommended products at the appropriate time." Based on this prompt, an algorithm is generated that delivers highly accurate suggestions. This approach can provide personalized support to individual users and improve the efficiency of their lives.

[0497] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0498] Step 1:

[0499] The server receives activity history data from the user's device. This data includes the user's activities, time spent, and past purchase history. The received data is stored in a database. The input is the user's activity history data, and the output is a record of the stored data.

[0500] Step 2:

[0501] The server analyzes stored behavioral history data. Using a machine learning algorithm implemented in Python, it performs pattern recognition to identify each user's behavioral patterns. Based on the analysis results, the server predicts the consumables and tasks the user will likely need next. The input is behavioral history data, and the output is behavioral patterns and prediction results.

[0502] Step 3:

[0503] The server generates specific suggestions for the user based on predictions. These suggestions include information on necessary tasks, products to purchase, and links to relevant online stores and special offers. The input is the prediction result, and the output is the content of the suggestions.

[0504] Step 4:

[0505] The server sends the generated proposal to the terminal. The terminal immediately notifies the user via a communication application and displays the proposal content. The input is the proposal content, and the output is the notification provided to the user.

[0506] Step 5:

[0507] The user receives a notification and decides whether to take the suggested action. If they click a purchase link or buy a recommended product, the result and selection information are recorded as feedback on the device.

[0508] Step 6:

[0509] The device sends feedback information to the server. The server uses this information to update its machine learning model and improve the accuracy of the next prediction. The input is the user's feedback information, and the output is the updated machine learning model.

[0510] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0511] The system according to the present invention has the function of providing optimized task suggestions and purchase suggestions based on an individual's behavior and emotional state. By combining a server, terminal, and emotion engine, this system achieves efficient user behavior management and personalized experiences based on emotions.

[0512] The server first receives user behavior history data and emotional state data collected by the emotion engine from the terminal and stores them in a database. The emotional state data includes information such as whether the user is stressed or relaxed.

[0513] Next, the server analyzes behavioral history data and emotional state data using machine learning algorithms to identify the user's behavioral patterns and adjust suggestions in response to changes in their emotions. As a result, the system predicts the next tasks and purchases that should be made, tailored to the user's mental and emotional state. The system anticipates emotional changes before they occur and suggests appropriate actions to the user, thereby encouraging better behavior.

[0514] Based on these analysis results, the server generates a list of appropriate tasks and purchases for the user. These suggestions include recommendations on when a user should perform a task, given their emotional state, and how they should choose their purchases.

[0515] The generated suggestions are sent to the terminal via a communication application. The terminal displays the suggestions received from the server in a format that is easy for the user to understand. For example, if the user is tired, it may suggest purchasing items to create a relaxing environment, providing detailed advice based on emotions.

[0516] Users check notifications on their devices, perform suggested tasks, or purchase products based on emotionally related suggestions. During this process, the user's actions and responses to emotional changes are recorded on the device and sent to the server via the emotion engine. The server uses this feedback to further update its machine learning model, continuously improving the quality and accuracy of its suggestions.

[0517] For example, if a user frequently experiences anxiety, the server can recommend consuming relaxing music or entertainment and encourage the purchase of related products (such as aromatherapy products or relaxation chairs). In this way, the goal is to provide optimal suggestions tailored to the user's emotional state, supporting not only daily work efficiency but also mental well-being.

[0518] The following describes the processing flow.

[0519] Step 1:

[0520] The device collects user behavior history data and emotional state data obtained by the emotion engine. The emotional state data includes information about the user's emotions obtained through technologies such as facial recognition and voice tone analysis.

[0521] Step 2:

[0522] The device sends collected behavioral history data and emotional state data to the server. The server stores this data in a database and manages it as an individual data stream.

[0523] Step 3:

[0524] The server uses machine learning algorithms to process the stored data and analyze user behavior patterns and emotional changes. The analysis identifies the user's current emotional state and tendencies.

[0525] Step 4:

[0526] Based on the analysis results, the server generates recommendations and purchase suggestions that take into account the user's current emotional state. This may include, for example, suggesting items that promote rest and enhance feelings of comfort when the user is feeling tired.

[0527] Step 5:

[0528] The server sends the generated proposals to the terminal via a communication application. The terminal displays the received proposals in its user interface for immediate review.

[0529] Step 6:

[0530] Users can review the displayed suggestions, perform the suggested tasks as needed, or click the purchase link to buy products online.

[0531] Step 7:

[0532] User reactions and selected actions are recorded on the device and sent back to the server as feedback. This is used to gain a more detailed understanding of user behavioral patterns and emotional changes.

[0533] Step 8:

[0534] The server receives feedback and updates its learning model to improve the accuracy of its suggestions. This allows it to continuously learn so that future suggestions become even more appropriate.

[0535] (Example 2)

[0536] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0537] In individual activities, there is a need for more effective and personalized suggestions based on behavioral history and emotional state, but conventional systems have been unable to meet this need. Furthermore, when making purchase suggestions, there has been no system that can accurately predict changes in emotions.

[0538] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0539] In this invention, the server includes means for aggregating and storing an individual's behavioral history data and emotional state data; means for analyzing the stored behavioral history data and emotional state data to optimize suggestions based on the individual's specific behaviors and emotional states; and means for generating suggestions for necessary tasks and purchases using an AI model generated from the analyzed data. This makes it possible to provide efficient and personalized suggestions while taking into account the individual's emotional state, thereby improving the accuracy of actions and the purchase of related products.

[0540] "Behavioral history data" refers to recorded information about an individual's daily activities and actions.

[0541] "Emotional state data" refers to information about the emotions and moods an individual is experiencing at a specific point in time.

[0542] "Aggregation" refers to the process of consolidating and managing multiple data sets in one place.

[0543] "Storage" refers to the act of permanently recording data so that it can be retrieved as needed.

[0544] "Analysis" refers to the process of identifying trends and patterns in collected data and using that information to gain specific insights.

[0545] A "generative AI model" refers to an artificial intelligence model that creates new information and suggestions based on a large amount of data.

[0546] A "proposal" refers to information that outlines actions or options to consider in order to achieve a specific objective.

[0547] "Communication technology" refers to the technical means and protocols used to send and receive data and information.

[0548] "Feedback" refers to an individual's response or result to a suggestion made by a system.

[0549] This invention is implemented as a system that utilizes individual behavioral history data and emotional state data to provide personalized suggestions to users. Specifically, a server, terminal, and emotion engine work together to perform these functions.

[0550] The server aggregates individual behavioral history data and emotional state data and stores it in a database system (e.g., Apache Cassandra or MySQL) for centralized data management. It also uses open-source machine learning libraries such as Scikit-learn and TensorFlow as an analysis platform for executing machine learning algorithms. This analysis helps understand behavioral patterns and emotional changes, generating personalized suggestions.

[0551] The device plays the initial role of collecting user behavioral history data and emotional state data. This data is acquired in real time by mobile devices and sensors. This enables suggestions that reflect the user's emotional state. The device receives suggestions from the server and provides a user interface that displays them to the user in an easy-to-understand visual way. For example, if the user wants to relax, it can suggest music or relaxation products.

[0552] Users take action or make purchases based on suggestions displayed on their devices. During this process, the user's reactions and operation history are recorded on the device as feedback data. This feedback is sent to a server and used to optimize future suggestions.

[0553] For example, if a user frequently experiences anxiety, the server might suggest purchasing relaxing music or aromatherapy products. A possible prompt could be input to the generative AI model in the form of, "What emotional state is the user currently in? How can we make appropriate suggestions based on that?"

[0554] This system makes it possible to leverage data on users' emotions and behavior to continuously provide personalized suggestions for improving their lives.

[0555] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0556] Step 1:

[0557] The device collects user behavior history data and emotional state data. Inputs include sensor information from the mobile device (GPS location data, app usage history, heart rate, etc.) and a user interface for evaluating emotional state. Based on this data, the device records the user's daily activities and their emotional state at the time. Outputs include behavior history data and emotional state data in a format that can be stored in a database.

[0558] Step 2:

[0559] The terminal sends the collected data to the server. The inputs used are the behavioral history data and emotional state data obtained in step 1. The output is encrypted data, which is sent to the server via a secure communication protocol. The server receives this data and stores it in its internal database.

[0560] Step 3:

[0561] The server analyzes the received data. It uses behavioral history data and emotional state data obtained from a database as input. To analyze the data, it uses a Python machine learning library to identify behavioral patterns and changes in emotional state. The output provides personalized behavioral patterns and emotional fluctuation information for the user.

[0562] Step 4:

[0563] The server uses a generative AI model to create suggestions. It uses the analysis results and prompt text obtained from step 3 (e.g., "What emotional state is the user currently in? How can we make appropriate suggestions based on that?") as input. Based on this data, it generates suggestions using natural language generation technology. The output is text suggestions regarding the next task or purchase.

[0564] Step 5:

[0565] The server sends the generated proposal to the terminal. The proposal text generated in step 4 is used as input. As output, the proposal is sent to the terminal using communication technology and displayed on the terminal. The user can visually confirm this.

[0566] Step 6:

[0567] The user reviews the suggestions on the device and takes the necessary actions. For example, they might access an online store to purchase a recommended relaxation item. The results of the user's actions, and the resulting emotional changes, are recorded again on the device as feedback data.

[0568] Step 7:

[0569] The device collects user feedback and sends it to the server. The input uses the behavioral results and emotional changes recorded in step 6. The output is encrypted feedback data, which is then sent to the server. The server uses this data to update its machine learning model and improve the accuracy of future suggestions.

[0570] (Application Example 2)

[0571] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0572] Conventional behavior management systems offer suggestions based on user behavior history, but they are insufficient in suggesting appropriate content that takes emotional states into account in real time. In particular, recommending content tailored to emotions is crucial for supporting users' mental health, but no system effectively addresses this. Therefore, the challenge is to suggest optimal content in real time based on the user's emotional state, thereby improving mental health and enabling efficient behavior management.

[0573] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0574] In this invention, the server includes means for collecting and storing an individual's behavioral history information, means for storing and analyzing emotional state information obtained from an emotion recognition device, and means for analyzing the stored behavioral history information and emotional state information to predict the individual's specific actions. This makes it possible to suggest optimal tasks, purchases, and content appropriate to the user's emotional state in real time.

[0575] "Personal behavioral history information" refers to information about specific actions and choices that a user has made in the past.

[0576] An "emotion recognition device" is a technological device that analyzes images, sounds, etc., to detect the user's emotional state.

[0577] "Emotional state information" refers to information that indicates the user's psychological state (e.g., stress, relaxation, happiness, etc.).

[0578] "Communication technology" refers to the technical means used to transmit data to users in remote locations.

[0579] "Proposal accuracy" is an indicator that shows how well the generated proposals match the user's current state and needs.

[0580] A "machine learning model" is an algorithm that learns patterns from past data and makes predictions based on new data.

[0581] "Feedback information" refers to the results of collecting user responses and opinions to suggestions.

[0582] A "link for online purchase" is a URL provided to directly purchase a product when buying it via the internet.

[0583] This system enables content delivery using user behavior history and emotional state information by having the user's device, including their smartphone, communicate with a server. The server uses data collected from the smartphone and other sensors (e.g., camera, microphone, heart rate sensor). The emotion recognition device utilizes emotion analysis libraries (e.g., Google Cloud Vision API, AWS Rekognition) to analyze image and audio data and generate information about the user's emotional state.

[0584] The server analyzes collected behavioral history and emotional state information using machine learning algorithms. Based on the analysis results, it predicts and generates content suitable for the user's behavioral patterns and current emotions. It also uses communication technology to send the generated suggestions to the user's device, making it easy for the user to receive the suggested content.

[0585] For example, if the system detects that a user is experiencing stress, it will suggest music or video content that is expected to have a relaxing effect on the device. This allows the user to improve their mental health and maintain a state in which they can perform their daily tasks efficiently.

[0586] An example of a prompt to input into the generating AI model is: "Analyze the user's facial expression detection data and recommend appropriate relaxation content when the emotional state is determined to be 'stressed'."

[0587] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0588] Step 1:

[0589] The device collects user activity history information from sensors (e.g., smartphone accelerometer and GPS) and application logs. This information is stored in a database as the user's movement path and application usage history. The input is raw sensor data and log data, and the output is activity history information.

[0590] Step 2:

[0591] The device detects the user's emotional state using its built-in camera and microphone. The acquired image and audio data are analyzed using an emotion analysis library (e.g., AWS Rekognition) to identify the user's emotional state (e.g., happy, stressed, relaxed). The input is image and audio data, and the output is emotional state information.

[0592] Step 3:

[0593] The server receives the collected behavioral history and emotional state information and stores it in a database. This process uses the received behavioral history and emotional state information as input and transforms it into an appropriate data structure for storage. The output is an organized database entry.

[0594] Step 4:

[0595] The server uses machine learning algorithms to analyze stored behavioral history and emotional state information. This analysis makes estimations based on the user's behavioral patterns and emotions, and identifies the content that should be suggested next. The input is stored data, and the output is a list of suggested content.

[0596] Step 5:

[0597] The server notifies the terminal of the generated content suggestions. Here, the content list is sent via a communication protocol (e.g., HTTP / HTTPS) and displayed in a format visible to the user. The input is the generated content list, and the output is the notification displayed on the user's terminal screen.

[0598] Step 6:

[0599] The user reviews the suggestions notified from their device and then views or purchases the selected content. These user actions are recorded by the device as feedback information, which is then sent back to the server. The input is the suggested content, and the output is the feedback information.

[0600] Step 7:

[0601] The server updates the machine learning model based on the feedback information to improve the proposal accuracy. In this step, the latest feedback information is used as input, and the model's learning parameters are updated. The output is the new machine learning model with improved proposal accuracy.

[0602] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0603] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0604] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0605] [Fourth Embodiment]

[0606] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0607] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0608] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0609] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0610] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0611] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0612] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0613] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0614] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0615] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0616] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0617] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0618] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0619] The system according to the present invention efficiently manages an individual's daily work and lifestyle activities, providing necessary action suggestions and notifications to prevent forgotten purchases. This system exchanges data between the server, terminal, and user, aiming for operation optimized for the individual.

[0620] The server first collects individual behavioral history data from the terminal. This behavioral history data includes the content of the tasks performed by the individual, the time they were performed, and the history of products purchased in the past. The server analyzes this data to identify each individual's behavioral patterns. In doing so, it uses machine learning algorithms to perform more sophisticated pattern recognition and predict the user's next actions and necessary purchases.

[0621] Based on the predictions obtained through analysis, the server generates personalized suggestions for the user's next actions and purchases. These suggestions include information such as what tasks the user should perform next and when would be the best time to perform them. For purchases, it also suggests the optimal time to buy and includes links to online stores where the items can be purchased.

[0622] The generated proposals are sent to the terminal via a communication application. The terminal then notifies the user of the proposals received from the server in an easy-to-understand format. This notification can be sent via a messaging application such as LINE, allowing the user to immediately review the notification and decide on an action based on the proposals.

[0623] Users check notifications received on their devices and perform suggested tasks or click purchase links to buy products. During this process, the user's actions and selected responses are recorded on the device, and this feedback information is sent to the server. The server uses this feedback to update its machine learning model and improve the accuracy of its suggestions.

[0624] For example, if the server analyzes a user's monthly spending patterns and discovers a tendency to purchase certain consumables at the end of the month, the server will suggest repurchasing those consumables near the end of the month and provide the user with a link to an online store where those consumables can be purchased. In this way, the present invention enables improvements in work efficiency and quality of life by providing personalized suggestions to individual users.

[0625] The following describes the processing flow.

[0626] Step 1:

[0627] The server receives user activity history data from the terminal and stores it in a database. This data includes information such as past work activities, purchase history, and products used.

[0628] Step 2:

[0629] The server analyzes the stored data using machine learning algorithms to identify user behavior patterns. This allows it to extract actions that are repeated on specific days of the week or at specific times, and predict future actions.

[0630] Step 3:

[0631] Based on the analysis results, the server generates a list of recommended tasks and purchases for the user. The generated list clearly specifies the next tasks to be performed and the items to be purchased.

[0632] Step 4:

[0633] The server creates a notification message containing details of the proposed tasks and purchases, and sends that message to the user's device.

[0634] Step 5:

[0635] The device displays notifications received from the server on the user interface. Users can check these notifications within applications such as LINE.

[0636] Step 6:

[0637] Users review the suggestion notifications received on their devices and, if necessary, take action or purchase products using the provided links.

[0638] Step 7:

[0639] Feedback regarding user actions and choices is recorded on the device. The device then sends this feedback information to the server.

[0640] Step 8:

[0641] The server receives feedback from users and updates its learning model, allowing for continuous improvement in the accuracy of its suggestions.

[0642] (Example 1)

[0643] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0644] Providing appropriate behavioral suggestions based on individuals' diverse daily activities and purchasing behaviors is challenging. Furthermore, effectively utilizing individual feedback to improve the accuracy of these suggestions is a major challenge. To meet this need for personalized optimization, there is a demand for technologies that can provide efficient and highly accurate suggestions.

[0645] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0646] In this invention, the server includes means for collecting and storing an individual's behavioral history information, means for analyzing the stored behavioral history information to identify the individual's behavioral patterns, and means for generating suggestions for necessary tasks and purchases based on predicted behavior using a machine learning algorithm. This enables highly accurate suggestion generation tailored to the individual's behavioral patterns and efficient action decisions based on those suggestions.

[0647] "Personal behavioral history information" refers to information about an individual's past activities and actions, including their frequency and content. This information is used to identify specific behavioral patterns of an individual.

[0648] A "machine learning algorithm" refers to a computational method that allows computers to learn patterns and rules from large amounts of data to make future predictions and decisions. This algorithm can accumulate experience from data and apply that knowledge to new data.

[0649] "Behavioral patterns" refer to an individual's specific habitual actions, tendencies, and recurring behaviors. By identifying these patterns, it becomes possible to predict future actions and necessary suggestions in detail.

[0650] "Proposal accuracy" refers to the degree to which the proposed solution is appropriate and useful to the individual. This accuracy is an indicator of how well the proposal matches the individual's actual behavior and needs.

[0651] "Communication methods" refer to the technologies, devices, and protocols used to send and receive information. These methods make it possible to deliver generated proposals to individuals and collect their feedback.

[0652] "Responsive design" refers to a web design technique that automatically optimizes the displayed content according to the user's device and screen size. This allows for consistent usability across various devices.

[0653] In order to implement this invention, it is necessary for stakeholders and devices such as servers, terminals, and users to work in coordination. The server first collects personal activity history information from the terminal. The terminal records data about the user's daily activities and purchasing behavior and sends it to the server. The server uses a database to receive and store this information. Specifically, database management systems such as MySQL and PostgreSQL can be used.

[0654] The server can use Python, the R language, and machine learning libraries such as TensorFlow and SciKit-Learn for data analysis. These help identify user behavior patterns from the dataset and build predictive models using machine learning algorithms. The server then generates appropriate action and purchase suggestions for individual users.

[0655] The generated proposals are sent from the server to the terminal, which then notifies the user via a messaging app. This notification technology can utilize general-purpose protocols such as WebSocket and HTTP / 2. The terminal employs responsive design to provide a user-friendly and easy-to-use interface.

[0656] Users review suggestions through their devices and decide on actions based on their content. Feedback on the user's choices and actions is recorded on the device and sent back to the server. The server processes this feedback as new data and uses it to update the machine learning model and improve the accuracy of the suggestions.

[0657] For example, if the server analyzes a user's spending patterns and discovers that they purchase a specific product every month, it will suggest the next purchase date and provide a link to purchase that product. This enables timely and appropriate suggestions tailored to the user. An example of a prompt to input into the generating AI model is, "Create a repurchase suggestion for a specific consumable based on the user's purchase history over the past three months."

[0658] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0659] Step 1:

[0660] The server collects personal activity history information from the terminal. The terminal periodically records user activity data and purchasing behavior and sends it to the server. This input data includes date and time, location, and information about purchased items. After receiving this data, the server performs a format conversion for storage in the database. For example, the data might be converted from JSON format to SQL table format. As output, a structured database entry is generated.

[0661] Step 2:

[0662] The server analyzes stored behavioral history information to identify user behavior patterns. In this step, database information is retrieved as input for analysis, and machine learning algorithms are used for the analysis. Specifically, the user's behavioral history is treated as time-series data, and pattern recognition is performed. Algorithms such as TensorFlow and SciKit-Learn are used to cluster behavioral patterns and extract features, and predictions of the next action are generated as output.

[0663] Step 3:

[0664] The server generates individual suggestions based on the analysis results. The prediction results generated in step 2 are used as input. Based on this information, it creates suggestions for the user's next actions and purchases. For example, if it suggests purchasing a specific product on a specific day, this suggestion will include a purchase link and the optimal time to do so. As output, it generates a data object of the suggested content.

[0665] Step 4:

[0666] The server sends the generated proposal to the terminal. The proposal data created in step 3 is used as input. This data is transferred to the terminal using a communication method. Specifically, data transfer is performed using a RESTful API or WebSocket. The terminal returns a confirmation of receipt as output.

[0667] Step 5:

[0668] The device notifies the user of suggestions received from the server. The input is suggestion information received from the server. This suggestion is sent to the user via a messaging app such as LINE. Specifically, a notification message is created and displayed according to the device's screen size using responsive design technology. The output is a visual notification to the user.

[0669] Step 6:

[0670] The user checks the notification and takes action based on the suggestion. The input is a suggestion notification from the device. If the user selects an action, the details are recorded on the device. For example, this could be an action such as clicking a purchase link to buy a product. As output, record data of the action is generated.

[0671] Step 7:

[0672] The terminal sends user behavior data to the server. The input is the behavior data recorded in step 6. By sending this to the server, it is saved again in the database and used as information for further analysis. Specifically, the data is encrypted and sent securely. The output is the behavior data saved in the database.

[0673] Step 8:

[0674] The server updates the machine learning model based on the collected feedback information. The input is the behavioral data received in step 7. The server uses this data to retrain the model and improve the accuracy of suggestions for subsequent steps. Specifically, it performs batch learning and fine-tunes the model with new data. The output is the updated machine learning model.

[0675] (Application Example 1)

[0676] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0677] Individuals often forget to purchase necessary items or miss the optimal timing for purchases in their daily work and lives. Furthermore, users struggle to make informed decisions about what products to buy and what actions to take, making it difficult to maintain an efficient lifestyle. This highlights the growing need for effective suggestion systems to improve quality of life.

[0678] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0679] In this invention, the server includes means for aggregating and storing individual behavioral history data, means for analyzing the stored behavioral history data and predicting specific actions of the individual, and means for providing consumables and recommended products at specific times based on the individual's past purchase history. This enables users to purchase necessary products at the optimal time, streamlining daily management and improving their quality of life.

[0680] "Activity history data" refers to information such as the content of activities and tasks performed by an individual on a daily basis, and the time spent performing them.

[0681] "Analysis" refers to the process of using stored data to identify regularities and patterns and predict individual behavior.

[0682] "Prediction" refers to the act of inferring the next tasks or purchases a user will need based on their past behavioral history.

[0683] "Suggestions" refer to recommending the best next course of action or items to purchase to a user based on their predicted behavior.

[0684] A "communication application" refers to software used to transmit information and notify users of proposed content.

[0685] "Individual responses" refer to users' responses and actions to suggestions, and recording them improves the accuracy of the data.

[0686] "Consumable goods" refer to items used in daily life that need to be repurchased over time.

[0687] A "recommended product" refers to an item that is suggested as the ideal next purchase based on the user's purchase history and behavioral patterns.

[0688] "Promotional and special offer information" refers to information about discounts or special treatment offered for specific products.

[0689] The system implementing this invention consists of a server, a user's terminal, and a communication application. The server implements a machine learning algorithm using Python and has an API using the Flask framework. MySQL is used as the database for aggregating, storing, and analyzing behavioral history data. The main processing performed by the server is to analyze the user's past behavioral history and identify individual behavioral patterns. This makes it possible to predict the user's next actions and the consumables they should purchase.

[0690] The user's device is assumed to be a smartphone, and an application developed using React Native will be installed. This application has the function of receiving notifications from the server in real time and providing information to the user through an intuitive interface. For communication, a general messaging software is used to ensure smooth communication with the user.

[0691] For example, if the server analyzes user behavior and determines that consumable item A is purchased in the third week of each month, a notification is sent to the device at the end of the second week stating, "We recommend checking your inventory and purchasing it this weekend if necessary." In this way, users can purchase necessary products at the optimal time. Furthermore, real-time promotional information is also provided, making purchasing behavior more efficient.

[0692] An example of a prompt given to a generative AI model is, "Design an algorithm that predicts a user's next purchase based on their past purchase history and provides notifications and recommended products at the appropriate time." Based on this prompt, an algorithm is generated that delivers highly accurate suggestions. This approach can provide personalized support to individual users and improve the efficiency of their lives.

[0693] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0694] Step 1:

[0695] The server receives activity history data from the user's device. This data includes the user's activities, time spent, and past purchase history. The received data is stored in a database. The input is the user's activity history data, and the output is a record of the stored data.

[0696] Step 2:

[0697] The server analyzes stored behavioral history data. Using a machine learning algorithm implemented in Python, it performs pattern recognition to identify each user's behavioral patterns. Based on the analysis results, the server predicts the consumables and tasks the user will likely need next. The input is behavioral history data, and the output is behavioral patterns and prediction results.

[0698] Step 3:

[0699] The server generates specific suggestions for the user based on predictions. These suggestions include information on necessary tasks, products to purchase, and links to relevant online stores and special offers. The input is the prediction result, and the output is the content of the suggestions.

[0700] Step 4:

[0701] The server sends the generated proposal to the terminal. The terminal immediately notifies the user via a communication application and displays the proposal content. The input is the proposal content, and the output is the notification provided to the user.

[0702] Step 5:

[0703] The user receives a notification and decides whether to take the suggested action. If they click a purchase link or buy a recommended product, the result and selection information are recorded as feedback on the device.

[0704] Step 6:

[0705] The device sends feedback information to the server. The server uses this information to update its machine learning model and improve the accuracy of the next prediction. The input is the user's feedback information, and the output is the updated machine learning model.

[0706] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0707] The system according to the present invention has the function of providing optimized task suggestions and purchase suggestions based on an individual's behavior and emotional state. By combining a server, terminal, and emotion engine, this system achieves efficient user behavior management and personalized experiences based on emotions.

[0708] The server first receives user behavior history data and emotional state data collected by the emotion engine from the terminal and stores them in a database. The emotional state data includes information such as whether the user is stressed or relaxed.

[0709] Next, the server analyzes behavioral history data and emotional state data using machine learning algorithms to identify the user's behavioral patterns and adjust suggestions in response to changes in their emotions. As a result, the system predicts the next tasks and purchases that should be made, tailored to the user's mental and emotional state. The system anticipates emotional changes before they occur and suggests appropriate actions to the user, thereby encouraging better behavior.

[0710] Based on these analysis results, the server generates a list of appropriate tasks and purchases for the user. These suggestions include recommendations on when a user should perform a task, given their emotional state, and how they should choose their purchases.

[0711] The generated suggestions are sent to the terminal via a communication application. The terminal displays the suggestions received from the server in a format that is easy for the user to understand. For example, if the user is tired, it may suggest purchasing items to create a relaxing environment, providing detailed advice based on emotions.

[0712] Users check notifications on their devices, perform suggested tasks, or purchase products based on emotionally related suggestions. During this process, the user's actions and responses to emotional changes are recorded on the device and sent to the server via the emotion engine. The server uses this feedback to further update its machine learning model, continuously improving the quality and accuracy of its suggestions.

[0713] For example, if a user frequently experiences anxiety, the server can recommend consuming relaxing music or entertainment and encourage the purchase of related products (such as aromatherapy products or relaxation chairs). In this way, the goal is to provide optimal suggestions tailored to the user's emotional state, supporting not only daily work efficiency but also mental well-being.

[0714] The following describes the processing flow.

[0715] Step 1:

[0716] The device collects user behavior history data and emotional state data obtained by the emotion engine. The emotional state data includes information about the user's emotions obtained through technologies such as facial recognition and voice tone analysis.

[0717] Step 2:

[0718] The device sends collected behavioral history data and emotional state data to the server. The server stores this data in a database and manages it as an individual data stream.

[0719] Step 3:

[0720] The server uses machine learning algorithms to process the stored data and analyze user behavior patterns and emotional changes. The analysis identifies the user's current emotional state and tendencies.

[0721] Step 4:

[0722] Based on the analysis results, the server generates recommendations and purchase suggestions that take into account the user's current emotional state. This may include, for example, suggesting items that promote rest and enhance feelings of comfort when the user is feeling tired.

[0723] Step 5:

[0724] The server sends the generated proposals to the terminal via a communication application. The terminal displays the received proposals in its user interface for immediate review.

[0725] Step 6:

[0726] Users can review the displayed suggestions, perform the suggested tasks as needed, or click the purchase link to buy products online.

[0727] Step 7:

[0728] User reactions and selected actions are recorded on the device and sent back to the server as feedback. This is used to gain a more detailed understanding of user behavioral patterns and emotional changes.

[0729] Step 8:

[0730] The server receives feedback and updates its learning model to improve the accuracy of its suggestions. This allows it to continuously learn so that future suggestions become even more appropriate.

[0731] (Example 2)

[0732] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0733] In individual activities, there is a need for more effective and personalized suggestions based on behavioral history and emotional state, but conventional systems have been unable to meet this need. Furthermore, when making purchase suggestions, there has been no system that can accurately predict changes in emotions.

[0734] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0735] In this invention, the server includes means for aggregating and storing an individual's behavioral history data and emotional state data; means for analyzing the stored behavioral history data and emotional state data to optimize suggestions based on the individual's specific behaviors and emotional states; and means for generating suggestions for necessary tasks and purchases using an AI model generated from the analyzed data. This makes it possible to provide efficient and personalized suggestions while taking into account the individual's emotional state, thereby improving the accuracy of actions and the purchase of related products.

[0736] "Behavioral history data" refers to recorded information about an individual's daily activities and actions.

[0737] "Emotional state data" refers to information about the emotions and moods an individual is experiencing at a specific point in time.

[0738] "Aggregation" refers to the process of consolidating and managing multiple data sets in one place.

[0739] "Storage" refers to the act of permanently recording data so that it can be retrieved as needed.

[0740] "Analysis" refers to the process of identifying trends and patterns in collected data and using that information to gain specific insights.

[0741] A "generative AI model" refers to an artificial intelligence model that creates new information and suggestions based on a large amount of data.

[0742] A "proposal" refers to information that outlines actions or options to consider in order to achieve a specific objective.

[0743] "Communication technology" refers to the technical means and protocols used to send and receive data and information.

[0744] "Feedback" refers to an individual's response or result to a suggestion made by a system.

[0745] This invention is implemented as a system that utilizes individual behavioral history data and emotional state data to provide personalized suggestions to users. Specifically, a server, terminal, and emotion engine work together to perform these functions.

[0746] The server aggregates individual behavioral history data and emotional state data and stores it in a database system (e.g., Apache Cassandra or MySQL) for centralized data management. It also uses open-source machine learning libraries such as Scikit-learn and TensorFlow as an analysis platform for executing machine learning algorithms. This analysis helps understand behavioral patterns and emotional changes, generating personalized suggestions.

[0747] The device plays the initial role of collecting user behavioral history data and emotional state data. This data is acquired in real time by mobile devices and sensors. This enables suggestions that reflect the user's emotional state. The device receives suggestions from the server and provides a user interface that displays them to the user in an easy-to-understand visual way. For example, if the user wants to relax, it can suggest music or relaxation products.

[0748] Users take action or make purchases based on suggestions displayed on their devices. During this process, the user's reactions and operation history are recorded on the device as feedback data. This feedback is sent to a server and used to optimize future suggestions.

[0749] For example, if a user frequently experiences anxiety, the server might suggest purchasing relaxing music or aromatherapy products. A possible prompt could be input to the generative AI model in the form of, "What emotional state is the user currently in? How can we make appropriate suggestions based on that?"

[0750] This system makes it possible to leverage data on users' emotions and behavior to continuously provide personalized suggestions for improving their lives.

[0751] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0752] Step 1:

[0753] The device collects user behavior history data and emotional state data. Inputs include sensor information from the mobile device (GPS location data, app usage history, heart rate, etc.) and a user interface for evaluating emotional state. Based on this data, the device records the user's daily activities and their emotional state at the time. Outputs include behavior history data and emotional state data in a format that can be stored in a database.

[0754] Step 2:

[0755] The terminal sends the collected data to the server. The inputs used are the behavioral history data and emotional state data obtained in step 1. The output is encrypted data, which is sent to the server via a secure communication protocol. The server receives this data and stores it in its internal database.

[0756] Step 3:

[0757] The server analyzes the received data. It uses behavioral history data and emotional state data obtained from a database as input. To analyze the data, it uses a Python machine learning library to identify behavioral patterns and changes in emotional state. The output provides personalized behavioral patterns and emotional fluctuation information for the user.

[0758] Step 4:

[0759] The server uses a generative AI model to create suggestions. It uses the analysis results and prompt text obtained from step 3 (e.g., "What emotional state is the user currently in? How can we make appropriate suggestions based on that?") as input. Based on this data, it generates suggestions using natural language generation technology. The output is text suggestions regarding the next task or purchase.

[0760] Step 5:

[0761] The server sends the generated proposal to the terminal. The proposal text generated in step 4 is used as input. As output, the proposal is sent to the terminal using communication technology and displayed on the terminal. The user can visually confirm this.

[0762] Step 6:

[0763] The user reviews the suggestions on the device and takes the necessary actions. For example, they might access an online store to purchase a recommended relaxation item. The results of the user's actions, and the resulting emotional changes, are recorded again on the device as feedback data.

[0764] Step 7:

[0765] The device collects user feedback and sends it to the server. The input uses the behavioral results and emotional changes recorded in step 6. The output is encrypted feedback data, which is then sent to the server. The server uses this data to update its machine learning model and improve the accuracy of future suggestions.

[0766] (Application Example 2)

[0767] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0768] Conventional behavior management systems offer suggestions based on user behavior history, but they are insufficient in suggesting appropriate content that takes emotional states into account in real time. In particular, recommending content tailored to emotions is crucial for supporting users' mental health, but no system effectively addresses this. Therefore, the challenge is to suggest optimal content in real time based on the user's emotional state, thereby improving mental health and enabling efficient behavior management.

[0769] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0770] In this invention, the server includes means for collecting and storing an individual's behavioral history information, means for storing and analyzing emotional state information obtained from an emotion recognition device, and means for analyzing the stored behavioral history information and emotional state information to predict the individual's specific actions. This makes it possible to suggest optimal tasks, purchases, and content appropriate to the user's emotional state in real time.

[0771] "Personal behavioral history information" refers to information about specific actions and choices that a user has made in the past.

[0772] An "emotion recognition device" is a technological device that analyzes images, sounds, etc., to detect the user's emotional state.

[0773] "Emotional state information" refers to information that indicates the user's psychological state (e.g., stress, relaxation, happiness, etc.).

[0774] "Communication technology" refers to the technical means used to transmit data to users in remote locations.

[0775] "Proposal accuracy" is an indicator that shows how well the generated proposals match the user's current state and needs.

[0776] A "machine learning model" is an algorithm that learns patterns from past data and makes predictions based on new data.

[0777] "Feedback information" refers to the results of collecting user responses and opinions to suggestions.

[0778] A "link for online purchase" is a URL provided to directly purchase a product when buying it via the internet.

[0779] This system enables content delivery using user behavior history and emotional state information by having the user's device, including their smartphone, communicate with a server. The server uses data collected from the smartphone and other sensors (e.g., camera, microphone, heart rate sensor). The emotion recognition device utilizes emotion analysis libraries (e.g., Google Cloud Vision API, AWS Rekognition) to analyze image and audio data and generate information about the user's emotional state.

[0780] The server analyzes collected behavioral history and emotional state information using machine learning algorithms. Based on the analysis results, it predicts and generates content suitable for the user's behavioral patterns and current emotions. It also uses communication technology to send the generated suggestions to the user's device, making it easy for the user to receive the suggested content.

[0781] For example, if the system detects that a user is experiencing stress, it will suggest music or video content that is expected to have a relaxing effect on the device. This allows the user to improve their mental health and maintain a state in which they can perform their daily tasks efficiently.

[0782] An example of a prompt to input into the generating AI model is: "Analyze the user's facial expression detection data and recommend appropriate relaxation content when the emotional state is determined to be 'stressed'."

[0783] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0784] Step 1:

[0785] The device collects user activity history information from sensors (e.g., smartphone accelerometer and GPS) and application logs. This information is stored in a database as the user's movement path and application usage history. The input is raw sensor data and log data, and the output is activity history information.

[0786] Step 2:

[0787] The device detects the user's emotional state using its built-in camera and microphone. The acquired image and audio data are analyzed using an emotion analysis library (e.g., AWS Rekognition) to identify the user's emotional state (e.g., happy, stressed, relaxed). The input is image and audio data, and the output is emotional state information.

[0788] Step 3:

[0789] The server receives the collected behavioral history and emotional state information and stores it in a database. This process uses the received behavioral history and emotional state information as input and transforms it into an appropriate data structure for storage. The output is an organized database entry.

[0790] Step 4:

[0791] The server uses machine learning algorithms to analyze stored behavioral history and emotional state information. This analysis makes estimations based on the user's behavioral patterns and emotions, and identifies the content that should be suggested next. The input is stored data, and the output is a list of suggested content.

[0792] Step 5:

[0793] The server notifies the terminal of the generated content suggestions. Here, the content list is sent via a communication protocol (e.g., HTTP / HTTPS) and displayed in a format visible to the user. The input is the generated content list, and the output is the notification displayed on the user's terminal screen.

[0794] Step 6:

[0795] The user reviews the suggestions notified from their device and then views or purchases the selected content. These user actions are recorded by the device as feedback information, which is then sent back to the server. The input is the suggested content, and the output is the feedback information.

[0796] Step 7:

[0797] The server updates the machine learning model based on the feedback information to improve the proposal accuracy. In this step, the latest feedback information is used as input, and the model's learning parameters are updated. The output is the new machine learning model with improved proposal accuracy.

[0798] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0799] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0800] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0801] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0802] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0803] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0804] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0805] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0806] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0807] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0808] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0809] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0810] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0811] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0812] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0813] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0814] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0815] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0816] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0817] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0818] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0819] The following is further disclosed regarding the embodiments described above.

[0820] (Claim 1)

[0821] A means of aggregating and storing individual behavioral history data,

[0822] A means of analyzing stored behavioral history data to predict specific behaviors of individuals,

[0823] A means of generating suggestions for the next necessary tasks or purchases based on predicted actions,

[0824] A means of notifying individuals of the generated proposals through a communication application,

[0825] A means of collecting individual responses to the proposed content and using it as data to improve prediction accuracy,

[0826] A system that includes this.

[0827] (Claim 2)

[0828] The system according to claim 1, which provides an environment in which an individual can make an immediate purchase by providing an online purchase link for the proposed purchase.

[0829] (Claim 3)

[0830] The system according to claim 1, which updates a machine learning model based on individual feedback information and improves the accuracy of its suggestions.

[0831] "Example 1"

[0832] (Claim 1)

[0833] Means for collecting and storing personal activity history information,

[0834] A means of analyzing stored behavioral history information to identify an individual's behavioral patterns,

[0835] A means of generating suggestions for necessary tasks and purchases based on predicted behavior using machine learning algorithms,

[0836] A means of notifying individuals of the generated proposals via communication means,

[0837] A means of collecting individual responses to proposals and using them as information to improve the accuracy of the proposals,

[0838] A method for a device to display notifications using responsive design,

[0839] Information processing device including

[0840] (Claim 2)

[0841] The information processing device according to claim 1, which provides a purchase link for the proposed purchase and provides an environment in which individuals can quickly make purchases.

[0842] (Claim 3)

[0843] The information processing device according to claim 1, which updates a machine learning algorithm based on individual response information and improves the accuracy of the proposal.

[0844] "Application Example 1"

[0845] (Claim 1)

[0846] A means of aggregating and storing individual behavioral history data,

[0847] A means of analyzing stored behavioral history data to predict specific behaviors of individuals,

[0848] A means of generating suggestions for the next necessary tasks or purchases based on predicted actions,

[0849] A means of notifying individuals of the generated proposals through a communication application,

[0850] A means of collecting individual responses to the proposed content and using it as data to improve prediction accuracy,

[0851] A means of providing consumables and recommended products at specific times based on an individual's past purchase history,

[0852] A means of providing information including promotions and benefits tailored to recommended products,

[0853] A system that includes this.

[0854] (Claim 2)

[0855] The system according to claim 1, which provides an environment in which an individual can make an immediate purchase by providing an online purchase link for the proposed purchase.

[0856] (Claim 3)

[0857] The system according to claim 1, which updates a machine learning model based on individual feedback information and improves the accuracy of its suggestions.

[0858] "Example 2 of combining an emotion engine"

[0859] (Claim 1)

[0860] A means of aggregating and storing individual behavioral history data and emotional state data,

[0861] A means for analyzing stored behavioral history data and emotional state data to optimize suggestions based on an individual's specific behaviors and emotional states,

[0862] A means of generating suggestions for the next necessary tasks and purchases using an AI model generated from the analyzed data,

[0863] A means of notifying individuals of generated suggestions via communication technology and displaying suggestions based on emotions,

[0864] A means of collecting individual reactions to the proposed content and the accompanying changes in emotions, and using this data to improve prediction accuracy,

[0865] A system that includes this.

[0866] (Claim 2)

[0867] The system according to claim 1, which provides access information so that an individual can immediately obtain the proposed purchase.

[0868] (Claim 3)

[0869] The system according to claim 1, which updates a machine learning model based on individual feedback information and emotional state to improve the accuracy of suggestions.

[0870] "Application example 2 when combining with an emotional engine"

[0871] (Claim 1)

[0872] Means for collecting and storing personal activity history information,

[0873] A means for storing and analyzing emotional state information acquired from an emotion recognition device,

[0874] A means for analyzing stored behavioral history information and emotional state information to predict specific behaviors of individuals,

[0875] A means of suggesting necessary tasks, purchases, and emotionally appropriate content based on predicted behavior and emotional state,

[0876] A means of notifying individuals of the generated proposals through communication technology,

[0877] A means of collecting individual responses to the proposed content and using that information to improve prediction accuracy,

[0878] A system that includes this.

[0879] (Claim 2)

[0880] The system according to claim 1, which provides a link to connect to for online purchase of proposed items, and provides an environment in which individuals can make purchases immediately.

[0881] (Claim 3)

[0882] The system according to claim 1, which updates a machine learning model based on individual feedback information and emotional state information to improve the accuracy of suggestions. [Explanation of Symbols]

[0883] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of aggregating and storing individual behavioral history data, A means of analyzing stored behavioral history data to predict specific behaviors of individuals, A means of generating suggestions for the next necessary tasks or purchases based on predicted actions, A means of notifying individuals of the generated proposals through a communication application, A means of collecting individual responses to the proposed content and using it as data to improve prediction accuracy, A system that includes this.

2. The system according to claim 1, which provides an environment in which an individual can make an immediate purchase by providing an online purchase link for the proposed purchase.

3. The system according to claim 1, which updates a machine learning model based on individual feedback information and improves the accuracy of its suggestions.

Images