System

Abstract

To provide a system.SOLUTION: The system includes: means for learning a route work at a predetermined timing of a user; means for recognizing the emotion of the user; and means for proposing an action to the user on the basis of the learned routine work and emotion information showing the emotion of the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] In daily life, it is not easy to manage time and seasonal routines. In particular, thinking about daily choices such as what to do, what to eat, and what to wear takes time and energy. [Means for solving the problem]

[0005] We provide a system that learns the user's routine work by time and season, and suggests actions based on that. The system makes suggestions based on the learned patterns in response to questions such as what the user should do, what they should eat, and what they should wear. This enables the user to efficiently manage their daily life choices. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16]FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The system of the present invention includes a means for learning the user's routine work by time and season, a means for suggesting actions to the user based on the learned routine work, and a means for communicating the suggested content to the user. Specifically, the system is equipped with an AI engine that learns the user's daily life patterns and habits, and makes action suggestions to the user based on the learning results. The suggested content includes options such as "menu," "clothes to wear today," and "procedures that need to be done at this time of year." These suggestions are made based on the learned patterns and communicated to the user when the user asks questions such as "What should I do today?", "What should I eat today?", and "What should I wear today?".

[0029] "Example 2"

[0030] The system of the present invention includes a means for learning the user's routine work by time and season, a means for suggesting actions to the user based on the learned routine work, and a means for communicating the suggested content to the user. Specifically, the system is equipped with an AI engine that learns the user's daily life patterns and habits, and makes action suggestions to the user based on the learning results. The suggested content includes options such as "menu," "clothes to wear today," and "procedures that need to be done at this time of year." These suggestions are made based on the learned patterns and communicated to the user when the user asks questions such as "What should I do today?", "What should I eat today?", and "What should I wear today?".

[0031] The process flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: Learn the user's daily patterns and habits The AI ​​engine observes the user's behavior and collects data. For example, it learns that the user wakes up at 7:00 every morning and has breakfast at 7:30.

[0034] Step 2: Based on the collected data, the AI ​​engine learns the user's routine work using machine learning techniques.

[0035] Step 3: Based on the learned routine work, the AI ​​engine suggests actions to the user. For example, when the user asks, "What should I do today?", the AI ​​engine suggests, "Wake up at 7:00 and have breakfast at 7:30."

[0036] Step 4: The proposal is communicated to the user. This is done through the notification function of the device (e.g. smartphone, PC) used by the user, or through a voice assistant.

[0037] Example 1

[0038] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0039] In today's busy lifestyles, it is difficult for users to efficiently manage their daily routines and seasonal activities. In particular, choosing daily meals, clothing, necessary procedures, etc. often takes time, which can be a source of stress. In addition, in order to optimize these choices, it is necessary to take into account the user's past behavioral data and current situation, but this is very difficult to do manually.

[0040] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0041] In this invention, the server includes a means for learning a user's routine work by time and season, a means for making action suggestions to the user based on the learned routine work, a means for transmitting the content of the suggestions to the user, a means for collecting user action data, a means for preprocessing the collected data, a means for training an AI engine using the preprocessed data, a means for accepting questions from the user, a means for generating optimal suggestions for the questions, and a means for transmitting the content of the suggestions to the user. This enables the user to efficiently manage daily routine work and seasonal actions and receive optimal suggestions.

[0042] "Means of learning the user's routine work by time and season" refers to technology that collects the user's daily life patterns and habits and uses this to predict the user's behavior.

[0043] The "means of suggesting actions to the user based on learned routine work" refers to technology that analyzes collected data and suggests optimal actions based on the user's past behavioral patterns.

[0044] The "means for communicating the content of the proposal to the user" refers to an interface or communication means for notifying the user of the generated proposal.

[0045] "Means of collecting user behavioral data" refers to technology for obtaining user location information, calendar information, past behavioral history, etc.

[0046] "Means of preprocessing collected data" refers to techniques for filling in missing values, removing outliers, converting data formats, etc. in collected data.

[0047] "Means for training an AI engine using preprocessed data" refers to a technology that uses preprocessed data as input to train an AI model and have it learn user behavior patterns.

[0048] The "means for accepting questions from the user" is an interface that enables the user to input questions to the system.

[0049] "Means for generating optimal suggestions in response to questions" refers to technology that generates optimal answers to questions from users based on a learned model.

[0050] The "means for communicating the content of the proposal to the user" refers to an interface or communication means for notifying the user of the generated proposal.

[0051] This invention relates to a system that learns a user's routine work by time and season and suggests optimal actions. This system includes a series of processes that collect user behavior data, learns using an AI engine, generates optimal suggestions in response to questions from the user, and communicates the suggested content to the user.

[0052] To collect user behavior data, the server obtains location information, calendar information, past behavior history, etc. from smartphones and wearable devices. For example, when a user enters an event such as "going to the gym" into the smartphone calendar, that information is sent to the server.

[0053] Next, the server preprocesses the collected data. Specifically, it complements missing values ​​in the data and removes outliers. It also converts the data into a format that is easy for the AI ​​engine to learn. For example, it converts location information from latitude and longitude to an address.

[0054] Using the preprocessed data, the server trains the AI ​​engine. Specifically, it uses machine learning frameworks such as TENSORFLOW (registered trademark) and PyTorch to model the user's behavioral patterns. For example, if a user has a habit of going to the gym every Monday, that pattern is learned.

[0055] The terminal provides an interface that accepts questions from the user. When the user inputs a question such as "What should I do today?", "What should I eat today?", or "What should I wear today?", the terminal sends the question to the server.

[0056] The server generates optimal suggestions for the received questions from the patterns it has learned. Specifically, it makes appropriate suggestions by taking into account the user's past behavioral history and current situation (weather, time of day, etc.). For example, if a user asks, "What should I eat today?" the server will suggest the optimal menu based on the user's past eating history, the current season, time of day, etc.

[0057] The device conveys the suggestions received from the server to the user. Specifically, it displays them in the form of text messages, voice messages, etc. For example, it displays a suggestion such as "Today is sunny and the temperature is 25 degrees, so it would be good to wear a light jacket and jeans."

[0058] Examples of prompts include:

[0059] "What should I do today?"

[0060] "What should I eat today?"

[0061] "What should I wear today?"

[0062] This enables users to efficiently manage their daily routine work and seasonal activities and receive optimal suggestions.

[0063] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0064] Step 1:

[0065] Collecting user behavior data

[0066] The server collects location information, calendar information, and past activity history from the user's smartphone or wearable device. For example, when a user enters an event such as "going to the gym" into the smartphone calendar, the information is sent to the server. The input data includes location information, calendar information, and activity history, which are then stored in a database.

[0067] Step 2:

[0068] Data Preprocessing

[0069] The server preprocesses the collected data. Specifically, it complements missing values ​​in the data and removes outliers. It also converts the data into a format that is easy for the AI ​​engine to learn from. For example, it converts location information from latitude and longitude to an address. The input data is the raw data collected in step 1, and the output data is the preprocessed, clean data.

[0070] Step 3:

[0071] AI engine learning

[0072] The server trains the AI ​​engine using the preprocessed data. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to model the user's behavioral patterns. For example, if a user has a habit of going to the gym every Monday, that pattern is learned. The input data is the preprocessed data, and the output data is the trained AI model.

[0073] Step 4:

[0074] Accepting questions from users

[0075] The terminal provides an interface that accepts questions from the user. When the user inputs a question such as "What should I do today?", "What should I eat today?", or "What should I wear today?", the terminal sends the question to the server. The input data is the user's question, and the output data is a request to the server.

[0076] Step 5:

[0077] Generate best-fit suggestions for your questions

[0078] The server generates optimal suggestions for the received questions from the patterns it has learned. Specifically, it makes appropriate suggestions by taking into account the user's past behavioral history and current situation (weather, time of day, etc.). For example, if a user asks, "What should I eat today?", the server will suggest the optimal menu based on the user's past meal history, the current season, time of day, etc. The input data is the user's question and the trained AI model, and the output data is the suggestions.

[0079] Step 6:

[0080] Communicating the proposal to users

[0081] The terminal conveys the suggestions received from the server to the user. Specifically, it displays them in the form of text messages, voice messages, etc. For example, it displays a suggestion such as "Today is sunny with a temperature of 25 degrees, so it would be good to wear a light jacket and jeans." The input data is the suggestion from the server, and the output data is a notification to the user.

[0082] (Application example 1)

[0083] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0084] Conventional systems can learn a user's daily life patterns and habits and make action suggestions, but they lack specific product suggestions and real-time notification functions to improve the shopping experience in physical stores. In addition, optimal product suggestions are not made taking into account the user's past purchase history and behavioral patterns, which may reduce user satisfaction. Furthermore, the system lacks the ability to provide appropriate answers to user questions.

[0085] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for learning the user's routine work by time and season, a means for suggesting actions to the user based on the learned routine work, a means for communicating the content of the suggestions to the user, a means for learning the user's past purchase history and behavioral patterns, a means for suggesting optimal products according to the current season and time period, a means for notifying the user of in-store special sale information and new product information in real time, and a means for providing answers to questions from the user. This makes it possible to improve the user's shopping experience and increase satisfaction.

[0086] "Means of learning the user's routine work by time and season" is a function for collecting and analyzing the user's daily life patterns and habits by time and season.

[0087] "A means of suggesting actions to the user based on learned routine work" is a function for suggesting optimal actions to the user based on collected user routine work data.

[0088] "Means of communicating the proposed content to the user" refers to a function for notifying the user of proposed actions or information.

[0089] "Means of learning about a user's past purchase history and behavioral patterns" refers to a function for collecting and analyzing a user's past purchase history and behavioral history.

[0090] "Means for suggesting optimal products according to the current season and time of day" refers to a function for selecting and suggesting optimal products to users based on the current season and time of day.

[0091] "Means of notifying users of in-store sales and new product information in real time" is a function for notifying users of sales and new product information in physical stores in real time.

[0092] "Means for providing answers to questions from users" is a function for providing appropriate answers to questions asked by users.

[0093] The system for carrying out this invention learns the user's routine work by time and season, suggests actions to the user based on the learned routine work, and communicates the contents of the suggestions to the user.Furthermore, it includes a function to learn the user's past purchase history and behavioral patterns, suggest optimal products according to the current season and time of day, and notify the user of in-store special sale information and new product information in real time.

[0094] System Configuration

[0095] Hardware

[0096] Server: A central processing unit for collecting data, analyzing it, and generating recommendations.

[0097] Device: The device used by a user, such as a smartphone or smart glasses.

[0098] Network: Infrastructure for data communication between servers and terminals.

[0099] software

[0100] AI engine: An algorithm that learns users' routines and purchase history to generate action suggestions.

[0101] Database: Storage for storing user behavior patterns, purchase history, and product information.

[0102] Notification system: Software that notifies users of offers, special offers, and new product information in real time.

[0103] Data processing and calculation

[0104] The server collects user behavior data and purchase history, and analyzes this data using an AI engine. Specifically, it processes and calculates the following data:

[0105] 1. Data collection: User behavior data and purchase history are sent from the device to the server.

[0106] 2. Data analysis: The AI ​​engine analyzes the collected data and learns the user's routine work and behavioral patterns.

[0107] 3. Action suggestion generation: Based on the learning results, optimal action suggestions are generated for the user.

[0108] 4. Real-time notifications: Proposals, sale information, and new product information are sent to users' devices in real time.

[0109] Examples

[0110] For example, if User A has a history of purchasing a "Hanami Bento" and a "Spring Jacket" in the spring, the system will learn this and suggest "Hay Fever Prevention Goods" and "Spring Jacket" in the spring. Also, when a user asks "What's recommended today?", the system will suggest the best products according to the current season and time of day.

[0111] Examples of prompt statements

[0112] When a user asks "What's the recommendation for today?", the system generates a prompt like this:

[0113] Build an assistant that learns the user's purchase history and suggests the best products based on the current season. If the user purchased "Hanami Bento" and "Spring Jacket" in spring, suggest recommended products for spring.

[0114] In this way, it is possible to realize a smart shopping assistant that learns the user's behavioral patterns and makes optimal product suggestions.

[0115] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0116] Step 1:

[0117] The user uses the terminal to input behavioral data and purchase history. The terminal sends this data to the server. The input data includes information on purchased products and timestamps of actions. The server receives this data and stores it in a database.

[0118] Step 2:

[0119] The server inputs the user's behavioral data and purchase history stored in the database into the AI ​​engine. The AI ​​engine analyzes this data and learns the user's routine work and behavioral patterns. Specifically, it performs time series analysis and clustering of the data to extract the user's behavioral patterns.

[0120] Step 3:

[0121] The server generates action suggestions based on the learning results. The AI ​​engine considers the user's behavioral patterns and the current season and time of day to generate optimal action suggestions. For example, in the spring season, it suggests "hay fever prevention goods" and "spring jackets." The generated suggestions are stored in a database.

[0122] Step 4:

[0123] A user uses a terminal to input a question such as "What's recommended today?" The terminal sends the question to a server, which analyzes the question and retrieves appropriate action suggestions from a database.

[0124] Step 5:

[0125] The server transmits the obtained action suggestions to the terminal. The terminal notifies the user of the received suggestions. The notification is made by a text message or a push notification. The user can confirm and execute the suggested action or product.

[0126] Step 6:

[0127] The server collects information on special sales and new products in the store in real time and stores it in a database. This information is obtained from the store's management system and external information services.

[0128] Step 7:

[0129] The server notifies the user's terminal of the sale information and new product information collected in real time. The terminal displays the received information to the user. The user can check the sale information and new product information and consider purchasing.

[0130] In this way, it is possible to realize a smart shopping assistant that learns the user's behavioral patterns and makes optimal product suggestions.

[0131] Example 2

[0132] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0133] In modern society, users lead busy lives and are required to efficiently manage their routine work according to time and season. However, conventional systems have difficulty fully learning users' behavioral patterns and habits and providing appropriate action suggestions. In addition, there was a lack of means to quickly provide optimal suggestions when users asked specific questions. This caused users to spend a lot of time and effort on daily choices and plans, preventing them from living an efficient life.

[0134] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0135] In this invention, the server includes: means for learning the user's routine work by time and season, means for suggesting actions to the user based on the learned routine work, means for transmitting the contents of the suggestions to the user, means for mounting an artificial intelligence engine that learns the user's daily life patterns and habits and for suggesting actions to the user based on the learning results, and means for making optimal suggestions based on the learned patterns and transmitting them to the user when the user asks questions such as "What should I do today?", "What should I eat today?", "What should I wear today?", etc. This enables the user to quickly receive optimal suggestions regarding daily choices and plans, enabling them to live an efficient life.

[0136] "User" refers to any individual or entity that uses the System.

[0137] "Routine work by time or season" refers to activities or tasks that a user performs regularly at specific times or seasons.

[0138] "Means of learning" refers to methods and technologies for analyzing user behavioral patterns and habits and accumulating them as data.

[0139] "Means for making action suggestions" refers to methods and technologies for presenting optimal actions and options to users based on learned data.

[0140] "Means for communicating the suggestions to the user" refers to the methods or technologies for notifying or displaying the generated suggestions to the user.

[0141] An "artificial intelligence engine" refers to software and algorithms for machine learning and data analysis.

[0142] "Question" refers to a request for information or inquiry made by a User to the System.

[0143] "Optimal suggestions" refer to the actions or options that are deemed most appropriate based on the user's past data and current situation.

[0144] The system of the present invention learns the user's routine work by time and season, and suggests actions to the user based on the results of that learning. Specifically, it is equipped with an artificial intelligence engine that learns the user's daily life patterns and habits, and suggests actions to the user based on the results of that learning. The suggestions include options such as "menu," "what to wear today," and "procedures that need to be done at this time of year."

[0145] Hardware and Software Used

[0146] Server: Collects, stores, analyzes data, and generates proposals. Specifically, it uses a database server and an AI engine (e.g., TensorFlow or PyTorch).

[0147] Terminal: The terminal where the user inputs data, sends questions, and displays the proposals. Specifically, a smartphone or PC is used.

[0148] User: Enters daily actions and choices and receives suggestions from the system.

[0149] Data collection

[0150] Users input their daily actions and choices into the device. For example, they record what they eat, what they wear, and what their plans are. The device collects this data and sends it to the server. For example, if a user enters "Today's breakfast is bread and coffee" into a smartphone app, that data is saved on the device. The device sends the collected data to the server periodically or in response to user operations.

[0151] Training the Data

[0152] The server receives the collected data and stores it in a database. For example, a user's dietary data and clothing data are stored in the server's database. The server uses an AI engine to learn from the stored data. Specifically, the server analyzes past data and models the user's behavioral patterns and seasonal routines. The server stores the learning results in a database.

[0153] Generate action suggestions

[0154] The user asks questions such as "What should I do today?", "What should I eat today?", and "What should I wear today?". For example, the user enters "What should I eat today?" into a smartphone app. The device sends this question to the server. Specifically, the user's question is sent from the device to the server as an HTTP request. The server generates optimal suggestions based on the learning results. For example, the server may suggest "It would be a good idea to make curry today" taking into account past eating patterns, the season, and the time of day. The server stores the generated suggestions in a database.

[0155] Communication of proposals

[0156] The server sends the generated suggestions to the device. Specifically, the server sends the suggestions to the device as an HTTP response. The device then displays the suggestions to the user. For example, the suggestions are displayed as smartphone notifications or in-app messages. When the user opens the app, the app displays the message, "It would be a good idea to make curry today."

[0157] Examples and prompts

[0158] Example 1: When a user asks, "What should I eat today?"

[0159] Prompt: "What should I eat today?"

[0160] Suggestion: "You might want to make curry today. Based on historical data, you often eat curry on Mondays."

[0161] Example 2: When a user asks, "What should I wear today?"

[0162] Prompt: "What should I wear today?"

[0163] Suggestion: "The temperature is cold today, so you should wear a sweater and a coat."

[0164] Example 3: When a user asks, "What steps do I need to take at this time?"

[0165] Prompt: "What procedures need to be completed at this time?"

[0166] Suggestion: "This month is tax return season, so get ready to go to the tax office."

[0167] In this way, the system learns the user's behavioral patterns and makes optimal suggestions.

[0168] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0169] Step 1: Data collection

[0170] The user inputs daily actions and choices into the terminal, for example recording what they eat, what they wear, what they plan to do, etc.

[0171] Input: A user types "Today's breakfast is bread and coffee" into an app on their smartphone.

[0172] The terminal collects these data and transmits them to a server. Specifically, the terminal uploads the collected data to the server periodically or in response to a user's operation.

[0173] Output: The collected data is sent to the server.

[0174] Step 2: Save your data

[0175] The server stores the received data in a database. For example, the user's food data and clothing data are stored in the server's database.

[0176] Input: User behavior data sent from the device.

[0177] The server uses a database management system (eg, MySQL or PostgreSQL) to store data.

[0178] Output: User behavior data stored in a database.

[0179] Step 3: Train the data

[0180] The server uses an AI engine (such as TensorFlow or PyTorch) to learn from the stored data. Specifically, the server analyzes past data and models the user's behavioral patterns and seasonal routines.

[0181] Input: User behavior data stored in the database.

[0182] The server applies machine learning algorithms to analyze the data and learn, for example, that users tend to eat curry on Mondays.

[0183] Output: A behavioral pattern model as a result of learning.

[0184] Step 4: Generate action suggestions

[0185] A user asks questions such as "What should I do today?", "What should I eat today?", "What should I wear today?", etc. For example, a user types "What should I eat today?" into a smartphone app.

[0186] Input: The user question (prompt).

[0187] The terminal sends this question to the server. Specifically, the user's question is sent from the terminal to the server as an HTTP request.

[0188] The server generates optimal suggestions based on the results of learning. For example, the server might suggest, "It would be good to make curry today," taking into account past eating patterns, the season, and the time of day.

[0189] Output: Generated action suggestions.

[0190] Step 5: Communicate your proposal

[0191] The server sends the generated proposal to the terminal. Specifically, the proposal content is sent from the server to the terminal as an HTTP response.

[0192] Input: Generated action suggestions.

[0193] The device will display the suggestions to the user. For example, it will appear as a smartphone notification or an in-app message. When the user opens the app, it will say, "It would be a good idea to make curry today."

[0194] Output: The suggestions displayed to the user.

[0195] (Application example 2)

[0196] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0197] Conventional behavior suggestion systems can learn a user's routine work and make suggestions, but there is a problem in that it is difficult to collect user behavior data in real time and make more accurate suggestions. In addition, the means by which users can receive suggestions are limited, and a method that allows them to understand the suggestions visually and intuitively is required.

[0198] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0199] In this invention, the server includes a means for learning the user's routine work by time and season, a means for making action suggestions to the user based on the learned routine work, a means for transmitting the content of the suggestions to the user, a wearable device equipped with a sensor for collecting the user's behavioral data, an artificial intelligence engine for analyzing the collected data and learning the user's routine, and a wearable device including a display for generating optimal suggestions based on the user's questions and visually displaying them. This makes it possible to collect the user's behavioral data in real time and visually provide highly accurate suggestions.

[0200] "User" means an individual or organization that uses the System.

[0201] "Routine work" refers to regular tasks and activities that users perform on a daily basis.

[0202] "Means of learning" are methods and technologies for collecting and analyzing user behavior data to recognize patterns.

[0203] "Action suggestions" are content that recommends actions to the user based on the learned routine work.

[0204] "Means of communicating the suggestions" refers to the methods or techniques used to notify the user of the suggested actions.

[0205] A "wearable device with sensors" is a wearable device equipped with sensors for collecting user behavioral data.

[0206] An "artificial intelligence engine" is a machine learning model or algorithm that analyzes collected data and learns user routines.

[0207] A "wearable device including a display" is a wearable device that has the ability to visually display information to a user.

[0208] The system for carrying out this invention learns the user's routine work by time and season, makes action suggestions based on the learned routine work, and communicates the suggested content to the user. Specifically, the system is implemented with the following configuration and procedure.

[0209] System Configuration

[0210] 1. Wearable devices equipped with sensors to collect user behavior data:

[0211] Wearable devices such as smart glasses and smart watches will be used.

[0212] These devices are equipped with cameras, microphones, GPS sensors, etc. to collect user behavior data in real time.

[0213] 2. An artificial intelligence engine that analyzes collected data and learns user routines:

[0214] An AI engine located on the server analyzes the collected data.

[0215] Use machine learning models to learn user behavior patterns and habits.

[0216] 3. A wearable device that includes a display that generates and visually displays optimal recommendations based on a user's questions:

[0217] Suggestions generated by the AI ​​engine are displayed on the display of the wearable device.

[0218] When the user asks a question using voice or gestures, suggestions appear on the display.

[0219] Program processing explanation

[0220] The server receives data from wearable devices equipped with sensors to collect user behavior data. These include smart glasses and smart watches. The collected data is analyzed by an artificial intelligence engine on the server. The AI ​​engine uses machine learning models to learn the user's routine and recognize behavioral patterns.

[0221] When a user asks a question such as "What should I eat today?", the AI ​​engine generates optimal suggestions based on the learned data. These suggestions are visually displayed on the display of the wearable device. For example, if a user asks "What should I eat today?", the display may show "I recommend curry for dinner tonight."

[0222] Examples of concrete examples and prompts

[0223] Examples

[0224] User Question: "What should I eat today?"

[0225] AI suggestion: "I recommend curry for dinner tonight."

[0226] Examples of prompt statements

[0227] Make suggestions for dinner today based on user behavior data. Users are asking, "What should I eat today?"

[0228] In this way, it is possible to collect user behavior data in real time and provide highly accurate visual suggestions.

[0229] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0230] Step 1:

[0231] A user puts on a wearable device. The wearable device is equipped with a camera, microphone, GPS sensor, etc., and these sensors collect the user's behavior data in real time. The input is the user's location information, audio data, video data, etc., and the output is the collection of this data.

[0232] Step 2:

[0233] The terminal transmits the collected behavioral data to the server. The input is the behavioral data transmitted from the wearable device, and the output is the data transmitted to the server. Specifically, the device uploads data to the server at regular intervals.

[0234] Step 3:

[0235] The server analyzes the behavioral data it receives. The input is the behavioral data sent from the device, and the output is the analysis results. In concrete terms, an artificial intelligence engine on the server analyzes the data and learns the user's behavioral patterns and routines.

[0236] Step 4:

[0237] A user asks a question to a wearable device. For example, the user asks by voice, "What should I eat today?" The input is the user's voice data, and the output is the recognition result of the voice data. In concrete terms, the device uses voice recognition technology to convert the user's question into text data.

[0238] Step 5:

[0239] The server generates optimal suggestions based on the user's question. The input is the user's question and analyzed behavioral data, and the output is the content of the suggestions. In concrete terms, the server generates optimal suggestions for the user's question based on the data learned by the artificial intelligence engine.

[0240] Step 6:

[0241] The server transmits the generated proposal to the terminal. The input is the generated proposal, and the output is the proposal transmitted to the terminal. In concrete terms, the server transmits the proposal to the wearable device.

[0242] Step 7:

[0243] The terminal visually displays the proposal. The input is the proposal sent from the server, and the output is the proposal displayed on the display. The specific operation is to display the proposal on the display of the wearable device and notify the user.

[0244] In this way, it is possible to collect user behavior data in real time and provide highly accurate visual suggestions.

[0245] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0246] "Example 1"

[0247] As one embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotions of a user. This system learns the user's routine work by time and season, makes action suggestions to the user based on the learned routine work, and transmits the suggested content to the user. Furthermore, the system recognizes the user's emotions and adjusts the action suggestions according to the emotions. Specifically, the emotion engine recognizes emotions from the user's tone of voice, facial expressions, behavior patterns, etc. For example, if the user speaks in a sad tone of voice, the emotion engine recognizes that the user is sad and transmits the information to the AI ​​engine. The AI ​​engine, after recognizing that the user is sad, makes action suggestions to the user. For example, it makes a suggestion such as "How about listening to relaxing music today?" In this way, the system of the present invention can make personalized suggestions that take the user's emotions into consideration.

[0248] "Example 2"

[0249] As one embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotions of a user. This system learns the user's routine work by time and season, makes action suggestions to the user based on the learned routine work, and transmits the suggested content to the user. Furthermore, the system recognizes the user's emotions and adjusts the action suggestions according to the emotions. Specifically, the emotion engine recognizes emotions from the user's tone of voice, facial expressions, behavior patterns, etc. For example, if the user speaks in a sad tone of voice, the emotion engine recognizes that the user is sad and transmits the information to the AI ​​engine. The AI ​​engine, after recognizing that the user is sad, makes action suggestions to the user. For example, it makes a suggestion such as "How about listening to relaxing music today?" In this way, the system of the present invention can make personalized suggestions that take the user's emotions into consideration.

[0250] The process flow of each embodiment will be described below.

[0251] "Example 1"

[0252] Step 1: The system learns the user's routine work by time and season. This learning is done from the user's behavioral history, selection history, etc.

[0253] Step 2: Based on the learned routine, the system will provide the user with action suggestions, which will vary according to the user's current time and season.

[0254] Step 3: The system recognizes the user's emotions using an emotion engine that recognizes the user's emotions from the user's tone of voice, facial expressions, behavioral patterns, etc.

[0255] Step 4: Based on the emotion engine's recognized emotion, the system will adjust the action suggestions. For example, if the user is sad, the system will suggest listening to relaxing music, etc., depending on the user's emotion.

[0256] Example 1

[0257] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0258] Conventional systems can learn a user's routine work by time and season and suggest actions based on that, but they cannot make personalized suggestions that take into account the user's emotions. This makes it difficult to provide optimal suggestions based on the user's emotional state.

[0259] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0260] In this invention, the server includes a means for learning the user's routine work by time and season, a means for making action suggestions to the user based on the learned routine work, a means for communicating the suggested content to the user, a means for recognizing the user's emotion, and a means for adjusting the action suggestions based on the recognized emotion, thereby enabling personalized suggestions according to the user's emotional state.

[0261] "User's routine work by time or season" refers to actions or habits that a user repeatedly performs at specific times of the day or during specific seasons.

[0262] "Learning means" refers to technologies and algorithms used to collect and analyze user behavior data to recognize patterns.

[0263] "Means for making action suggestions" refers to technologies and algorithms for suggesting optimal actions to users based on learned user routine work.

[0264] "Means for communicating suggestions to the user" refers to an interface or device for notifying the user of the generated action suggestions.

[0265] "Means of recognizing user emotions" refers to technologies and algorithms for analyzing and recognizing emotions from a user's tone of voice, facial expressions, behavioral patterns, etc.

[0266] "Means for tailoring action suggestions based on recognized emotions" refers to technologies or algorithms for appropriately modifying the content of action suggestions based on the user's emotional state.

[0267] The present invention is a system that learns a user's routine work by time and season, suggests actions based on the learned routine work, and further recognizes the user's emotions to adjust the suggested content. A specific embodiment of this system will be described below.

[0268] System Configuration

[0269] Hardware and Software

[0270] The server is equipped with an AI engine for collecting and analyzing user behavior data. This AI engine uses machine learning algorithms to learn the user's routine work. It also has an emotion engine that recognizes emotions from the user's tone of voice, facial expressions, behavioral patterns, etc. The terminal is a device such as a smartphone or tablet, and plays a role in conveying the suggestions from the server to the user.

[0271] Data collection and learning

[0272] The server collects data about the user's daily life. For example, the user inputs information such as "Wake up at 7am," "Eat breakfast at 8am," and "Get home at 6pm" through a smartphone app. This data is input into an AI engine, which analyzes the user's behavioral patterns and models routines by time and season.

[0273] Generate action suggestions

[0274] The server generates action suggestions for the user based on the learning results. For example, when a user asks, "What should I do today?", the server uses an AI engine to generate optimal suggestions. Specifically, it makes suggestions such as, "The weather is nice today, so how about going for a walk?"

[0275] Communication of proposals

[0276] The device transmits the suggestions received from the server to the user. The user can receive the suggestions through a device such as a smartphone or tablet. For example, if the user asks "What should I eat today?" on their smartphone, the device will display "It would be good to eat a salad with lots of vegetables today."

[0277] Emotion recognition and suggestion adjustment

[0278] The server recognizes the user's emotions using an emotion engine. The emotion engine analyzes emotions from the user's tone of voice, facial expressions, behavioral patterns, etc. For example, if the user speaks in a sad tone of voice, the emotion engine recognizes the user as sad and transmits that information to the AI ​​engine. The server adjusts behavior suggestions based on the emotion information received from the emotion engine. For example, if the user is recognized as sad, the server makes a suggestion such as, "How about listening to some relaxing music today?"

[0279] Examples

[0280] Example 1: User questions and suggestions

[0281] When a user asks, "What should I eat today?", the server takes into account past data and the current season and suggests, "How about eating a salad with lots of vegetables today?" The device displays this suggestion to the user.

[0282] Example 2: Sentiment-based suggestions

[0283] When a user asks "What should I do today?" in a sad tone of voice, the emotion engine recognizes that the user is sad, and the server suggests "How about listening to some relaxing music today?" The device displays this suggestion to the user.

[0284] Examples of prompt statements

[0285] "What should I do today?"

[0286] "What should I eat today?"

[0287] "What should I wear today?"

[0288] In this way, the system of the present invention can make personalized suggestions that take into account the user's routine and emotions.

[0289] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0290] Step 1:

[0291] The server collects data about the user's daily life. The user inputs information such as "wake up at 7am", "have breakfast at 8am", and "return home at 6pm" through a smartphone app. This data is sent to the server and stored in a database. The input data is the user's behavioral patterns, and the output is the stored behavioral data.

[0292] Step 2:

[0293] The server inputs the collected data into the AI ​​engine, which learns the user's routine work. The AI ​​engine uses machine learning algorithms to analyze the user's behavioral patterns and model routines by time and season. For example, the AI ​​engine learns a pattern that "the user wakes up at 7am on weekdays and 8am on weekends." The input data is the stored behavioral data, and the output is a model of the learned routine work.

[0294] Step 3:

[0295] The server generates action suggestions for the user based on the learning results. When the user asks, "What should I do today?", the server uses an AI engine to generate optimal suggestions. For example, it may suggest, "The weather is nice today, so how about going for a walk?" The input data is the user's question and the learned routine work model, and the output is the generated action suggestions.

[0296] Step 4:

[0297] The terminal transmits the suggestions received from the server to the user. The user can receive the suggestions through a terminal such as a smartphone or tablet. For example, if the user asks on their smartphone, "What should I eat today?", the terminal will display, "It would be good to eat a salad with lots of vegetables today." The input data is the suggestions from the server, and the output is the suggestions that are displayed to the user.

[0298] Step 5:

[0299] The server uses an emotion engine to recognize the user's emotions. The emotion engine analyzes emotions from the user's tone of voice, facial expressions, behavioral patterns, etc. For example, if a user speaks in a sad tone of voice, the emotion engine recognizes that the user is sad and transmits that information to the AI ​​engine. The input data are the user's tone of voice, facial expressions, and behavioral patterns, and the output is the recognized emotion information.

[0300] Step 6:

[0301] The server adjusts the behavioral suggestions based on the emotion information received from the emotion engine. For example, if the user is recognized as sad, the server will suggest, "How about listening to some relaxing music today?" The input data are the recognized emotion information and the learned routine work model, and the output is the adjusted behavioral suggestions.

[0302] (Application example 1)

[0303] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0304] Conventional behavior suggestion systems can make suggestions based on the user's routine work, but they cannot make suggestions that take into account the user's emotions and shopping patterns, so personalized suggestions are lacking. In addition, there are limited means to effectively communicate the contents of the suggestions to the user, which means that the system cannot fully improve user convenience.

[0305] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0306] In this invention, the server includes a means for learning a user's routine work by time and season, a means for making an action suggestion to the user based on the learned routine work, a means for transmitting the content of the suggestion to the user, a means for recognizing the user's emotion, a means for adjusting the action suggestion based on the recognized emotion, a means for learning the user's shopping pattern, a means for making a product suggestion based on the learned shopping pattern, and a means for transmitting the content of the suggestion through a smartphone notification or a voice assistant. This enables personalized suggestions that take into account the user's emotion and shopping pattern, thereby improving user convenience.

[0307] "Means of learning the user's routine work by time and season" is a function for recording and analyzing the user's daily life patterns and habits according to time and season.

[0308] "Means of suggesting actions to the user based on learned routine work" is a function for suggesting optimal actions to the user based on recorded routine work data.

[0309] "Means of communicating the content of the suggestions to the user" refers to a function for notifying the user of the content of the action suggestions, and is carried out through smartphone notifications or voice assistants.

[0310] "Means of recognizing the user's emotions" refers to a function for analyzing and recognizing emotions from the user's tone of voice, facial expressions, behavioral patterns, etc.

[0311] The "means for adjusting action suggestions based on the recognized emotions" is a function for appropriately changing the content of action suggestions according to the user's emotional state.

[0312] "Means of learning user shopping patterns" refers to a function for recording and analyzing a user's past purchasing history and behavior.

[0313] The "means for making product suggestions based on learned shopping patterns" is a function for suggesting optimal products to users based on recorded shopping pattern data.

[0314] "Means of communicating the content of suggestions through smartphone notifications or voice assistants" refers to a function for communicating the content of action suggestions or product suggestions to users using the notification function of a smartphone or a voice assistant.

[0315] The system for implementing this invention learns the user's routine work by time and season, makes action suggestions based on the learned routine work, and communicates the suggested content to the user. It also has a function of recognizing the user's emotions and adjusting the action suggestions based on the emotions. Furthermore, it learns the user's shopping patterns and makes product suggestions based on the learned shopping patterns.

[0316] Hardware and Software Used

[0317] Hardware:

[0318] Smartphone (iOS or ANDROID (registered trademark))

[0319] software:

[0320] Python

[0321] TextBlob (a library for sentiment analysis)

[0322] Data processing and calculation

[0323] server:

[0324] The server records and analyzes the user's daily life patterns and habits to learn the user's time and seasonal routines, which includes storing the activities performed by the user and the timestamps of those activities in a database.

[0325] Device:

[0326] The device (smartphone) receives input from the user and recognizes emotions based on the input. For emotion recognition, the TextBlob library is used to analyze the user's input text and classify it as positive, negative, or neutral.

[0327] User:

[0328] The user asks questions such as "What should I do today?", "What should I eat today?", "What should I wear today?", "What are today's recommended products?", etc. through their smartphone. In response, the server generates optimal suggestions based on the learned routine work, shopping patterns, and recognized emotions, and conveys them to the user through smartphone notifications or voice assistants.

[0329] Examples

[0330] For example, if a user types "I'm a little tired today," the device analyzes this input using the TextBlob library and recognizes it as a negative emotion. The server receives this emotion information and, taking into account the user's past routine work and shopping patterns, generates suggestions such as "I recommend products that will help you relax today." These suggestions are communicated to the user through smartphone notifications and voice assistants.

[0331] Examples of prompt statements

[0332] Examples of prompts for a generative AI model might include the following:

[0333] Develop a system that recognizes the user's emotions and suggests actions based on those emotions. If a user types, "I'm a bit tired today," suggest products that will help them relax.

[0334] In this way, personalized suggestions that take into account the user's emotions and shopping patterns are possible, increasing user convenience.

[0335] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0336] Step 1:

[0337] A user types "I'm a little tired today" through their smartphone. The typed text is sent to the terminal.

[0338] Step 2:

[0339] The device receives the input text and performs sentiment analysis using the TextBlob library, specifically classifying the text as positive, negative, or neutral. The input is the user's text, and the output is the sentiment classification result.

[0340] Step 3:

[0341] The terminal transmits the result of emotion analysis to the server. The input is the emotion classification result, and the output is the data transmission to the server.

[0342] Step 4:

[0343] The server receives the emotion analysis results and retrieves the user's past routine work and shopping patterns from the database. The input is the emotion classification result, and the output is the data of the user's past routine work and shopping patterns.

[0344] Step 5:

[0345] The server generates optimal action suggestions for the user based on the acquired data. For example, if the user is showing negative emotions, it will suggest products that will help them relax. The input is data on past routine work and shopping patterns and the results of emotion classification, and the output is action suggestions.

[0346] Step 6:

[0347] The server transmits the generated action proposal to the terminal. The input is the action proposal, and the output is data transmission to the terminal.

[0348] Step 7:

[0349] The device notifies the user of the action suggestions it receives. Specifically, it communicates the content of the suggestions using the notification function of the smartphone or a voice assistant. The input is the action suggestions, and the output is a notification to the user.

[0350] In this way, personalized suggestions are made based on the user's emotions and past behavioral patterns.

[0351] Example 2

[0352] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0353] Conventional systems suggested actions based on the user's routine work, but did not take the user's emotions into account. As a result, appropriate suggestions could not be made according to the user's emotional state, making it difficult to improve user satisfaction. In addition, responses to questions from users were uniform, and personalized suggestions tailored to the individual user's situation were lacking.

[0354] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0355] In this invention, the server includes a means for learning the user's routine work by time and season, a means for making an action suggestion to the user based on the learned routine work, a means for communicating the suggestion content to the user, a means for recognizing the user's emotion, and a means for adjusting the action suggestion based on the recognized emotion, thereby enabling personalized suggestions that take into account the user's routine work and emotion.

[0356] A "means for learning a user's time and seasonal routine work" is a device or program for collecting and analyzing a user's daily life patterns and habits.

[0357] The "means for suggesting actions to the user based on learned routine work" refers to a device or program for suggesting optimal actions to the user based on collected and analyzed data.

[0358] The "means for communicating the content of the suggestion to the user" refers to a device or program for notifying the user of the generated action suggestion.

[0359] The "means for recognizing a user's emotions" refers to a device or program for detecting and recognizing emotions from a user's tone of voice, facial expressions, behavioral patterns, etc.

[0360] The "means for adjusting action suggestions based on the recognized emotion" refers to a device or program for modifying the content of action suggestions in accordance with the user's emotional state and making optimal suggestions.

[0361] The present invention is a system that learns a user's routine work by time and season, suggests actions based on the learned routine work, and further recognizes the user's emotions to adjust the suggested content. A specific embodiment of this system will be described below.

[0362] System Configuration

[0363] Hardware

[0364] Server: A central processing unit for data collection, analysis, proposal generation, and emotion recognition.

[0365] Device: The device a user uses, such as a smartphone or tablet.

[0366] Sensors: Microphones and cameras to recognize the user's tone of voice and facial expressions.

[0367] software

[0368] AI engine: Software that learns a user's routine work and generates action suggestions.

[0369] Emotion engine: Software for recognizing user emotions and communicating that information to the AI ​​engine.

[0370] Data collection and learning

[0371] The server collects user behavior data, including what time the user wakes up, what meals they eat, what clothes they wear, etc. Specifically, it collects data from smartphones and wearable devices.

[0372] Example: Information entered by a user into a smartphone app or activity data obtained from a wearable device is sent to a server.

[0373] Generate action suggestions

[0374] The server uses an AI engine to learn the user's routine work based on the collected data, and the AI ​​engine analyzes time and seasonal patterns to generate a model that predicts user behavior.

[0375] Example: A server analyzes historical data to determine whether users tend to behave in certain ways at certain times.

[0376] Communicating the proposal

[0377] The server transmits the generated suggestions to the user. When the user inputs a question via a device (such as a smartphone or tablet), the server responds with the best suggestions for that question.

[0378] Example: A user types a question into a smartphone app and receives suggestions from the server.

[0379] Emotion recognition and suggestion adjustment

[0380] The server uses an emotion engine to recognize the user's emotion, which is determined from the user's tone of voice, facial expression, behavioral patterns, etc.

[0381] Example: Emotions are recognized by analyzing the tone of voice when a user speaks through a smartphone microphone and facial expressions captured through the camera.

[0382] The server then uses the information obtained from the emotion engine to suggest actions based on the user's emotions. For example, if the server detects that the user is sad, it will suggest, "How about listening to some relaxing music today?"

[0383] Examples of prompt statements

[0384] "What should I do today?"

[0385] "What should I eat today?"

[0386] "What should I wear today?"

[0387] In this way, the inventive system can make personalized suggestions that take into account the user's routine and emotions.

[0388] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0389] Step 1:

[0390] User data collection

[0391] The server collects user behavior data. As input, it receives data provided by the user from a smartphone or wearable device. This data includes the user's wake-up time, what they ate, what they were wearing, etc. The server stores this data in a database for later analysis.

[0392] Specific operation: Information entered by the user into the smartphone app and activity data acquired from the wearable device are sent to the server, which receives the data and stores it in a database.

[0393] Step 2:

[0394] Learning routine work

[0395] The server uses the AI ​​engine to learn the user's routine based on the collected data. As input, it uses past behavioral data stored in a database. The AI ​​engine analyzes these data and finds patterns over time and seasons. As output, it generates a model that predicts the user's behavior.

[0396] How it works: The server analyzes past data and finds out if a user tends to behave in a certain way at certain times. For example, it learns that a user wakes up at 7am every morning and eats breakfast at 8am.

[0397] Step 3:

[0398] Generate action suggestions

[0399] The server generates action suggestions for the user based on the learned routine work. As input, it uses the prediction model generated by the AI ​​engine and questions from the user. As output, it generates optimal action suggestions and provides them to the user.

[0400] What happens: If a user asks, "What should I eat today?" the server uses past data to suggest, "How about a salad with lots of vegetables today?"

[0401] Step 4:

[0402] Communication of proposals

[0403] The server communicates the generated suggestions to the user, uses the generated action suggestions as input, and sends the suggestions to the user's device as output.

[0404] How it works: A user enters a question into a smartphone app and receives suggestions from the server, which then sends the suggestions to the user's device, where the user can review them.

[0405] Step 5:

[0406] Emotion recognition

[0407] The server uses an emotion engine to recognize the user's emotions. As input, it uses the user's tone of voice, facial expressions, and behavioral patterns. As output, it recognizes the user's emotional state and conveys that information to the AI ​​engine.

[0408] Specific operation: The system recognizes emotions by analyzing the tone of voice the user speaks through the smartphone microphone and facial expressions captured through the camera. For example, if the user speaks in a sad tone, the emotion engine transmits that information to the AI ​​engine.

[0409] Step 6:

[0410] Adjusting behavior suggestions according to emotions

[0411] The server proposes actions based on the user's emotions based on the information obtained from the emotion engine. As input, it uses the emotion information from the emotion engine and the prediction model of the AI ​​engine. As output, it generates optimal action proposals based on emotions and provides them to the user.

[0412] Specific behavior: If the user is recognized as sad, the server will make a suggestion such as "How about listening to some relaxing music today?" The server will send this suggestion to the user's device, and the user will confirm it.

[0413] (Application example 2)

[0414] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0415] Conventional systems suggested actions based on the user's time and seasonal routines, but they lacked personalized suggestions that took into account the user's emotions and purchasing history. In addition, they did not offer real-time product suggestions to improve the shopping experience in physical stores. This led to problems such as reduced user satisfaction and convenience.

[0416] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0417] In this invention, the server includes a means for learning a user's routine work by time and season, a means for making an action suggestion to the user based on the learned routine work, a means for transmitting the suggestion content to the user, a means for recognizing the user's emotion, a means for adjusting the action suggestion based on the recognized emotion, a means for learning the user's purchase history, a means for making a product suggestion based on the learned purchase history, and a means for displaying the suggestion content to the user in real time. This enables personalized suggestions that take into account the user's emotion and purchase history, thereby improving the shopping experience at a physical store.

[0418] "User's routine work by time or season" refers to actions or habits that a user repeatedly performs at specific times of the day or during specific seasons.

[0419] "Means of learning" refers to the technologies and algorithms used to collect and analyze user behavioral patterns and habits as data.

[0420] "Means for making action suggestions" refers to technologies or algorithms that present optimal actions and options to users based on learned data.

[0421] "Means of communicating suggestions to users" refers to any interface or device used to inform users of suggested actions or options.

[0422] "Means of recognizing user emotions" refers to technologies and algorithms for analyzing and recognizing emotions from a user's tone of voice, facial expressions, behavioral patterns, etc.

[0423] "Means for tailoring action suggestions based on recognized emotions" means techniques or algorithms for modifying the content of action suggestions in response to the user's emotional state.

[0424] "Means of learning a user's purchasing history" refers to technologies and algorithms for collecting and analyzing data on products a user has previously purchased.

[0425] "Means for making product suggestions based on learned purchasing history" refers to technologies and algorithms for suggesting optimal products based on a user's purchasing history.

[0426] "Means for displaying suggestions to the user in real time" refers to an interface or device for instantly informing the user of suggested products or actions.

[0427] The system for implementing this invention learns the user's routine work by time and season, makes action suggestions based on the learned routine work, and communicates the suggested content to the user. It also recognizes the user's emotions and adjusts the action suggestions based on the recognized emotions. It also learns the user's purchase history, makes product suggestions based on the learned purchase history, and displays the suggested content to the user in real time.

[0428] Hardware and Software Used

[0429] Hardware: smart glasses, camera, microphone, server

[0430] Software: Emotion recognition model (OpenCV), clustering algorithm (KMeans), generative AI model

[0431] Data processing and calculation

[0432] 1. Learn user's time and seasonal routines:

[0433] The server collects data on users' behavioral patterns and habits and analyzes them using a clustering algorithm (KMeans), thereby learning the user's routine work.

[0434] 2. Generate action suggestions:

[0435] Based on the learned data, generative AI models are used to suggest optimal actions and options to users, and these suggestions are provided as answers to the user's questions.

[0436] 3. Emotion recognition:

[0437] The camera and microphone installed in the smart glasses are used to collect the user's facial expressions and tone of voice, which are then analyzed using an emotion recognition model (OpenCV), allowing the system to recognize the user's emotions.

[0438] 4. Tailoring behavioral suggestions based on emotions:

[0439] Based on the emotions recognized, the generative AI model will tailor action suggestions, for example making suggestions to help users relax if they are sad.

[0440] 5. Learning purchase history:

[0441] The server collects the user's past purchase history and analyzes it using a clustering algorithm (KMeans), thereby learning the user's purchase history.

[0442] 6. Product proposal generation:

[0443] Based on the learned purchasing history, a generative AI model is used to suggest optimal products, which are then displayed in real time on the smart glasses' display.

[0444] Examples

[0445] User's purchasing history: In the past, they have purchased T-shirts and sunglasses.

[0446] Current season: Summer.

[0447] User emotion: "happy" recognized from camera footage.

[0448] Examples of prompt statements

[0449] "Learn your users' purchasing history and suggest the best products based on the current season and sentiment."

[0450] "Please suggest products that users are likely to purchase in the summer."

[0451] "Suggest products that users are likely to purchase when they're in a happy mood."

[0452] In this way, personalized suggestions can be made that take into account the user's emotions and purchasing history, improving the in-store shopping experience.

[0453] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0454] Step 1:

[0455] The server learns the user's routine work by time and season. Specifically, it collects the user's behavioral patterns and habits as data and analyzes them using a clustering algorithm (KMeans). The input is the user's behavioral data, and the output is the user's routine work pattern.

[0456] Step 2:

[0457] The server makes action suggestions to the user based on the learned routine work. Using a generative AI model, it suggests optimal actions and options to the user. The input required is the routine work pattern obtained in step 1, and the output is the action suggestions for the user.

[0458] Step 3:

[0459] The terminal communicates the proposed content to the user. Specifically, it displays the proposed content on the display of the smart glasses. As input, the proposed action obtained in step 2 is required, and as output, the user can visually confirm the proposed content.

[0460] Step 4:

[0461] The device recognizes the user's emotions. It uses the camera and microphone installed in the smart glasses to collect the user's facial expressions and tone of voice, and analyzes them using an emotion recognition model (OpenCV). The input requires the user's facial expressions and tone of voice data, and the output is the user's emotional state.

[0462] Step 5:

[0463] The server adjusts the action suggestions based on the recognized emotions. Using a generative AI model, it generates action suggestions according to the user's emotional state. As input, it requires the emotional state data obtained in step 4, and as output, it obtains the adjusted action suggestions.

[0464] Step 6:

[0465] The server learns the user's purchase history. It collects the user's past purchase history and analyzes it using a clustering algorithm (KMeans). As input, it requires the user's purchase history data, and as output, it obtains the user's purchase pattern.

[0466] Step 7:

[0467] The server makes product suggestions based on the learned purchasing history. It uses a generative AI model to suggest the best products for the user. As input, it requires the purchasing patterns obtained in step 6, and as output, it obtains product suggestions for the user.

[0468] Step 8:

[0469] The terminal then displays the,suggested content to the user in real time.,The suggested products and actions are instantly displayed,on the display of the smart glasses.,As input, the product suggestions obtained in,step 7 are required, and as output, the user can,check the suggested content in real time.

[0470] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0471] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0472] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0473] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0474] [Second embodiment]

[0475] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0476] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0477] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0478] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0479] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0480] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0481] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0482] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0483] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0484] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0485] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0486] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0487] "Example 1"

[0488] The system of the present invention includes a means for learning the user's routine work by time and season, a means for suggesting actions to the user based on the learned routine work, and a means for communicating the suggested content to the user. Specifically, the system is equipped with an AI engine that learns the user's daily life patterns and habits, and makes action suggestions to the user based on the learning results. The suggested content includes options such as "menu," "clothes to wear today," and "procedures that need to be done at this time of year." These suggestions are made based on the learned patterns and communicated to the user when the user asks questions such as "What should I do today?", "What should I eat today?", and "What should I wear today?".

[0489] "Example 2"

[0490] The system of the present invention includes a means for learning the user's routine work by time and season, a means for suggesting actions to the user based on the learned routine work, and a means for communicating the suggested content to the user. Specifically, the system is equipped with an AI engine that learns the user's daily life patterns and habits, and makes action suggestions to the user based on the learning results. The suggested content includes options such as "menu," "clothes to wear today," and "procedures that need to be done at this time of year." These suggestions are made based on the learned patterns and communicated to the user when the user asks questions such as "What should I do today?", "What should I eat today?", and "What should I wear today?".

[0491] The process flow of each embodiment will be described below.

[0492] "Example 1"

[0493] Step 1: Learn the user's daily patterns and habits The AI ​​engine observes the user's behavior and collects data. For example, it learns that the user wakes up at 7:00 every morning and has breakfast at 7:30.

[0494] Step 2: Based on the collected data, the AI ​​engine learns the user's routine work using machine learning techniques.

[0495] Step 3: Based on the learned routine work, the AI ​​engine suggests actions to the user. For example, when the user asks, "What should I do today?", the AI ​​engine suggests, "Wake up at 7:00 and have breakfast at 7:30."

[0496] Step 4: The proposal is communicated to the user. This is done through the notification function of the device (e.g. smartphone, PC) used by the user, or through a voice assistant.

[0497] Example 1

[0498] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0499] In today's busy lifestyles, it is difficult for users to efficiently manage their daily routines and seasonal activities. In particular, choosing daily meals, clothing, necessary procedures, etc. often takes time, which can be a source of stress. In addition, in order to optimize these choices, it is necessary to take into account the user's past behavioral data and current situation, but this is very difficult to do manually.

[0500] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0501] In this invention, the server includes a means for learning a user's routine work by time and season, a means for making action suggestions to the user based on the learned routine work, a means for transmitting the content of the suggestions to the user, a means for collecting user action data, a means for preprocessing the collected data, a means for training an AI engine using the preprocessed data, a means for accepting questions from the user, a means for generating optimal suggestions for the questions, and a means for transmitting the content of the suggestions to the user. This enables the user to efficiently manage daily routine work and seasonal actions and receive optimal suggestions.

[0502] "Means of learning the user's routine work by time and season" refers to technology that collects the user's daily life patterns and habits and uses this to predict the user's behavior.

[0503] The "means of suggesting actions to the user based on learned routine work" refers to technology that analyzes collected data and suggests optimal actions based on the user's past behavioral patterns.

[0504] The "means for communicating the content of the proposal to the user" refers to an interface or communication means for notifying the user of the generated proposal.

[0505] "Means of collecting user behavioral data" refers to technology for obtaining user location information, calendar information, past behavioral history, etc.

[0506] "Means of preprocessing collected data" refers to techniques for filling in missing values, removing outliers, converting data formats, etc. in collected data.

[0507] "Means for training an AI engine using preprocessed data" refers to a technology that uses preprocessed data as input to train an AI model and have it learn user behavior patterns.

[0508] The "means for accepting questions from the user" is an interface that enables the user to input questions to the system.

[0509] "Means for generating optimal suggestions in response to questions" refers to technology that generates optimal answers to questions from users based on a learned model.

[0510] The "means for communicating the content of the proposal to the user" refers to an interface or communication means for notifying the user of the generated proposal.

[0511] This invention relates to a system that learns a user's routine work by time and season and suggests optimal actions. This system includes a series of processes that collect user behavior data, learns using an AI engine, generates optimal suggestions in response to questions from the user, and communicates the suggested content to the user.

[0512] To collect user behavior data, the server obtains location information, calendar information, past behavior history, etc. from smartphones and wearable devices. For example, when a user enters an event such as "going to the gym" into the smartphone calendar, that information is sent to the server.

[0513] Next, the server preprocesses the collected data. Specifically, it complements missing values ​​in the data and removes outliers. It also converts the data into a format that is easy for the AI ​​engine to learn. For example, it converts location information from latitude and longitude to an address.

[0514] Using the preprocessed data, the server trains the AI ​​engine. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to model the user's behavioral patterns. For example, if a user has a habit of going to the gym every Monday, it will learn that pattern.

[0515] The terminal provides an interface that accepts questions from the user. When the user inputs a question such as "What should I do today?", "What should I eat today?", or "What should I wear today?", the terminal sends the question to the server.

[0516] The server generates optimal suggestions for the received questions from the patterns it has learned. Specifically, it makes appropriate suggestions by taking into account the user's past behavioral history and current situation (weather, time of day, etc.). For example, if a user asks, "What should I eat today?" the server will suggest the optimal menu based on the user's past eating history, the current season, time of day, etc.

[0517] The device conveys the suggestions received from the server to the user. Specifically, it displays them in the form of text messages, voice messages, etc. For example, it displays a suggestion such as "Today is sunny and the temperature is 25 degrees, so it would be good to wear a light jacket and jeans."

[0518] Examples of prompts include:

[0519] "What should I do today?"

[0520] "What should I eat today?"

[0521] "What should I wear today?"

[0522] This enables users to efficiently manage their daily routine work and seasonal activities and receive optimal suggestions.

[0523] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0524] Step 1:

[0525] Collecting user behavior data

[0526] The server collects location information, calendar information, and past activity history from the user's smartphone or wearable device. For example, when a user enters an event such as "going to the gym" into the smartphone calendar, the information is sent to the server. The input data includes location information, calendar information, and activity history, which are then stored in a database.

[0527] Step 2:

[0528] Data Preprocessing

[0529] The server preprocesses the collected data. Specifically, it complements missing values ​​in the data and removes outliers. It also converts the data into a format that is easy for the AI ​​engine to learn from. For example, it converts location information from latitude and longitude to an address. The input data is the raw data collected in step 1, and the output data is the preprocessed, clean data.

[0530] Step 3:

[0531] AI engine learning

[0532] The server trains the AI ​​engine using the preprocessed data. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to model the user's behavioral patterns. For example, if a user has a habit of going to the gym every Monday, that pattern is learned. The input data is the preprocessed data, and the output data is the trained AI model.

[0533] Step 4:

[0534] Accepting questions from users

[0535] The terminal provides an interface that accepts questions from the user. When the user inputs a question such as "What should I do today?", "What should I eat today?", or "What should I wear today?", the terminal sends the question to the server. The input data is the user's question, and the output data is a request to the server.

[0536] Step 5:

[0537] Generate best-fit suggestions for your questions

[0538] The server generates optimal suggestions for the received questions from the patterns it has learned. Specifically, it makes appropriate suggestions by taking into account the user's past behavioral history and current situation (weather, time of day, etc.). For example, if a user asks, "What should I eat today?", the server will suggest the optimal menu based on the user's past meal history, the current season, time of day, etc. The input data is the user's question and the trained AI model, and the output data is the suggestions.

[0539] Step 6:

[0540] Communicating the proposal to users

[0541] The terminal conveys the suggestions received from the server to the user. Specifically, it displays them in the form of text messages, voice messages, etc. For example, it displays a suggestion such as "Today is sunny with a temperature of 25 degrees, so it would be good to wear a light jacket and jeans." The input data is the suggestion from the server, and the output data is a notification to the user.

[0542] (Application example 1)

[0543] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0544] Conventional systems can learn a user's daily life patterns and habits and make action suggestions, but they lack specific product suggestions and real-time notification functions to improve the shopping experience in physical stores. In addition, optimal product suggestions are not made taking into account the user's past purchase history and behavioral patterns, which may reduce user satisfaction. Furthermore, the system lacks the ability to provide appropriate answers to user questions.

[0545] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for learning the user's routine work by time and season, a means for suggesting actions to the user based on the learned routine work, a means for communicating the content of the suggestions to the user, a means for learning the user's past purchase history and behavioral patterns, a means for suggesting optimal products according to the current season and time period, a means for notifying the user of in-store special sale information and new product information in real time, and a means for providing answers to questions from the user. This makes it possible to improve the user's shopping experience and increase satisfaction.

[0546] "Means of learning the user's routine work by time and season" is a function for collecting and analyzing the user's daily life patterns and habits by time and season.

[0547] "A means of suggesting actions to the user based on learned routine work" is a function for suggesting optimal actions to the user based on collected user routine work data.

[0548] "Means of communicating the proposed content to the user" refers to a function for notifying the user of proposed actions or information.

[0549] "Means of learning about a user's past purchase history and behavioral patterns" refers to a function for collecting and analyzing a user's past purchase history and behavioral history.

[0550] "Means for suggesting optimal products according to the current season and time of day" refers to a function for selecting and suggesting optimal products to users based on the current season and time of day.

[0551] "Means of notifying users of in-store sales and new product information in real time" is a function for notifying users of sales and new product information in physical stores in real time.

[0552] "Means for providing answers to questions from users" is a function for providing appropriate answers to questions asked by users.

[0553] The system for carrying out this invention learns the user's routine work by time and season, suggests actions to the user based on the learned routine work, and communicates the contents of the suggestions to the user.Furthermore, it includes a function to learn the user's past purchase history and behavioral patterns, suggest optimal products according to the current season and time of day, and notify the user of in-store special sale information and new product information in real time.

[0554] System Configuration

[0555] Hardware

[0556] Server: A central processing unit for collecting data, analyzing it, and generating recommendations.

[0557] Device: The device used by the user, such as a smartphone or smart glasses.

[0558] Network: Infrastructure for data communication between servers and terminals.

[0559] software

[0560] AI engine: An algorithm that learns users' routines and purchase history to generate action suggestions.

[0561] Database: Storage for storing user behavior patterns, purchase history, and product information.

[0562] Notification system: Software that notifies users of offers, special offers, and new product information in real time.

[0563] Data processing and calculation

[0564] The server collects user behavior data and purchase history, and analyzes this data using an AI engine. Specifically, it processes and calculates the following data:

[0565] 1. Data collection: User behavior data and purchase history are sent from the device to the server.

[0566] 2. Data analysis: The AI ​​engine analyzes the collected data and learns the user's routine work and behavioral patterns.

[0567] 3. Action suggestion generation: Based on the learning results, optimal action suggestions are generated for the user.

[0568] 4. Real-time notifications: Proposals, sale information, and new product information are sent to users' devices in real time.

[0569] Examples

[0570] For example, if User A has a history of purchasing a "Hanami Bento" and a "Spring Jacket" in the spring, the system will learn this and suggest "Hay Fever Prevention Goods" and "Spring Jacket" in the spring. Also, when a user asks "What's recommended today?", the system will suggest the best products according to the current season and time of day.

[0571] Examples of prompt statements

[0572] When a user asks "What's the recommendation for today?", the system generates a prompt like this:

[0573] Build an assistant that learns the user's purchase history and suggests the best products based on the current season. If the user purchased "Hanami Bento" and "Spring Jacket" in spring, suggest recommended products for spring.

[0574] In this way, it is possible to realize a smart shopping assistant that learns the user's behavioral patterns and makes optimal product suggestions.

[0575] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0576] Step 1:

[0577] The user uses the terminal to input behavioral data and purchase history. The terminal sends this data to the server. The input data includes information on purchased products and timestamps of actions. The server receives this data and stores it in a database.

[0578] Step 2:

[0579] The server inputs the user's behavioral data and purchase history stored in the database into the AI ​​engine. The AI ​​engine analyzes this data and learns the user's routine work and behavioral patterns. Specifically, it performs time series analysis and clustering of the data to extract the user's behavioral patterns.

[0580] Step 3:

[0581] The server generates action suggestions based on the learning results. The AI ​​engine considers the user's behavioral patterns and the current season and time of day to generate optimal action suggestions. For example, in the spring season, it suggests "hay fever prevention goods" and "spring jackets." The generated suggestions are stored in a database.

[0582] Step 4:

[0583] A user uses a terminal to input a question such as "What's recommended today?" The terminal sends the question to a server, which analyzes the question and retrieves appropriate action suggestions from a database.

[0584] Step 5:

[0585] The server transmits the obtained action suggestions to the terminal. The terminal notifies the user of the received suggestions. The notification is made by a text message or a push notification. The user can confirm and execute the suggested action or product.

[0586] Step 6:

[0587] The server collects information on special sales and new products in the store in real time and stores it in a database. This information is obtained from the store's management system and external information services.

[0588] Step 7:

[0589] The server notifies the user's terminal of the sale information and new product information collected in real time. The terminal displays the received information to the user. The user can check the sale information and new product information and consider purchasing.

[0590] In this way, it is possible to realize a smart shopping assistant that learns the user's behavioral patterns and makes optimal product suggestions.

[0591] Example 2

[0592] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0593] In modern society, users lead busy lives and are required to efficiently manage their routine work according to time and season. However, conventional systems have difficulty fully learning users' behavioral patterns and habits and providing appropriate action suggestions. In addition, there was a lack of means to quickly provide optimal suggestions when users asked specific questions. This caused users to spend a lot of time and effort on daily choices and plans, preventing them from living an efficient life.

[0594] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0595] In this invention, the server includes: means for learning the user's routine work by time and season, means for suggesting actions to the user based on the learned routine work, means for transmitting the contents of the suggestions to the user, means for mounting an artificial intelligence engine that learns the user's daily life patterns and habits and for suggesting actions to the user based on the learning results, and means for making optimal suggestions based on the learned patterns and transmitting them to the user when the user asks questions such as "What should I do today?", "What should I eat today?", "What should I wear today?", etc. This enables the user to quickly receive optimal suggestions regarding daily choices and plans, enabling them to live an efficient life.

[0596] "User" refers to any individual or entity that uses the System.

[0597] "Routine work by time or season" refers to activities or tasks that a user performs regularly at specific times or seasons.

[0598] "Means of learning" refers to methods and technologies for analyzing user behavioral patterns and habits and accumulating them as data.

[0599] "Means for making action suggestions" refers to methods and technologies for presenting optimal actions and options to users based on learned data.

[0600] "Means for communicating the suggestions to the user" refers to the methods or technologies for notifying or displaying the generated suggestions to the user.

[0601] An "artificial intelligence engine" refers to software and algorithms for machine learning and data analysis.

[0602] "Question" refers to a request for information or inquiry made by a User to the System.

[0603] "Optimal suggestions" refer to the actions or options that are deemed most appropriate based on the user's past data and current situation.

[0604] The system of the present invention learns the user's routine work by time and season, and suggests actions to the user based on the results of that learning. Specifically, it is equipped with an artificial intelligence engine that learns the user's daily life patterns and habits, and suggests actions to the user based on the results of that learning. The suggestions include options such as "menu," "what to wear today," and "procedures that need to be done at this time of year."

[0605] Hardware and Software Used

[0606] Server: Collects, stores, analyzes data, and generates proposals. Specifically, it uses a database server and an AI engine (e.g., TensorFlow or PyTorch).

[0607] Terminal: The terminal where the user inputs data, sends questions, and displays the proposals. Specifically, a smartphone or PC is used.

[0608] User: Enters daily actions and choices and receives suggestions from the system.

[0609] Data collection

[0610] Users input their daily actions and choices into the device. For example, they record what they eat, what they wear, and what their plans are. The device collects this data and sends it to the server. For example, if a user enters "Today's breakfast is bread and coffee" into a smartphone app, that data is saved on the device. The device sends the collected data to the server periodically or in response to user operations.

[0611] Training the Data

[0612] The server receives the collected data and stores it in a database. For example, a user's dietary data and clothing data are stored in the server's database. The server uses an AI engine to learn from the stored data. Specifically, the server analyzes past data and models the user's behavioral patterns and seasonal routines. The server stores the learning results in a database.

[0613] Generate action suggestions

[0614] The user asks questions such as "What should I do today?", "What should I eat today?", and "What should I wear today?". For example, the user enters "What should I eat today?" into a smartphone app. The device sends this question to the server. Specifically, the user's question is sent from the device to the server as an HTTP request. The server generates optimal suggestions based on the learning results. For example, the server may suggest "It would be a good idea to make curry today" taking into account past eating patterns, the season, and the time of day. The server stores the generated suggestions in a database.

[0615] Communication of proposals

[0616] The server sends the generated suggestions to the device. Specifically, the server sends the suggestions to the device as an HTTP response. The device then displays the suggestions to the user. For example, the suggestions are displayed as smartphone notifications or in-app messages. When the user opens the app, the app displays the message, "It would be a good idea to make curry today."

[0617] Examples and prompts

[0618] Example 1: When a user asks, "What should I eat today?"

[0619] Prompt: "What should I eat today?"

[0620] Suggestion: "You might want to make curry today. Based on historical data, you often eat curry on Mondays."

[0621] Example 2: When a user asks, "What should I wear today?"

[0622] Prompt: "What should I wear today?"

[0623] Suggestion: "The temperature is cold today, so you should wear a sweater and a coat."

[0624] Example 3: When a user asks, "What steps do I need to take at this time?"

[0625] Prompt: "What procedures need to be completed at this time?"

[0626] Suggestion: "This month is tax return season, so get ready to go to the tax office."

[0627] In this way, the system learns the user's behavioral patterns and makes optimal suggestions.

[0628] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0629] Step 1: Data collection

[0630] The user inputs daily actions and choices into the terminal, for example recording what they eat, what they wear, what they plan to do, etc.

[0631] Input: A user types "Today's breakfast is bread and coffee" into an app on their smartphone.

[0632] The terminal collects these data and transmits them to a server. Specifically, the terminal uploads the collected data to the server periodically or in response to a user's operation.

[0633] Output: The collected data is sent to the server.

[0634] Step 2: Save your data

[0635] The server stores the received data in a database. For example, the user's food data and clothing data are stored in the server's database.

[0636] Input: User behavior data sent from the device.

[0637] The server uses a database management system (e.g., MySQL or PostgreSQL) to store the data.

[0638] Output: User behavior data stored in a database.

[0639] Step 3: Train the data

[0640] The server uses an AI engine (such as TensorFlow or PyTorch) to learn from the stored data. Specifically, the server analyzes past data and models the user's behavioral patterns and seasonal routines.

[0641] Input: User behavior data stored in the database.

[0642] The server applies machine learning algorithms to analyze the data and learn, for example, that users tend to eat curry on Mondays.

[0643] Output: A behavioral pattern model as a result of learning.

[0644] Step 4: Generate action suggestions

[0645] A user asks questions such as "What should I do today?", "What should I eat today?", "What should I wear today?", etc. For example, a user types "What should I eat today?" into a smartphone app.

[0646] Input: The user question (prompt).

[0647] The terminal sends this question to the server. Specifically, the user's question is sent from the terminal to the server as an HTTP request.

[0648] The server generates optimal suggestions based on the results of learning. For example, the server might suggest, "It would be good to make curry today," taking into account past eating patterns, the season, and the time of day.

[0649] Output: Generated action suggestions.

[0650] Step 5: Communicate your proposal

[0651] The server sends the generated proposal to the terminal. Specifically, the proposal content is sent from the server to the terminal as an HTTP response.

[0652] Input: Generated action suggestions.

[0653] The device will display the suggestions to the user. For example, it will appear as a smartphone notification or an in-app message. When the user opens the app, it will say, "It would be a good idea to make curry today."

[0654] Output: The suggestions displayed to the user.

[0655] (Application example 2)

[0656] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0657] Conventional behavior suggestion systems can learn a user's routine work and make suggestions, but there is a problem in that it is difficult to collect user behavior data in real time and make more accurate suggestions. In addition, the means by which users can receive suggestions are limited, and a method that allows them to understand the suggestions visually and intuitively is required.

[0658] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0659] In this invention, the server includes a means for learning the user's routine work by time and season, a means for making action suggestions to the user based on the learned routine work, a means for transmitting the content of the suggestions to the user, a wearable device equipped with a sensor for collecting the user's behavioral data, an artificial intelligence engine for analyzing the collected data and learning the user's routine, and a wearable device including a display for generating optimal suggestions based on the user's questions and visually displaying them. This makes it possible to collect the user's behavioral data in real time and visually provide highly accurate suggestions.

[0660] "User" means an individual or organization that uses the System.

[0661] "Routine work" refers to regular tasks and activities that users perform on a daily basis.

[0662] "Means of learning" are methods and technologies for collecting and analyzing user behavior data to recognize patterns.

[0663] "Action suggestions" are content that recommends actions to the user based on the learned routine work.

[0664] "Means of communicating the suggestions" refers to the methods or techniques used to notify the user of the suggested actions.

[0665] A "wearable device with sensors" is a wearable device equipped with sensors for collecting user behavioral data.

[0666] An "artificial intelligence engine" is a machine learning model or algorithm that analyzes collected data and learns user routines.

[0667] A "wearable device including a display" is a wearable device that has the ability to visually display information to a user.

[0668] The system for carrying out this invention learns the user's routine work by time and season, makes action suggestions based on the learned routine work, and communicates the suggested content to the user. Specifically, the system is implemented with the following configuration and procedure.

[0669] System Configuration

[0670] 1. Wearable devices equipped with sensors to collect user behavior data:

[0671] Wearable devices such as smart glasses and smart watches will be used.

[0672] These devices are equipped with cameras, microphones, GPS sensors, etc. to collect user behavior data in real time.

[0673] 2. An artificial intelligence engine that analyzes collected data and learns user routines:

[0674] An AI engine located on the server analyzes the collected data.

[0675] Use machine learning models to learn user behavior patterns and habits.

[0676] 3. A wearable device that includes a display that generates and visually displays optimal recommendations based on a user's questions:

[0677] Suggestions generated by the AI ​​engine are displayed on the display of the wearable device.

[0678] When the user asks a question using voice or gestures, suggestions appear on the display.

[0679] Program processing explanation

[0680] The server receives data from wearable devices equipped with sensors to collect user behavior data. These include smart glasses and smart watches. The collected data is analyzed by an artificial intelligence engine on the server. The AI ​​engine uses machine learning models to learn the user's routine and recognize behavioral patterns.

[0681] When a user asks a question such as "What should I eat today?", the AI ​​engine generates optimal suggestions based on the learned data. These suggestions are visually displayed on the display of the wearable device. For example, if a user asks "What should I eat today?", the display may show "I recommend curry for dinner tonight."

[0682] Examples of concrete examples and prompts

[0683] Examples

[0684] User Question: "What should I eat today?"

[0685] AI suggestion: "I recommend curry for dinner tonight."

[0686] Examples of prompt statements

[0687] Make suggestions for dinner today based on user behavior data. Users are asking, "What should I eat today?"

[0688] In this way, it is possible to collect user behavior data in real time and provide highly accurate visual suggestions.

[0689] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0690] Step 1:

[0691] A user puts on a wearable device. The wearable device is equipped with a camera, microphone, GPS sensor, etc., and these sensors collect the user's behavior data in real time. The input is the user's location information, audio data, video data, etc., and the output is the collection of this data.

[0692] Step 2:

[0693] The terminal transmits the collected behavioral data to the server. The input is the behavioral data transmitted from the wearable device, and the output is the data transmitted to the server. Specifically, the device uploads data to the server at regular intervals.

[0694] Step 3:

[0695] The server analyzes the behavioral data it receives. The input is the behavioral data sent from the device, and the output is the analysis results. In concrete terms, an artificial intelligence engine on the server analyzes the data and learns the user's behavioral patterns and routines.

[0696] Step 4:

[0697] A user asks a question to a wearable device. For example, the user asks by voice, "What should I eat today?" The input is the user's voice data, and the output is the recognition result of the voice data. In concrete terms, the device uses voice recognition technology to convert the user's question into text data.

[0698] Step 5:

[0699] The server generates optimal suggestions based on the user's question. The input is the user's question and analyzed behavioral data, and the output is the content of the suggestions. In concrete terms, the server generates optimal suggestions for the user's question based on the data learned by the artificial intelligence engine.

[0700] Step 6:

[0701] The server transmits the generated proposal to the terminal. The input is the generated proposal, and the output is the proposal transmitted to the terminal. In concrete terms, the server transmits the proposal to the wearable device.

[0702] Step 7:

[0703] The terminal visually displays the proposal. The input is the proposal sent from the server, and the output is the proposal displayed on the display. The specific operation is to display the proposal on the display of the wearable device and notify the user.

[0704] In this way, it is possible to collect user behavior data in real time and provide highly accurate visual suggestions.

[0705] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0706] "Example 1"

[0707] As one embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotions of a user. This system learns the user's routine work by time and season, makes action suggestions to the user based on the learned routine work, and transmits the suggested content to the user. Furthermore, the system recognizes the user's emotions and adjusts the action suggestions according to the emotions. Specifically, the emotion engine recognizes emotions from the user's tone of voice, facial expressions, behavior patterns, etc. For example, if the user speaks in a sad tone of voice, the emotion engine recognizes that the user is sad and transmits the information to the AI ​​engine. The AI ​​engine, after recognizing that the user is sad, makes action suggestions to the user. For example, it makes a suggestion such as "How about listening to relaxing music today?" In this way, the system of the present invention can make personalized suggestions that take the user's emotions into consideration.

[0708] "Example 2"

[0709] As one embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotions of a user. This system learns the user's routine work by time and season, makes action suggestions to the user based on the learned routine work, and transmits the suggested content to the user. Furthermore, the system recognizes the user's emotions and adjusts the action suggestions according to the emotions. Specifically, the emotion engine recognizes emotions from the user's tone of voice, facial expressions, behavior patterns, etc. For example, if the user speaks in a sad tone of voice, the emotion engine recognizes that the user is sad and transmits the information to the AI ​​engine. The AI ​​engine, after recognizing that the user is sad, makes action suggestions to the user. For example, it makes a suggestion such as "How about listening to relaxing music today?" In this way, the system of the present invention can make personalized suggestions that take the user's emotions into consideration.

[0710] The process flow of each embodiment will be described below.

[0711] "Example 1"

[0712] Step 1: The system learns the user's routine work by time and season. This learning is done from the user's behavioral history, selection history, etc.

[0713] Step 2: Based on the learned routine, the system will provide the user with action suggestions, which will vary according to the user's current time and season.

[0714] Step 3: The system recognizes the user's emotions using an emotion engine that recognizes the user's emotions from the user's tone of voice, facial expressions, behavioral patterns, etc.

[0715] Step 4: Based on the emotion engine's recognized emotion, the system will adjust the action suggestions. For example, if the user is sad, the system will suggest listening to relaxing music, etc., depending on the user's emotion.

[0716] Example 1

[0717] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0718] Conventional systems can learn a user's routine work by time and season and suggest actions based on that, but they cannot make personalized suggestions that take into account the user's emotions. This makes it difficult to provide optimal suggestions based on the user's emotional state.

[0719] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0720] In this invention, the server includes a means for learning the user's routine work by time and season, a means for making action suggestions to the user based on the learned routine work, a means for communicating the suggested content to the user, a means for recognizing the user's emotion, and a means for adjusting the action suggestions based on the recognized emotion, thereby enabling personalized suggestions according to the user's emotional state.

[0721] "User's routine work by time or season" refers to actions or habits that a user repeatedly performs at specific times of the day or during specific seasons.

[0722] "Learning means" refers to technologies and algorithms used to collect and analyze user behavior data to recognize patterns.

[0723] "Means for making action suggestions" refers to technologies and algorithms for suggesting optimal actions to users based on learned user routine work.

[0724] "Means for communicating suggestions to the user" refers to an interface or device for notifying the user of the generated action suggestions.

[0725] "Means of recognizing user emotions" refers to technologies and algorithms for analyzing and recognizing emotions from a user's tone of voice, facial expressions, behavioral patterns, etc.

[0726] "Means for tailoring action suggestions based on recognized emotions" refers to technologies or algorithms for appropriately modifying the content of action suggestions based on the user's emotional state.

[0727] The present invention is a system that learns a user's routine work by time and season, suggests actions based on the learned routine work, and further recognizes the user's emotions to adjust the suggested content. A specific embodiment of this system will be described below.

[0728] System Configuration

[0729] Hardware and Software

[0730] The server is equipped with an AI engine for collecting and analyzing user behavior data. This AI engine uses machine learning algorithms to learn the user's routine work. It also has an emotion engine that recognizes emotions from the user's tone of voice, facial expressions, behavioral patterns, etc. The terminal is a device such as a smartphone or tablet, and plays a role in conveying the suggestions from the server to the user.

[0731] Data collection and learning

[0732] The server collects data about the user's daily life. For example, the user inputs information such as "Wake up at 7am," "Eat breakfast at 8am," and "Get home at 6pm" through a smartphone app. This data is input into an AI engine, which analyzes the user's behavioral patterns and models routines by time and season.

[0733] Generate action suggestions

[0734] The server generates action suggestions for the user based on the learning results. For example, when a user asks, "What should I do today?", the server uses an AI engine to generate optimal suggestions. Specifically, it makes suggestions such as, "The weather is nice today, so how about going for a walk?"

[0735] Communication of proposals

[0736] The device transmits the suggestions received from the server to the user. The user can receive the suggestions through a device such as a smartphone or tablet. For example, if the user asks "What should I eat today?" on their smartphone, the device will display "It would be good to eat a salad with lots of vegetables today."

[0737] Emotion recognition and suggestion adjustment

[0738] The server recognizes the user's emotions using an emotion engine. The emotion engine analyzes emotions from the user's tone of voice, facial expressions, behavioral patterns, etc. For example, if the user speaks in a sad tone of voice, the emotion engine recognizes the user as sad and transmits that information to the AI ​​engine. The server adjusts behavior suggestions based on the emotion information received from the emotion engine. For example, if the user is recognized as sad, the server makes a suggestion such as, "How about listening to some relaxing music today?"

[0739] Examples

[0740] Example 1: User questions and suggestions

[0741] When a user asks, "What should I eat today?", the server takes into account past data and the current season and suggests, "How about eating a salad with lots of vegetables today?" The device displays this suggestion to the user.

[0742] Example 2: Sentiment-based suggestions

[0743] When a user asks "What should I do today?" in a sad tone of voice, the emotion engine recognizes that the user is sad, and the server suggests "How about listening to some relaxing music today?" The device displays this suggestion to the user.

[0744] Examples of prompt statements

[0745] "What should I do today?"

[0746] "What should I eat today?"

[0747] "What should I wear today?"

[0748] In this way, the system of the present invention can make personalized suggestions that take into account the user's routine and emotions.

[0749] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0750] Step 1:

[0751] The server collects data about the user's daily life. The user inputs information such as "wake up at 7am", "have breakfast at 8am", and "return home at 6pm" through a smartphone app. This data is sent to the server and stored in a database. The input data is the user's behavioral patterns, and the output is the stored behavioral data.

[0752] Step 2:

[0753] The server inputs the collected data into the AI ​​engine, which learns the user's routine work. The AI ​​engine uses machine learning algorithms to analyze the user's behavioral patterns and model routines by time and season. For example, the AI ​​engine learns a pattern that "the user wakes up at 7am on weekdays and 8am on weekends." The input data is the stored behavioral data, and the output is a model of the learned routine work.

[0754] Step 3:

[0755] The server generates action suggestions for the user based on the learning results. When the user asks, "What should I do today?", the server uses an AI engine to generate optimal suggestions. For example, it may suggest, "The weather is nice today, so how about going for a walk?" The input data is the user's question and the learned routine work model, and the output is the generated action suggestions.

[0756] Step 4:

[0757] The terminal transmits the suggestions received from the server to the user. The user can receive the suggestions through a terminal such as a smartphone or tablet. For example, if the user asks on their smartphone, "What should I eat today?", the terminal will display, "It would be good to eat a salad with lots of vegetables today." The input data is the suggestions from the server, and the output is the suggestions that are displayed to the user.

[0758] Step 5:

[0759] The server uses an emotion engine to recognize the user's emotions. The emotion engine analyzes emotions from the user's tone of voice, facial expressions, behavioral patterns, etc. For example, if a user speaks in a sad tone of voice, the emotion engine recognizes that the user is sad and transmits that information to the AI ​​engine. The input data are the user's tone of voice, facial expressions, and behavioral patterns, and the output is the recognized emotion information.

[0760] Step 6:

[0761] The server adjusts the behavioral suggestions based on the emotion information received from the emotion engine. For example, if the user is recognized as sad, the server will suggest, "How about listening to some relaxing music today?" The input data are the recognized emotion information and the learned routine work model, and the output is the adjusted behavioral suggestions.

[0762] (Application example 1)

[0763] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0764] Conventional behavior suggestion systems can make suggestions based on the user's routine work, but they cannot make suggestions that take into account the user's emotions and shopping patterns, so personalized suggestions are lacking. In addition, there are limited means to effectively communicate the contents of the suggestions to the user, which means that the system cannot fully improve user convenience.

[0765] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0766] In this invention, the server includes a means for learning a user's routine work by time and season, a means for making an action suggestion to the user based on the learned routine work, a means for transmitting the content of the suggestion to the user, a means for recognizing the user's emotion, a means for adjusting the action suggestion based on the recognized emotion, a means for learning the user's shopping pattern, a means for making a product suggestion based on the learned shopping pattern, and a means for transmitting the content of the suggestion through a smartphone notification or a voice assistant. This enables personalized suggestions that take into account the user's emotion and shopping pattern, thereby improving user convenience.

[0767] "Means of learning the user's routine work by time and season" is a function for recording and analyzing the user's daily life patterns and habits according to time and season.

[0768] "Means of suggesting actions to the user based on learned routine work" is a function for suggesting optimal actions to the user based on recorded routine work data.

[0769] "Means of communicating the content of the suggestions to the user" refers to a function for notifying the user of the content of the action suggestions, and is carried out through smartphone notifications or voice assistants.

[0770] "Means of recognizing the user's emotions" refers to a function for analyzing and recognizing emotions from the user's tone of voice, facial expressions, behavioral patterns, etc.

[0771] The "means for adjusting action suggestions based on the recognized emotions" is a function for appropriately changing the content of action suggestions according to the user's emotional state.

[0772] "Means of learning user shopping patterns" refers to a function for recording and analyzing a user's past purchasing history and behavior.

[0773] The "means for making product suggestions based on learned shopping patterns" is a function for suggesting optimal products to users based on recorded shopping pattern data.

[0774] "Means of communicating the content of suggestions through smartphone notifications or voice assistants" refers to a function for communicating the content of action suggestions or product suggestions to users using the notification function of a smartphone or a voice assistant.

[0775] The system for implementing this invention learns the user's routine work by time and season, makes action suggestions based on the learned routine work, and communicates the suggested content to the user. It also has a function of recognizing the user's emotions and adjusting the action suggestions based on the emotions. Furthermore, it learns the user's shopping patterns and makes product suggestions based on the learned shopping patterns.

[0776] Hardware and Software Used

[0777] Hardware:

[0778] Smartphone (iOS or Android)

[0779] software:

[0780] Python

[0781] TextBlob (a library for sentiment analysis)

[0782] Data processing and calculation

[0783] server:

[0784] The server records and analyzes the user's daily life patterns and habits to learn the user's time and seasonal routines, which includes storing the activities performed by the user and the timestamps of those activities in a database.

[0785] Device:

[0786] The device (smartphone) receives input from the user and recognizes emotions based on the input. For emotion recognition, the TextBlob library is used to analyze the user's input text and classify it as positive, negative, or neutral.

[0787] User:

[0788] The user asks questions such as "What should I do today?", "What should I eat today?", "What should I wear today?", "What are today's recommended products?", etc. through their smartphone. In response, the server generates optimal suggestions based on the learned routine work, shopping patterns, and recognized emotions, and conveys them to the user through smartphone notifications or voice assistants.

[0789] Examples

[0790] For example, if a user types "I'm a little tired today," the device analyzes this input using the TextBlob library and recognizes it as a negative emotion. The server receives this emotion information and, taking into account the user's past routine work and shopping patterns, generates suggestions such as "I recommend products that will help you relax today." These suggestions are communicated to the user through smartphone notifications and voice assistants.

[0791] Examples of prompt statements

[0792] Examples of prompts for a generative AI model might include the following:

[0793] Develop a system that recognizes the user's emotions and suggests actions based on those emotions. If a user types, "I'm a bit tired today," suggest products that will help them relax.

[0794] In this way, personalized suggestions that take into account the user's emotions and shopping patterns are possible, increasing user convenience.

[0795] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0796] Step 1:

[0797] A user types "I'm a little tired today" through their smartphone. The typed text is sent to the terminal.

[0798] Step 2:

[0799] The device receives the input text and performs sentiment analysis using the TextBlob library, specifically classifying the text as positive, negative, or neutral. The input is the user's text, and the output is the sentiment classification result.

[0800] Step 3:

[0801] The terminal transmits the result of emotion analysis to the server. The input is the emotion classification result, and the output is the data transmission to the server.

[0802] Step 4:

[0803] The server receives the emotion analysis results and retrieves the user's past routine work and shopping patterns from the database. The input is the emotion classification result, and the output is the data of the user's past routine work and shopping patterns.

[0804] Step 5:

[0805] The server generates optimal action suggestions for the user based on the acquired data. For example, if the user is showing negative emotions, it will suggest products that will help them relax. The input is data on past routine work and shopping patterns and the results of emotion classification, and the output is action suggestions.

[0806] Step 6:

[0807] The server transmits the generated action proposal to the terminal. The input is the action proposal, and the output is data transmission to the terminal.

[0808] Step 7:

[0809] The device notifies the user of the action suggestions it receives. Specifically, it communicates the content of the suggestions using the notification function of the smartphone or a voice assistant. The input is the action suggestions, and the output is a notification to the user.

[0810] In this way, personalized suggestions are made based on the user's emotions and past behavioral patterns.

[0811] Example 2

[0812] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0813] Conventional systems suggested actions based on the user's routine work, but did not take the user's emotions into account. As a result, appropriate suggestions could not be made according to the user's emotional state, making it difficult to improve user satisfaction. In addition, responses to questions from users were uniform, and personalized suggestions tailored to the individual user's situation were lacking.

[0814] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0815] In this invention, the server includes a means for learning the user's routine work by time and season, a means for making an action suggestion to the user based on the learned routine work, a means for communicating the suggestion content to the user, a means for recognizing the user's emotion, and a means for adjusting the action suggestion based on the recognized emotion, thereby enabling personalized suggestions that take into account the user's routine work and emotion.

[0816] A "means for learning a user's time and seasonal routine work" is a device or program for collecting and analyzing a user's daily life patterns and habits.

[0817] The "means for suggesting actions to the user based on learned routine work" refers to a device or program for suggesting optimal actions to the user based on collected and analyzed data.

[0818] The "means for communicating the content of the suggestion to the user" refers to a device or program for notifying the user of the generated action suggestion.

[0819] The "means for recognizing a user's emotions" refers to a device or program for detecting and recognizing emotions from a user's tone of voice, facial expressions, behavioral patterns, etc.

[0820] The "means for adjusting action suggestions based on the recognized emotion" refers to a device or program for modifying the content of action suggestions in accordance with the user's emotional state and making optimal suggestions.

[0821] The present invention is a system that learns a user's routine work by time and season, suggests actions based on the learned routine work, and further recognizes the user's emotions to adjust the suggested content. A specific embodiment of this system will be described below.

[0822] System Configuration

[0823] Hardware

[0824] Server: A central processing unit for data collection, analysis, proposal generation, and emotion recognition.

[0825] Device: The device a user uses, such as a smartphone or tablet.

[0826] Sensors: Microphones and cameras to recognize the user's tone of voice and facial expressions.

[0827] software

[0828] AI engine: Software that learns a user's routine work and generates action suggestions.

[0829] Emotion engine: Software for recognizing user emotions and communicating that information to the AI ​​engine.

[0830] Data collection and learning

[0831] The server collects user behavior data, including what time the user wakes up, what meals they eat, what clothes they wear, etc. Specifically, it collects data from smartphones and wearable devices.

[0832] Example: Information entered by a user into a smartphone app or activity data obtained from a wearable device is sent to a server.

[0833] Generate action suggestions

[0834] The server uses an AI engine to learn the user's routine work based on the collected data, and the AI ​​engine analyzes time and seasonal patterns to generate a model that predicts user behavior.

[0835] Example: A server analyzes historical data to determine whether users tend to behave in certain ways at certain times.

[0836] Communicating the proposal

[0837] The server transmits the generated suggestions to the user. When the user inputs a question via a device (such as a smartphone or tablet), the server responds with the best suggestions for that question.

[0838] Example: A user types a question into a smartphone app and receives suggestions from the server.

[0839] Emotion recognition and suggestion adjustment

[0840] The server uses an emotion engine to recognize the user's emotion, which is determined from the user's tone of voice, facial expression, behavioral patterns, etc.

[0841] Example: Emotions are recognized by analyzing the tone of voice when a user speaks through a smartphone microphone and facial expressions captured through the camera.

[0842] The server then uses the information obtained from the emotion engine to suggest actions based on the user's emotions. For example, if the server detects that the user is sad, it will suggest, "How about listening to some relaxing music today?"

[0843] Examples of prompt statements

[0844] "What should I do today?"

[0845] "What should I eat today?"

[0846] "What should I wear today?"

[0847] In this way, the inventive system can make personalized suggestions that take into account the user's routine and emotions.

[0848] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0849] Step 1:

[0850] User data collection

[0851] The server collects user behavior data. As input, it receives data provided by the user from a smartphone or wearable device. This data includes the user's wake-up time, what they ate, what they were wearing, etc. The server stores this data in a database for later analysis.

[0852] Specific operation: Information entered by the user into the smartphone app and activity data acquired from the wearable device are sent to the server, which receives the data and stores it in a database.

[0853] Step 2:

[0854] Learning routine work

[0855] The server uses the AI ​​engine to learn the user's routine based on the collected data. As input, it uses past behavioral data stored in a database. The AI ​​engine analyzes these data and finds patterns over time and seasons. As output, it generates a model that predicts the user's behavior.

[0856] How it works: The server analyzes past data and finds out if a user tends to behave in a certain way at certain times. For example, it learns that a user wakes up at 7am every morning and eats breakfast at 8am.

[0857] Step 3:

[0858] Generate action suggestions

[0859] The server generates action suggestions for the user based on the learned routine work. As input, it uses the prediction model generated by the AI ​​engine and questions from the user. As output, it generates optimal action suggestions and provides them to the user.

[0860] What happens: If a user asks, "What should I eat today?" the server uses past data to suggest, "How about a salad with lots of vegetables today?"

[0861] Step 4:

[0862] Communicating the proposal

[0863] The server communicates the generated suggestions to the user, uses the generated action suggestions as input, and sends the suggestions to the user's device as output.

[0864] How it works: A user enters a question into a smartphone app and receives suggestions from the server, which then sends the suggestions to the user's device, where the user can review them.

[0865] Step 5:

[0866] Emotion recognition

[0867] The server uses an emotion engine to recognize the user's emotions. As input, it uses the user's tone of voice, facial expressions, and behavioral patterns. As output, it recognizes the user's emotional state and conveys that information to the AI ​​engine.

[0868] Specific operation: The system recognizes emotions by analyzing the tone of voice the user speaks through the smartphone microphone and facial expressions captured through the camera. For example, if the user speaks in a sad tone, the emotion engine transmits that information to the AI ​​engine.

[0869] Step 6:

[0870] Tailoring behavior suggestions based on emotions

[0871] The server proposes actions based on the user's emotions based on the information obtained from the emotion engine. As input, it uses the emotion information from the emotion engine and the prediction model of the AI ​​engine. As output, it generates optimal action proposals based on emotions and provides them to the user.

[0872] Specific behavior: If the user is recognized as sad, the server will make a suggestion such as "How about listening to some relaxing music today?" The server will send this suggestion to the user's device, and the user will confirm it.

[0873] (Application example 2)

[0874] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0875] Conventional systems suggested actions based on the user's time and seasonal routines, but they lacked personalized suggestions that took into account the user's emotions and purchasing history. In addition, they did not offer real-time product suggestions to improve the shopping experience in physical stores. This led to problems such as reduced user satisfaction and convenience.

[0876] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0877] In this invention, the server includes a means for learning a user's routine work by time and season, a means for making an action suggestion to the user based on the learned routine work, a means for transmitting the suggestion content to the user, a means for recognizing the user's emotion, a means for adjusting the action suggestion based on the recognized emotion, a means for learning the user's purchase history, a means for making a product suggestion based on the learned purchase history, and a means for displaying the suggestion content to the user in real time. This enables personalized suggestions that take into account the user's emotion and purchase history, thereby improving the shopping experience at a physical store.

[0878] "User's routine work by time or season" refers to actions or habits that a user repeatedly performs at specific times of the day or during specific seasons.

[0879] "Means of learning" refers to the technologies and algorithms used to collect and analyze user behavioral patterns and habits as data.

[0880] "Means for making action suggestions" refers to technologies or algorithms that present optimal actions and options to users based on learned data.

[0881] "Means of communicating suggestions to users" refers to any interface or device used to inform users of suggested actions or options.

[0882] "Means of recognizing user emotions" refers to technologies and algorithms for analyzing and recognizing emotions from a user's tone of voice, facial expressions, behavioral patterns, etc.

[0883] "Means for tailoring action suggestions based on recognized emotions" means techniques or algorithms for modifying the content of action suggestions in response to the user's emotional state.

[0884] "Means of learning a user's purchasing history" refers to technologies and algorithms for collecting and analyzing data on products a user has previously purchased.

[0885] "Means for making product suggestions based on learned purchasing history" refers to technologies and algorithms for suggesting optimal products based on a user's purchasing history.

[0886] "Means for displaying suggestions to the user in real time" refers to an interface or device for instantly informing the user of suggested products or actions.

[0887] The system for implementing this invention learns the user's routine work by time and season, makes action suggestions based on the learned routine work, and communicates the suggested content to the user. It also recognizes the user's emotions and adjusts the action suggestions based on the recognized emotions. It also learns the user's purchase history, makes product suggestions based on the learned purchase history, and displays the suggested content to the user in real time.

[0888] Hardware and Software Used

[0889] Hardware: smart glasses, camera, microphone, server

[0890] Software: Emotion recognition model (OpenCV), clustering algorithm (KMeans), generative AI model

[0891] Data processing and calculation

[0892] 1. Learn user's time and seasonal routines:

[0893] The server collects data on users' behavioral patterns and habits and analyzes them using a clustering algorithm (KMeans), thereby learning the user's routine work.

[0894] 2. Generate action suggestions:

[0895] Based on the learned data, generative AI models are used to suggest optimal actions and options to users, and these suggestions are provided as answers to the user's questions.

[0896] 3. Emotion recognition:

[0897] The camera and microphone installed in the smart glasses are used to collect the user's facial expressions and tone of voice, which are then analyzed using an emotion recognition model (OpenCV), allowing the system to recognize the user's emotions.

[0898] 4. Tailoring behavioral suggestions based on emotions:

[0899] Based on the emotions recognized, the generative AI model will tailor action suggestions, for example making suggestions to help users relax if they are sad.

[0900] 5. Learning purchase history:

[0901] The server collects the user's past purchase history and analyzes it using a clustering algorithm (KMeans), thereby learning the user's purchase history.

[0902] 6. Generate product proposals:

[0903] Based on the learned purchasing history, a generative AI model is used to suggest optimal products, which are then displayed in real time on the smart glasses' display.

[0904] Examples

[0905] User's purchasing history: In the past, they have purchased T-shirts and sunglasses.

[0906] Current season: Summer.

[0907] User emotion: "happy" recognized from camera footage.

[0908] Examples of prompt statements

[0909] "Learn your users' purchasing history and suggest the best products based on the current season and sentiment."

[0910] "Please suggest products that users are likely to purchase in the summer."

[0911] "Suggest products that users are likely to purchase when they're in a happy mood."

[0912] In this way, personalized suggestions can be made that take into account the user's emotions and purchasing history, improving the in-store shopping experience.

[0913] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0914] Step 1:

[0915] The server learns the user's routine work by time and season. Specifically, it collects the user's behavioral patterns and habits as data and analyzes them using a clustering algorithm (KMeans). The input is the user's behavioral data, and the output is the user's routine work pattern.

[0916] Step 2:

[0917] The server makes action suggestions to the user based on the learned routine work. Using a generative AI model, it suggests optimal actions and options to the user. The input required is the routine work pattern obtained in step 1, and the output is the action suggestions for the user.

[0918] Step 3:

[0919] The terminal communicates the proposed content to the user. Specifically, it displays the proposed content on the display of the smart glasses. As input, the proposed action obtained in step 2 is required, and as output, the user can visually confirm the proposed content.

[0920] Step 4:

[0921] The device recognizes the user's emotions. It uses the camera and microphone installed in the smart glasses to collect the user's facial expressions and tone of voice, and analyzes them using an emotion recognition model (OpenCV). The input requires the user's facial expressions and tone of voice data, and the output is the user's emotional state.

[0922] Step 5:

[0923] The server adjusts the action suggestions based on the recognized emotions. Using a generative AI model, it generates action suggestions according to the user's emotional state. As input, it requires the emotional state data obtained in step 4, and as output, it obtains the adjusted action suggestions.

[0924] Step 6:

[0925] The server learns the user's purchase history. It collects the user's past purchase history and analyzes it using a clustering algorithm (KMeans). As input, it requires the user's purchase history data, and as output, it obtains the user's purchase pattern.

[0926] Step 7:

[0927] The server makes product suggestions based on the learned purchasing history. It uses a generative AI model to suggest the best products for the user. As input, it requires the purchasing patterns obtained in step 6, and as output, it obtains product suggestions for the user.

[0928] Step 8:

[0929] The terminal then displays the,suggested content to the user in real time.,The suggested products and actions are instantly displayed,on the display of the smart glasses.,As input, the product suggestions obtained in,step 7 are required, and as output, the user can,check the suggested content in real time.

[0930] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0931] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0932] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0933] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0934] [Third embodiment]

[0935] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0936] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0937] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0938] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0939] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0940] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0941] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0942] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0943] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0944] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0945] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0946] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0947] "Example 1"

[0948] The system of the present invention includes a means for learning the user's routine work by time and season, a means for suggesting actions to the user based on the learned routine work, and a means for communicating the suggested content to the user. Specifically, the system is equipped with an AI engine that learns the user's daily life patterns and habits, and makes action suggestions to the user based on the learning results. The suggested content includes options such as "menu," "clothes to wear today," and "procedures that need to be done at this time of year." These suggestions are made based on the learned patterns and communicated to the user when the user asks questions such as "What should I do today?", "What should I eat today?", and "What should I wear today?".

[0949] "Example 2"

[0950] The system of the present invention includes a means for learning the user's routine work by time and season, a means for suggesting actions to the user based on the learned routine work, and a means for communicating the suggested content to the user. Specifically, the system is equipped with an AI engine that learns the user's daily life patterns and habits, and makes action suggestions to the user based on the learning results. The suggested content includes options such as "menu," "clothes to wear today," and "procedures that need to be done at this time of year." These suggestions are made based on the learned patterns and communicated to the user when the user asks questions such as "What should I do today?", "What should I eat today?", and "What should I wear today?".

[0951] The process flow of each embodiment will be described below.

[0952] "Example 1"

[0953] Step 1: Learn the user's daily patterns and habits The AI ​​engine observes the user's behavior and collects data. For example, it learns that the user wakes up at 7:00 every morning and has breakfast at 7:30.

[0954] Step 2: Based on the collected data, the AI ​​engine learns the user's routine work using machine learning techniques.

[0955] Step 3: Based on the learned routine work, the AI ​​engine suggests actions to the user. For example, when the user asks, "What should I do today?", the AI ​​engine suggests, "Wake up at 7:00 and have breakfast at 7:30."

[0956] Step 4: The proposal is communicated to the user. This is done through the notification function of the device (e.g. smartphone, PC) used by the user, or through a voice assistant.

[0957] Example 1

[0958] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0959] In today's busy lifestyles, it is difficult for users to efficiently manage their daily routines and seasonal activities. In particular, choosing daily meals, clothing, necessary procedures, etc. often takes time, which can be a source of stress. In addition, in order to optimize these choices, it is necessary to take into account the user's past behavioral data and current situation, but this is very difficult to do manually.

[0960] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0961] In this invention, the server includes a means for learning a user's routine work by time and season, a means for making action suggestions to the user based on the learned routine work, a means for transmitting the content of the suggestions to the user, a means for collecting user action data, a means for preprocessing the collected data, a means for training an AI engine using the preprocessed data, a means for accepting questions from the user, a means for generating optimal suggestions for the questions, and a means for transmitting the content of the suggestions to the user. This enables the user to efficiently manage daily routine work and seasonal actions and receive optimal suggestions.

[0962] "Means of learning the user's routine work by time and season" refers to technology that collects the user's daily life patterns and habits and uses this to predict the user's behavior.

[0963] The "means of suggesting actions to the user based on learned routine work" refers to technology that analyzes collected data and suggests optimal actions based on the user's past behavioral patterns.

[0964] The "means for communicating the content of the proposal to the user" refers to an interface or communication means for notifying the user of the generated proposal.

[0965] "Means of collecting user behavioral data" refers to technology for obtaining user location information, calendar information, past behavioral history, etc.

[0966] "Means of preprocessing collected data" refers to techniques for filling in missing values, removing outliers, converting data formats, etc. in collected data.

[0967] "Means for training an AI engine using preprocessed data" refers to a technology that uses preprocessed data as input to train an AI model and have it learn user behavior patterns.

[0968] The "means for accepting questions from the user" is an interface that enables the user to input questions to the system.

[0969] "Means for generating optimal suggestions in response to questions" refers to technology that generates optimal answers to questions from users based on a learned model.

[0970] The "means for communicating the content of the proposal to the user" refers to an interface or communication means for notifying the user of the generated proposal.

[0971] This invention relates to a system that learns a user's routine work by time and season and suggests optimal actions. This system includes a series of processes that collect user behavior data, learns using an AI engine, generates optimal suggestions in response to questions from the user, and communicates the suggested content to the user.

[0972] To collect user behavior data, the server obtains location information, calendar information, past behavior history, etc. from smartphones and wearable devices. For example, when a user enters an event such as "going to the gym" into the smartphone calendar, that information is sent to the server.

[0973] Next, the server preprocesses the collected data. Specifically, it complements missing values ​​in the data and removes outliers. It also converts the data into a format that is easy for the AI ​​engine to learn from. For example, it converts location information from latitude and longitude to an address.

[0974] Using the preprocessed data, the server trains the AI ​​engine. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to model the user's behavioral patterns. For example, if a user has a habit of going to the gym every Monday, it will learn that pattern.

[0975] The terminal provides an interface that accepts questions from the user. When the user inputs a question such as "What should I do today?", "What should I eat today?", or "What should I wear today?", the terminal sends the question to the server.

[0976] The server generates optimal suggestions for the received questions from the patterns it has learned. Specifically, it makes appropriate suggestions by taking into account the user's past behavioral history and current situation (weather, time of day, etc.). For example, if a user asks, "What should I eat today?" the server will suggest the optimal menu based on the user's past eating history, the current season, time of day, etc.

[0977] The device conveys the suggestions received from the server to the user. Specifically, it displays them in the form of text messages, voice messages, etc. For example, it displays a suggestion such as "Today is sunny and the temperature is 25 degrees, so it would be good to wear a light jacket and jeans."

[0978] Examples of prompts include:

[0979] "What should I do today?"

[0980] "What should I eat today?"

[0981] "What should I wear today?"

[0982] This enables users to efficiently manage their daily routine work and seasonal activities and receive optimal suggestions.

[0983] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0984] Step 1:

[0985] Collecting user behavior data

[0986] The server collects location information, calendar information, and past activity history from the user's smartphone or wearable device. For example, when a user enters an event such as "going to the gym" into the smartphone calendar, the information is sent to the server. The input data includes location information, calendar information, and activity history, which are then stored in a database.

[0987] Step 2:

[0988] Data Preprocessing

[0989] The server preprocesses the collected data. Specifically, it complements missing values ​​in the data and removes outliers. It also converts the data into a format that is easy for the AI ​​engine to learn from. For example, it converts location information from latitude and longitude to an address. The input data is the raw data collected in step 1, and the output data is the preprocessed, clean data.

[0990] Step 3:

[0991] AI engine learning

[0992] The server trains the AI ​​engine using the preprocessed data. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to model the user's behavioral patterns. For example, if a user has a habit of going to the gym every Monday, that pattern is learned. The input data is the preprocessed data, and the output data is the trained AI model.

[0993] Step 4:

[0994] Accepting questions from users

[0995] The terminal provides an interface that accepts questions from the user. When the user inputs a question such as "What should I do today?", "What should I eat today?", or "What should I wear today?", the terminal sends the question to the server. The input data is the user's question, and the output data is a request to the server.

[0996] Step 5:

[0997] Generate best-fit suggestions for your questions

[0998] The server generates optimal suggestions for the received questions from the patterns it has learned. Specifically, it makes appropriate suggestions by taking into account the user's past behavioral history and current situation (weather, time of day, etc.). For example, if a user asks, "What should I eat today?", the server will suggest the optimal menu based on the user's past meal history, the current season, time of day, etc. The input data is the user's question and the trained AI model, and the output data is the suggestions.

[0999] Step 6:

[1000] Communicating the proposal to users

[1001] The terminal conveys the suggestions received from the server to the user. Specifically, it displays them in the form of text messages, voice messages, etc. For example, it displays a suggestion such as "Today is sunny with a temperature of 25 degrees, so it would be good to wear a light jacket and jeans." The input data is the suggestion from the server, and the output data is a notification to the user.

[1002] (Application example 1)

[1003] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1004] Conventional systems can learn a user's daily life patterns and habits and make action suggestions, but they lack specific product suggestions and real-time notification functions to improve the shopping experience in physical stores. In addition, optimal product suggestions are not made taking into account the user's past purchase history and behavioral patterns, which may reduce user satisfaction. Furthermore, the system lacks the ability to provide appropriate answers to user questions.

[1005] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for learning the user's routine work by time and season, a means for suggesting actions to the user based on the learned routine work, a means for communicating the content of the suggestions to the user, a means for learning the user's past purchase history and behavioral patterns, a means for suggesting optimal products according to the current season and time period, a means for notifying the user of in-store special sale information and new product information in real time, and a means for providing answers to questions from the user. This makes it possible to improve the user's shopping experience and increase satisfaction.

[1006] "Means of learning the user's routine work by time and season" is a function for collecting and analyzing the user's daily life patterns and habits by time and season.

[1007] "A means of suggesting actions to the user based on learned routine work" is a function for suggesting optimal actions to the user based on collected user routine work data.

[1008] "Means of communicating the proposed content to the user" refers to a function for notifying the user of proposed actions or information.

[1009] "Means of learning about a user's past purchase history and behavioral patterns" refers to a function for collecting and analyzing a user's past purchase history and behavioral history.

[1010] "Means for suggesting optimal products according to the current season and time of day" refers to a function for selecting and suggesting optimal products to users based on the current season and time of day.

[1011] "Means of notifying users of in-store sales and new product information in real time" is a function for notifying users of sales and new product information in physical stores in real time.

[1012] "Means for providing answers to questions from users" is a function for providing appropriate answers to questions asked by users.

[1013] The system for carrying out this invention learns the user's routine work by time and season, suggests actions to the user based on the learned routine work, and communicates the contents of the suggestions to the user.Furthermore, it includes a function to learn the user's past purchase history and behavioral patterns, suggest optimal products according to the current season and time of day, and notify the user of in-store special sale information and new product information in real time.

[1014] System Configuration

[1015] Hardware

[1016] Server: A central processing unit for collecting data, analyzing it, and generating recommendations.

[1017] Device: The device used by a user, such as a smartphone or smart glasses.

[1018] Network: Infrastructure for data communication between servers and terminals.

[1019] software

[1020] AI engine: An algorithm that learns users' routines and purchase history to generate action suggestions.

[1021] Database: Storage for storing user behavior patterns, purchase history, and product information.

[1022] Notification system: Software that notifies users of offers, special offers, and new product information in real time.

[1023] Data processing and calculation

[1024] The server collects user behavior data and purchase history, and analyzes this data using an AI engine. Specifically, it processes and calculates the following data:

[1025] 1. Data collection: User behavior data and purchase history are sent from the device to the server.

[1026] 2. Data analysis: The AI ​​engine analyzes the collected data and learns the user's routine work and behavioral patterns.

[1027] 3. Action suggestion generation: Based on the learning results, optimal action suggestions are generated for the user.

[1028] 4. Real-time notifications: Proposals, sale information, and new product information are sent to users' devices in real time.

[1029] Examples

[1030] For example, if User A has a history of purchasing a "Hanami Bento" and a "Spring Jacket" in the spring, the system will learn this and suggest "Hay Fever Prevention Goods" and "Spring Jacket" in the spring. Also, when a user asks "What's recommended today?", the system will suggest the best products according to the current season and time of day.

[1031] Examples of prompt statements

[1032] When a user asks "What's the recommendation for today?", the system generates a prompt like this:

[1033] Build an assistant that learns the user's purchase history and suggests the best products based on the current season. If the user purchased "Hanami Bento" and "Spring Jacket" in spring, suggest recommended products for spring.

[1034] In this way, it is possible to realize a smart shopping assistant that learns the user's behavioral patterns and makes optimal product suggestions.

[1035] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1036] Step 1:

[1037] The user uses the terminal to input behavioral data and purchase history. The terminal sends this data to the server. The input data includes information on purchased products and timestamps of actions. The server receives this data and stores it in a database.

[1038] Step 2:

[1039] The server inputs the user's behavioral data and purchase history stored in the database into the AI ​​engine. The AI ​​engine analyzes this data and learns the user's routine work and behavioral patterns. Specifically, it performs time series analysis and clustering of the data to extract the user's behavioral patterns.

[1040] Step 3:

[1041] The server generates action suggestions based on the learning results. The AI ​​engine considers the user's behavioral patterns and the current season and time of day to generate optimal action suggestions. For example, in the spring season, it suggests "hay fever prevention goods" and "spring jackets." The generated suggestions are stored in a database.

[1042] Step 4:

[1043] A user uses a terminal to input a question such as "What's recommended today?" The terminal sends the question to a server, which analyzes the question and retrieves appropriate action suggestions from a database.

[1044] Step 5:

[1045] The server transmits the obtained action suggestions to the terminal. The terminal notifies the user of the received suggestions. The notification is made by a text message or a push notification. The user can confirm and execute the suggested action or product.

[1046] Step 6:

[1047] The server collects information on special sales and new products in the store in real time and stores it in a database. This information is obtained from the store's management system and external information services.

[1048] Step 7:

[1049] The server notifies the user's terminal of the sale information and new product information collected in real time. The terminal displays the received information to the user. The user can check the sale information and new product information and consider purchasing.

[1050] In this way, it is possible to realize a smart shopping assistant that learns the user's behavioral patterns and makes optimal product suggestions.

[1051] Example 2

[1052] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1053] In modern society, users lead busy lives and are required to efficiently manage their routine work according to time and season. However, conventional systems have difficulty fully learning users' behavioral patterns and habits and providing appropriate action suggestions. In addition, there was a lack of means to quickly provide optimal suggestions when users asked specific questions. This caused users to spend a lot of time and effort on daily choices and plans, preventing them from living an efficient life.

[1054] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1055] In this invention, the server includes: means for learning the user's routine work by time and season, means for suggesting actions to the user based on the learned routine work, means for transmitting the contents of the suggestions to the user, means for mounting an artificial intelligence engine that learns the user's daily life patterns and habits and for suggesting actions to the user based on the learning results, and means for making optimal suggestions based on the learned patterns and transmitting them to the user when the user asks questions such as "What should I do today?", "What should I eat today?", "What should I wear today?", etc. This enables the user to quickly receive optimal suggestions regarding daily choices and plans, enabling them to live an efficient life.

[1056] "User" refers to any individual or entity that uses the System.

[1057] "Routine work by time or season" refers to activities or tasks that a user performs regularly at specific times or seasons.

[1058] "Means of learning" refers to methods and technologies for analyzing user behavioral patterns and habits and accumulating them as data.

[1059] "Means for making action suggestions" refers to methods and technologies for presenting optimal actions and options to users based on learned data.

[1060] "Means for communicating the suggestions to the user" refers to the methods or technologies for notifying or displaying the generated suggestions to the user.

[1061] An "artificial intelligence engine" refers to software and algorithms for machine learning and data analysis.

[1062] "Question" refers to a request for information or inquiry made by a User to the System.

[1063] "Optimal suggestions" refer to the actions or options that are deemed most appropriate based on the user's past data and current situation.

[1064] The system of the present invention learns the user's routine work by time and season, and suggests actions to the user based on the results of that learning. Specifically, it is equipped with an artificial intelligence engine that learns the user's daily life patterns and habits, and suggests actions to the user based on the results of that learning. The suggestions include options such as "menu," "what to wear today," and "procedures that need to be done at this time of year."

[1065] Hardware and Software Used

[1066] Server: Collects, stores, analyzes data, and generates proposals. Specifically, it uses a database server and an AI engine (e.g., TensorFlow or PyTorch).

[1067] Terminal: The terminal where the user inputs data, sends questions, and displays the proposals. Specifically, a smartphone or PC is used.

[1068] User: Enters daily actions and choices and receives suggestions from the system.

[1069] Data collection

[1070] Users input their daily actions and choices into the device. For example, they record what they eat, what they wear, and what their plans are. The device collects this data and sends it to the server. For example, if a user enters "Today's breakfast is bread and coffee" into a smartphone app, that data is saved on the device. The device sends the collected data to the server periodically or in response to user operations.

[1071] Training the Data

[1072] The server receives the collected data and stores it in a database. For example, a user's dietary data and clothing data are stored in the server's database. The server uses an AI engine to learn from the stored data. Specifically, the server analyzes past data and models the user's behavioral patterns and seasonal routines. The server stores the learning results in a database.

[1073] Generate action suggestions

[1074] The user asks questions such as "What should I do today?", "What should I eat today?", and "What should I wear today?". For example, the user enters "What should I eat today?" into a smartphone app. The device sends this question to the server. Specifically, the user's question is sent from the device to the server as an HTTP request. The server generates optimal suggestions based on the learning results. For example, the server may suggest "It would be a good idea to make curry today" taking into account past eating patterns, the season, and the time of day. The server stores the generated suggestions in a database.

[1075] Communicating the proposal

[1076] The server sends the generated suggestions to the device. Specifically, the server sends the suggestions to the device as an HTTP response. The device then displays the suggestions to the user. For example, the suggestions are displayed as notifications on the smartphone or messages within the app. When the user opens the app, the app displays the message, "It would be a good idea to make curry today."

[1077] Examples and prompts

[1078] Example 1: When a user asks, "What should I eat today?"

[1079] Prompt: "What should I eat today?"

[1080] Suggestion: "You might want to make curry today. Based on historical data, you often eat curry on Mondays."

[1081] Example 2: When a user asks, "What should I wear today?"

[1082] Prompt: "What should I wear today?"

[1083] Suggestion: "The temperature is cold today, so you should wear a sweater and a coat."

[1084] Example 3: When a user asks, "What steps do I need to take at this time?"

[1085] Prompt: "What procedures need to be completed at this time?"

[1086] Suggestion: "This month is tax return season, so get ready to go to the tax office."

[1087] In this way, the system learns the user's behavioral patterns and makes optimal suggestions.

[1088] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1089] Step 1: Data collection

[1090] The user inputs daily actions and choices into the terminal, for example recording what they eat, what they wear, what they plan to do, etc.

[1091] Input: A user types "Today's breakfast is bread and coffee" into an app on their smartphone.

[1092] The terminal collects these data and transmits them to a server. Specifically, the terminal uploads the collected data to the server periodically or in response to a user's operation.

[1093] Output: The collected data is sent to the server.

[1094] Step 2: Save your data

[1095] The server stores the received data in a database. For example, the user's food data and clothing data are stored in the server's database.

[1096] Input: User behavior data sent from the device.

[1097] The server uses a database management system (e.g., MySQL or PostgreSQL) to store the data.

[1098] Output: User behavior data stored in a database.

[1099] Step 3: Train the data

[1100] The server uses an AI engine (such as TensorFlow or PyTorch) to learn from the stored data. Specifically, the server analyzes past data and models the user's behavioral patterns and seasonal routines.

[1101] Input: User behavior data stored in the database.

[1102] The server applies machine learning algorithms to analyze the data and learn, for example, that users tend to eat curry on Mondays.

[1103] Output: A behavioral pattern model as a result of learning.

[1104] Step 4: Generate action suggestions

[1105] A user asks questions such as "What should I do today?", "What should I eat today?", "What should I wear today?", etc. For example, a user types "What should I eat today?" into a smartphone app.

[1106] Input: The user question (prompt).

[1107] The terminal sends this question to the server. Specifically, the user's question is sent from the terminal to the server as an HTTP request.

[1108] The server generates optimal suggestions based on the results of learning. For example, the server might suggest, "It would be good to make curry today," taking into account past eating patterns, the season, and the time of day.

[1109] Output: Generated action suggestions.

[1110] Step 5: Communicate your proposal

[1111] The server sends the generated proposal to the terminal. Specifically, the proposal content is sent from the server to the terminal as an HTTP response.

[1112] Input: Generated action suggestions.

[1113] The device will display the suggestions to the user. For example, it will appear as a smartphone notification or an in-app message. When the user opens the app, it will say, "It would be a good idea to make curry today."

[1114] Output: The suggestions displayed to the user.

[1115] (Application example 2)

[1116] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1117] Conventional behavior suggestion systems can learn a user's routine work and make suggestions, but there is a problem in that it is difficult to collect user behavior data in real time and make more accurate suggestions. In addition, the means by which users can receive suggestions are limited, and a method that allows them to understand the suggestions visually and intuitively is required.

[1118] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1119] In this invention, the server includes a means for learning the user's routine work by time and season, a means for making action suggestions to the user based on the learned routine work, a means for transmitting the content of the suggestions to the user, a wearable device equipped with a sensor for collecting the user's behavioral data, an artificial intelligence engine for analyzing the collected data and learning the user's routine, and a wearable device including a display for generating optimal suggestions based on the user's questions and visually displaying them. This makes it possible to collect the user's behavioral data in real time and visually provide highly accurate suggestions.

[1120] "User" means an individual or organization that uses the System.

[1121] "Routine work" refers to regular tasks and activities that users perform on a daily basis.

[1122] "Means of learning" are methods and technologies for collecting and analyzing user behavior data to recognize patterns.

[1123] "Action suggestions" are content that recommends actions to the user based on the learned routine work.

[1124] "Means of communicating the suggestions" refers to the methods or techniques used to notify the user of the suggested actions.

[1125] A "wearable device with sensors" is a wearable device equipped with sensors for collecting user behavioral data.

[1126] An "artificial intelligence engine" is a machine learning model or algorithm that analyzes collected data and learns user routines.

[1127] A "wearable device including a display" is a wearable device that has the ability to visually display information to a user.

[1128] The system for carrying out this invention learns the user's routine work by time and season, makes action suggestions based on the learned routine work, and communicates the suggested content to the user. Specifically, the system is implemented with the following configuration and procedure.

[1129] System Configuration

[1130] 1. Wearable devices equipped with sensors to collect user behavior data:

[1131] Wearable devices such as smart glasses and smart watches will be used.

[1132] These devices are equipped with cameras, microphones, GPS sensors, etc. to collect user behavior data in real time.

[1133] 2. An artificial intelligence engine that analyzes collected data and learns user routines:

[1134] An AI engine located on the server analyzes the collected data.

[1135] Use machine learning models to learn user behavior patterns and habits.

[1136] 3. A wearable device that includes a display that generates and visually displays optimal recommendations based on a user's questions:

[1137] Suggestions generated by the AI ​​engine are displayed on the display of the wearable device.

[1138] When the user asks a question using voice or gestures, suggestions appear on the display.

[1139] Program processing explanation

[1140] The server receives data from wearable devices equipped with sensors to collect user behavior data. These include smart glasses and smart watches. The collected data is analyzed by an artificial intelligence engine on the server. The AI ​​engine uses machine learning models to learn the user's routine and recognize behavioral patterns.

[1141] When a user asks a question such as "What should I eat today?", the AI ​​engine generates optimal suggestions based on the learned data. These suggestions are visually displayed on the display of the wearable device. For example, if a user asks "What should I eat today?", the display may show "I recommend curry for dinner tonight."

[1142] Examples of concrete examples and prompts

[1143] Examples

[1144] User Question: "What should I eat today?"

[1145] AI suggestion: "I recommend curry for dinner tonight."

[1146] Examples of prompt statements

[1147] Make suggestions for dinner today based on user behavior data. Users are asking, "What should I eat today?"

[1148] In this way, it is possible to collect user behavior data in real time and provide highly accurate visual suggestions.

[1149] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1150] Step 1:

[1151] A user puts on a wearable device. The wearable device is equipped with a camera, microphone, GPS sensor, etc., and these sensors collect the user's behavior data in real time. The input is the user's location information, audio data, video data, etc., and the output is the collection of this data.

[1152] Step 2:

[1153] The terminal transmits the collected behavioral data to the server. The input is the behavioral data transmitted from the wearable device, and the output is the data transmitted to the server. Specifically, the device uploads data to the server at regular intervals.

[1154] Step 3:

[1155] The server analyzes the behavioral data it receives. The input is the behavioral data sent from the device, and the output is the analysis results. In concrete terms, an artificial intelligence engine on the server analyzes the data and learns the user's behavioral patterns and routines.

[1156] Step 4:

[1157] A user asks a question to a wearable device. For example, the user asks by voice, "What should I eat today?" The input is the user's voice data, and the output is the recognition result of the voice data. In concrete terms, the device uses voice recognition technology to convert the user's question into text data.

[1158] Step 5:

[1159] The server generates optimal suggestions based on the user's question. The input is the user's question and analyzed behavioral data, and the output is the content of the suggestions. In concrete terms, the server generates optimal suggestions for the user's question based on the data learned by the artificial intelligence engine.

[1160] Step 6:

[1161] The server transmits the generated proposal to the terminal. The input is the generated proposal, and the output is the proposal transmitted to the terminal. In concrete terms, the server transmits the proposal to the wearable device.

[1162] Step 7:

[1163] The terminal visually displays the proposal. The input is the proposal sent from the server, and the output is the proposal displayed on the display. The specific operation is to display the proposal on the display of the wearable device and notify the user.

[1164] In this way, it is possible to collect user behavior data in real time and provide highly accurate visual suggestions.

[1165] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1166] "Example 1"

[1167] As one embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotions of a user. This system learns the user's routine work by time and season, makes action suggestions to the user based on the learned routine work, and transmits the suggested content to the user. Furthermore, the system recognizes the user's emotions and adjusts the action suggestions according to the emotions. Specifically, the emotion engine recognizes emotions from the user's tone of voice, facial expressions, behavior patterns, etc. For example, if the user speaks in a sad tone of voice, the emotion engine recognizes that the user is sad and transmits the information to the AI ​​engine. The AI ​​engine, after recognizing that the user is sad, makes action suggestions to the user. For example, it makes a suggestion such as "How about listening to relaxing music today?" In this way, the system of the present invention can make personalized suggestions that take the user's emotions into consideration.

[1168] "Example 2"

[1169] As one embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotions of a user. This system learns the user's routine work by time and season, makes action suggestions to the user based on the learned routine work, and transmits the suggested content to the user. Furthermore, the system recognizes the user's emotions and adjusts the action suggestions according to the emotions. Specifically, the emotion engine recognizes emotions from the user's tone of voice, facial expressions, behavior patterns, etc. For example, if the user speaks in a sad tone of voice, the emotion engine recognizes that the user is sad and transmits the information to the AI ​​engine. The AI ​​engine, after recognizing that the user is sad, makes action suggestions to the user. For example, it makes a suggestion such as "How about listening to relaxing music today?" In this way, the system of the present invention can make personalized suggestions that take the user's emotions into consideration.

[1170] The process flow of each embodiment will be described below.

[1171] "Example 1"

[1172] Step 1: The system learns the user's routine work by time and season. This learning is done from the user's behavioral history, selection history, etc.

[1173] Step 2: Based on the learned routine, the system will provide the user with action suggestions, which will vary according to the user's current time and season.

[1174] Step 3: The system recognizes the user's emotions using an emotion engine that recognizes the user's emotions from the user's tone of voice, facial expressions, behavioral patterns, etc.

[1175] Step 4: Based on the emotion engine's recognized emotion, the system will adjust the action suggestions. For example, if the user is sad, the system will suggest listening to relaxing music, etc., depending on the user's emotion.

[1176] Example 1

[1177] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1178] Conventional systems can learn a user's routine work by time and season and suggest actions based on that, but they cannot make personalized suggestions that take into account the user's emotions. This makes it difficult to provide optimal suggestions based on the user's emotional state.

[1179] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1180] In this invention, the server includes a means for learning the user's routine work by time and season, a means for making action suggestions to the user based on the learned routine work, a means for communicating the suggested content to the user, a means for recognizing the user's emotion, and a means for adjusting the action suggestions based on the recognized emotion, thereby enabling personalized suggestions according to the user's emotional state.

[1181] "User's routine work by time or season" refers to actions or habits that a user repeatedly performs at specific times of the day or during specific seasons.

[1182] "Learning means" refers to technologies and algorithms used to collect and analyze user behavior data to recognize patterns.

[1183] "Means for making action suggestions" refers to technologies and algorithms for suggesting optimal actions to users based on learned user routine work.

[1184] "Means for communicating suggestions to the user" refers to an interface or device for notifying the user of the generated action suggestions.

[1185] "Means of recognizing user emotions" refers to technologies and algorithms for analyzing and recognizing emotions from a user's tone of voice, facial expressions, behavioral patterns, etc.

[1186] "Means for tailoring action suggestions based on recognized emotions" refers to technologies or algorithms for appropriately modifying the content of action suggestions based on the user's emotional state.

[1187] The present invention is a system that learns a user's routine work by time and season, suggests actions based on the learned routine work, and further recognizes the user's emotions to adjust the suggested content. A specific embodiment of this system will be described below.

[1188] System Configuration

[1189] Hardware and Software

[1190] The server is equipped with an AI engine for collecting and analyzing user behavior data. This AI engine uses machine learning algorithms to learn the user's routine work. It also has an emotion engine that recognizes emotions from the user's tone of voice, facial expressions, behavioral patterns, etc. The terminal is a device such as a smartphone or tablet, and plays a role in conveying the suggestions from the server to the user.

[1191] Data collection and learning

[1192] The server collects data about the user's daily life. For example, the user inputs information such as "Wake up at 7am," "Eat breakfast at 8am," and "Get home at 6pm" through a smartphone app. This data is input into an AI engine, which analyzes the user's behavioral patterns and models routines by time and season.

[1193] Generate action suggestions

[1194] The server generates action suggestions for the user based on the learning results. For example, when a user asks, "What should I do today?", the server uses an AI engine to generate optimal suggestions. Specifically, it makes suggestions such as, "The weather is nice today, so how about going for a walk?"

[1195] Communication of proposals

[1196] The device transmits the suggestions received from the server to the user. The user can receive the suggestions through a device such as a smartphone or tablet. For example, if the user asks "What should I eat today?" on their smartphone, the device will display "It would be good to eat a salad with lots of vegetables today."

[1197] Emotion recognition and suggestion adjustment

[1198] The server recognizes the user's emotions using an emotion engine. The emotion engine analyzes emotions from the user's tone of voice, facial expressions, behavioral patterns, etc. For example, if the user speaks in a sad tone of voice, the emotion engine recognizes the user as sad and transmits that information to the AI ​​engine. The server adjusts behavior suggestions based on the emotion information received from the emotion engine. For example, if the user is recognized as sad, the server makes a suggestion such as, "How about listening to some relaxing music today?"

[1199] Examples

[1200] Example 1: User questions and suggestions

[1201] When a user asks, "What should I eat today?", the server takes into account past data and the current season and suggests, "How about eating a salad with lots of vegetables today?" The device displays this suggestion to the user.

[1202] Example 2: Sentiment-based suggestions

[1203] When a user asks "What should I do today?" in a sad tone of voice, the emotion engine recognizes that the user is sad, and the server suggests "How about listening to some relaxing music today?" The device displays this suggestion to the user.

[1204] Examples of prompt statements

[1205] "What should I do today?"

[1206] "What should I eat today?"

[1207] "What should I wear today?"

[1208] In this way, the system of the present invention can make personalized suggestions that take into account the user's routine and emotions.

[1209] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1210] Step 1:

[1211] The server collects data about the user's daily life. The user inputs information such as "wake up at 7am", "have breakfast at 8am", and "return home at 6pm" through a smartphone app. This data is sent to the server and stored in a database. The input data is the user's behavioral patterns, and the output is the stored behavioral data.

[1212] Step 2:

[1213] The server inputs the collected data into the AI ​​engine, which learns the user's routine work. The AI ​​engine uses machine learning algorithms to analyze the user's behavioral patterns and model routines by time and season. For example, the AI ​​engine learns a pattern that "the user wakes up at 7am on weekdays and 8am on weekends." The input data is the stored behavioral data, and the output is a model of the learned routine work.

[1214] Step 3:

[1215] The server generates action suggestions for the user based on the learning results. When the user asks, "What should I do today?", the server uses an AI engine to generate optimal suggestions. For example, it may suggest, "The weather is nice today, so how about going for a walk?" The input data is the user's question and the learned routine work model, and the output is the generated action suggestions.

[1216] Step 4:

[1217] The terminal transmits the suggestions received from the server to the user. The user can receive the suggestions through a terminal such as a smartphone or tablet. For example, if the user asks on their smartphone, "What should I eat today?", the terminal will display, "It would be good to eat a salad with lots of vegetables today." The input data is the suggestions from the server, and the output is the suggestions that are displayed to the user.

[1218] Step 5:

[1219] The server uses an emotion engine to recognize the user's emotions. The emotion engine analyzes emotions from the user's tone of voice, facial expressions, behavioral patterns, etc. For example, if a user speaks in a sad tone of voice, the emotion engine recognizes that the user is sad and transmits that information to the AI ​​engine. The input data are the user's tone of voice, facial expressions, and behavioral patterns, and the output is the recognized emotion information.

[1220] Step 6:

[1221] The server adjusts the behavioral suggestions based on the emotion information received from the emotion engine. For example, if the user is recognized as sad, the server will suggest, "How about listening to some relaxing music today?" The input data are the recognized emotion information and the learned routine work model, and the output is the adjusted behavioral suggestions.

[1222] (Application example 1)

[1223] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1224] Conventional behavior suggestion systems can make suggestions based on the user's routine work, but they cannot make suggestions that take into account the user's emotions and shopping patterns, so personalized suggestions are lacking. In addition, there are limited means to effectively communicate the contents of the suggestions to the user, which means that the system cannot fully improve user convenience.

[1225] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1226] In this invention, the server includes a means for learning a user's routine work by time and season, a means for making an action suggestion to the user based on the learned routine work, a means for transmitting the content of the suggestion to the user, a means for recognizing the user's emotion, a means for adjusting the action suggestion based on the recognized emotion, a means for learning the user's shopping pattern, a means for making a product suggestion based on the learned shopping pattern, and a means for transmitting the content of the suggestion through a smartphone notification or a voice assistant. This enables personalized suggestions that take into account the user's emotion and shopping pattern, thereby improving user convenience.

[1227] "Means of learning the user's routine work by time and season" is a function for recording and analyzing the user's daily life patterns and habits according to time and season.

[1228] "Means of suggesting actions to the user based on learned routine work" is a function for suggesting optimal actions to the user based on recorded routine work data.

[1229] "Means of communicating the content of the suggestions to the user" refers to a function for notifying the user of the content of the action suggestions, and is carried out through smartphone notifications or voice assistants.

[1230] "Means of recognizing the user's emotions" refers to a function for analyzing and recognizing emotions from the user's tone of voice, facial expressions, behavioral patterns, etc.

[1231] The "means for adjusting action suggestions based on the recognized emotions" is a function for appropriately changing the content of action suggestions according to the user's emotional state.

[1232] "Means of learning user shopping patterns" refers to a function for recording and analyzing a user's past purchasing history and behavior.

[1233] The "means for making product suggestions based on learned shopping patterns" is a function for suggesting optimal products to users based on recorded shopping pattern data.

[1234] "Means of communicating the content of suggestions through smartphone notifications or voice assistants" refers to a function for communicating the content of action suggestions or product suggestions to users using the notification function of a smartphone or a voice assistant.

[1235] The system for implementing this invention learns the user's routine work by time and season, makes action suggestions based on the learned routine work, and communicates the suggested content to the user. It also has a function of recognizing the user's emotions and adjusting the action suggestions based on the emotions. Furthermore, it learns the user's shopping patterns and makes product suggestions based on the learned shopping patterns.

[1236] Hardware and Software Used

[1237] Hardware:

[1238] Smartphone (iOS or Android)

[1239] software:

[1240] Python

[1241] TextBlob (a library for sentiment analysis)

[1242] Data processing and calculation

[1243] server:

[1244] The server records and analyzes the user's daily life patterns and habits to learn the user's time and seasonal routines, which includes storing the activities performed by the user and the timestamps of those activities in a database.

[1245] Device:

[1246] The device (smartphone) receives input from the user and recognizes emotions based on the input. For emotion recognition, the TextBlob library is used to analyze the user's input text and classify it as positive, negative, or neutral.

[1247] User:

[1248] The user asks questions such as "What should I do today?", "What should I eat today?", "What should I wear today?", "What are today's recommended products?", etc. through their smartphone. In response, the server generates optimal suggestions based on the learned routine work, shopping patterns, and recognized emotions, and conveys them to the user through smartphone notifications or voice assistants.

[1249] Examples

[1250] For example, if a user types "I'm a little tired today," the device analyzes this input using the TextBlob library and recognizes it as a negative emotion. The server receives this emotion information and, taking into account the user's past routine work and shopping patterns, generates suggestions such as "I recommend products that will help you relax today." These suggestions are communicated to the user through smartphone notifications and voice assistants.

[1251] Examples of prompt statements

[1252] Examples of prompts for a generative AI model might include the following:

[1253] Develop a system that recognizes the user's emotions and suggests actions based on those emotions. If a user types, "I'm a bit tired today," suggest products that will help them relax.

[1254] In this way, personalized suggestions that take into account the user's emotions and shopping patterns are possible, increasing user convenience.

[1255] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1256] Step 1:

[1257] A user types "I'm a little tired today" through their smartphone. The typed text is sent to the terminal.

[1258] Step 2:

[1259] The device receives the input text and performs sentiment analysis using the TextBlob library, specifically classifying the text as positive, negative, or neutral. The input is the user's text, and the output is the sentiment classification result.

[1260] Step 3:

[1261] The terminal transmits the result of emotion analysis to the server. The input is the emotion classification result, and the output is the data transmission to the server.

[1262] Step 4:

[1263] The server receives the emotion analysis results and retrieves the user's past routine work and shopping patterns from the database. The input is the emotion classification result, and the output is the data of the user's past routine work and shopping patterns.

[1264] Step 5:

[1265] The server generates optimal action suggestions for the user based on the acquired data. For example, if the user is showing negative emotions, it will suggest products that will help them relax. The input is data on past routine work and shopping patterns and the results of emotion classification, and the output is action suggestions.

[1266] Step 6:

[1267] The server transmits the generated action proposal to the terminal. The input is the action proposal, and the output is data transmission to the terminal.

[1268] Step 7:

[1269] The device notifies the user of the action suggestions it receives. Specifically, it communicates the content of the suggestions using the notification function of the smartphone or a voice assistant. The input is the action suggestions, and the output is a notification to the user.

[1270] In this way, personalized suggestions are made based on the user's emotions and past behavioral patterns.

[1271] Example 2

[1272] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1273] Conventional systems suggested actions based on the user's routine work, but did not take the user's emotions into account. As a result, appropriate suggestions could not be made according to the user's emotional state, making it difficult to improve user satisfaction. In addition, responses to questions from users were uniform, and personalized suggestions tailored to the individual user's situation were lacking.

[1274] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1275] In this invention, the server includes a means for learning the user's routine work by time and season, a means for making an action suggestion to the user based on the learned routine work, a means for communicating the suggestion content to the user, a means for recognizing the user's emotion, and a means for adjusting the action suggestion based on the recognized emotion, thereby enabling personalized suggestions that take into account the user's routine work and emotion.

[1276] "Means for learning a user's time and seasonal routine work" refers to a device or program for collecting and analyzing a user's daily life patterns and habits.

[1277] The "means for suggesting actions to the user based on learned routine work" refers to a device or program for suggesting optimal actions to the user based on collected and analyzed data.

[1278] The "means for communicating the content of the suggestion to the user" refers to a device or program for notifying the user of the generated action suggestion.

[1279] The "means for recognizing a user's emotions" refers to a device or program for detecting and recognizing emotions from a user's tone of voice, facial expressions, behavioral patterns, etc.

[1280] The "means for adjusting action suggestions based on the recognized emotion" refers to a device or program for modifying the content of action suggestions in accordance with the user's emotional state and making optimal suggestions.

[1281] The present invention is a system that learns a user's routine work by time and season, suggests actions based on the learned routine work, and further recognizes the user's emotions to adjust the suggested content. A specific embodiment of this system will be described below.

[1282] System Configuration

[1283] Hardware

[1284] Server: A central processing unit for data collection, analysis, proposal generation, and emotion recognition.

[1285] Device: The device a user uses, such as a smartphone or tablet.

[1286] Sensors: Microphones and cameras to recognize the user's tone of voice and facial expressions.

[1287] software

[1288] AI engine: Software that learns a user's routine work and generates action suggestions.

[1289] Emotion engine: Software for recognizing user emotions and communicating that information to the AI ​​engine.

[1290] Data collection and learning

[1291] The server collects user behavior data, including what time the user wakes up, what meals they eat, what clothes they wear, etc. Specifically, it collects data from smartphones and wearable devices.

[1292] Example: Information entered by a user into a smartphone app or activity data obtained from a wearable device is sent to a server.

[1293] Generate action suggestions

[1294] The server uses an AI engine to learn the user's routine work based on the collected data, and the AI ​​engine analyzes time and seasonal patterns to generate a model that predicts user behavior.

[1295] Example: A server analyzes historical data to determine whether users tend to behave in certain ways at certain times.

[1296] Communication of proposals

[1297] The server transmits the generated suggestions to the user. When the user inputs a question via a device (such as a smartphone or tablet), the server responds with the best suggestions for that question.

[1298] Example: A user types a question into a smartphone app and receives suggestions from the server.

[1299] Emotion recognition and suggestion adjustment

[1300] The server uses an emotion engine to recognize the user's emotion, which is determined from the user's tone of voice, facial expression, behavioral patterns, etc.

[1301] Example: Emotions are recognized by analyzing the tone of voice when a user speaks through a smartphone microphone and facial expressions captured through the camera.

[1302] The server then uses the information obtained from the emotion engine to suggest actions based on the user's emotions. For example, if the server detects that the user is sad, it will suggest, "How about listening to some relaxing music today?"

[1303] Examples of prompt statements

[1304] "What should I do today?"

[1305] "What should I eat today?"

[1306] "What should I wear today?"

[1307] In this way, the inventive system can make personalized suggestions that take into account the user's routine and emotions.

[1308] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1309] Step 1:

[1310] User data collection

[1311] The server collects user behavior data. As input, it receives data provided by the user from a smartphone or wearable device. This data includes the user's wake-up time, what they ate, what they were wearing, etc. The server stores this data in a database for later analysis.

[1312] Specific operation: Information entered by the user into the smartphone app and activity data acquired from the wearable device are sent to the server, which receives the data and stores it in a database.

[1313] Step 2:

[1314] Learning routine work

[1315] The server uses the AI ​​engine to learn the user's routine based on the collected data. As input, it uses past behavioral data stored in a database. The AI ​​engine analyzes these data and finds patterns over time and seasons. As output, it generates a model that predicts the user's behavior.

[1316] How it works: The server analyzes past data and finds out if a user tends to behave in a certain way at certain times. For example, it learns that a user wakes up at 7am every morning and eats breakfast at 8am.

[1317] Step 3:

[1318] Generate action suggestions

[1319] The server generates action suggestions for the user based on the learned routine work. As input, it uses the prediction model generated by the AI ​​engine and questions from the user. As output, it generates optimal action suggestions and provides them to the user.

[1320] What happens: If a user asks, "What should I eat today?" the server uses past data to suggest, "How about a salad with lots of vegetables today?"

[1321] Step 4:

[1322] Communication of proposals

[1323] The server communicates the generated suggestions to the user, uses the generated action suggestions as input, and sends the suggestions to the user's device as output.

[1324] How it works: A user enters a question into a smartphone app and receives suggestions from the server, which then sends the suggestions to the user's device, where the user can review them.

[1325] Step 5:

[1326] Emotion recognition

[1327] The server uses an emotion engine to recognize the user's emotions. As input, it uses the user's tone of voice, facial expressions, and behavioral patterns. As output, it recognizes the user's emotional state and conveys that information to the AI ​​engine.

[1328] Specific operation: The system recognizes emotions by analyzing the tone of voice the user speaks through the smartphone microphone and facial expressions captured through the camera. For example, if the user speaks in a sad tone, the emotion engine transmits that information to the AI ​​engine.

[1329] Step 6:

[1330] Tailoring behavior suggestions based on emotions

[1331] The server proposes actions based on the user's emotions based on the information obtained from the emotion engine. As input, it uses the emotion information from the emotion engine and the prediction model of the AI ​​engine. As output, it generates optimal action proposals based on emotions and provides them to the user.

[1332] Specific behavior: If the user is recognized as sad, the server will make a suggestion such as "How about listening to some relaxing music today?" The server will send this suggestion to the user's device, and the user will confirm it.

[1333] (Application example 2)

[1334] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1335] Conventional systems suggested actions based on the user's time and seasonal routines, but they lacked personalized suggestions that took into account the user's emotions and purchasing history. In addition, they did not offer real-time product suggestions to improve the shopping experience in physical stores. This led to problems such as reduced user satisfaction and convenience.

[1336] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1337] In this invention, the server includes a means for learning a user's routine work by time and season, a means for making an action suggestion to the user based on the learned routine work, a means for transmitting the suggestion content to the user, a means for recognizing the user's emotion, a means for adjusting the action suggestion based on the recognized emotion, a means for learning the user's purchase history, a means for making a product suggestion based on the learned purchase history, and a means for displaying the suggestion content to the user in real time. This enables personalized suggestions that take into account the user's emotion and purchase history, thereby improving the shopping experience at a physical store.

[1338] "User's routine work by time or season" refers to actions or habits that a user repeatedly performs at specific times of the day or during specific seasons.

[1339] "Means of learning" refers to the technologies and algorithms used to collect and analyze user behavioral patterns and habits as data.

[1340] "Means for making action suggestions" refers to technologies or algorithms that present optimal actions and options to users based on learned data.

[1341] "Means of communicating suggestions to users" refers to any interface or device used to inform users of suggested actions or options.

[1342] "Means of recognizing user emotions" refers to technologies and algorithms for analyzing and recognizing emotions from a user's tone of voice, facial expressions, behavioral patterns, etc.

[1343] "Means for tailoring action suggestions based on recognized emotions" means techniques or algorithms for modifying the content of action suggestions in response to the user's emotional state.

[1344] "Means of learning a user's purchasing history" refers to technologies and algorithms for collecting and analyzing data on products a user has previously purchased.

[1345] "Means for making product suggestions based on learned purchasing history" refers to technologies and algorithms for suggesting optimal products based on a user's purchasing history.

[1346] "Means for displaying suggestions to the user in real time" refers to an interface or device for instantly informing the user of suggested products or actions.

[1347] The system for implementing this invention learns the user's routine work by time and season, makes action suggestions based on the learned routine work, and communicates the suggested content to the user. It also recognizes the user's emotions and adjusts the action suggestions based on the recognized emotions. It also learns the user's purchase history, makes product suggestions based on the learned purchase history, and displays the suggested content to the user in real time.

[1348] Hardware and Software Used

[1349] Hardware: smart glasses, camera, microphone, server

[1350] Software: Emotion recognition model (OpenCV), clustering algorithm (KMeans), generative AI model

[1351] Data processing and calculation

[1352] 1. Learn user's time and seasonal routines:

[1353] The server collects data on users' behavioral patterns and habits and analyzes them using a clustering algorithm (KMeans), thereby learning the user's routine work.

[1354] 2. Generate action suggestions:

[1355] Based on the learned data, generative AI models are used to suggest optimal actions and options to users, and these suggestions are provided as answers to the user's questions.

[1356] 3. Emotion recognition:

[1357] The camera and microphone installed in the smart glasses are used to collect the user's facial expressions and tone of voice, which are then analyzed using an emotion recognition model (OpenCV), allowing the system to recognize the user's emotions.

[1358] 4. Tailoring behavioral suggestions based on emotions:

[1359] Based on the emotions recognized, the generative AI model will tailor action suggestions, for example making suggestions to help users relax if they are sad.

[1360] 5. Learning purchase history:

[1361] The server collects the user's past purchase history and analyzes it using a clustering algorithm (KMeans), thereby learning the user's purchase history.

[1362] 6. Generate product proposals:

[1363] Based on the learned purchasing history, a generative AI model is used to suggest optimal products, which are then displayed in real time on the smart glasses' display.

[1364] Examples

[1365] User's purchasing history: In the past, they have purchased T-shirts and sunglasses.

[1366] Current season: Summer.

[1367] User emotion: "happy" recognized from camera footage.

[1368] Examples of prompt statements

[1369] "Learn your users' purchasing history and suggest the best products based on the current season and sentiment."

[1370] "Please suggest products that users are likely to purchase in the summer."

[1371] "Suggest products that users are likely to purchase when they're in a happy mood."

[1372] In this way, personalized suggestions can be made that take into account the user's emotions and purchasing history, improving the in-store shopping experience.

[1373] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1374] Step 1:

[1375] The server learns the user's routine work by time and season. Specifically, it collects the user's behavioral patterns and habits as data and analyzes them using a clustering algorithm (KMeans). The input is the user's behavioral data, and the output is the user's routine work pattern.

[1376] Step 2:

[1377] The server makes action suggestions to the user based on the learned routine work. Using a generative AI model, it suggests optimal actions and options to the user. The input required is the routine work pattern obtained in step 1, and the output is the action suggestions for the user.

[1378] Step 3:

[1379] The terminal communicates the proposed content to the user. Specifically, it displays the proposed content on the display of the smart glasses. As input, the proposed action obtained in step 2 is required, and as output, the user can visually confirm the proposed content.

[1380] Step 4:

[1381] The device recognizes the user's emotions. It uses the camera and microphone installed in the smart glasses to collect the user's facial expressions and tone of voice, and analyzes them using an emotion recognition model (OpenCV). The input requires the user's facial expressions and tone of voice data, and the output is the user's emotional state.

[1382] Step 5:

[1383] The server adjusts the action suggestions based on the recognized emotions. Using a generative AI model, it generates action suggestions according to the user's emotional state. As input, it requires the emotional state data obtained in step 4, and as output, it obtains the adjusted action suggestions.

[1384] Step 6:

[1385] The server learns the user's purchase history. It collects the user's past purchase history and analyzes it using a clustering algorithm (KMeans). As input, it requires the user's purchase history data, and as output, it obtains the user's purchase pattern.

[1386] Step 7:

[1387] The server makes product suggestions based on the learned purchasing history. It uses a generative AI model to suggest the best products for the user. As input, it requires the purchasing patterns obtained in step 6, and as output, it obtains product suggestions for the user.

[1388] Step 8:

[1389] The terminal then displays the,suggested content to the user in real time.,The suggested products and actions are instantly displayed,on the display of the smart glasses.,As input, the product suggestions obtained in,step 7 are required, and as output, the user can,check the suggested content in real time.

[1390] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1391] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1392] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1393] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1394] [Fourth embodiment]

[1395] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1396] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1397] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1398] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1399] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1400] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1401] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1402] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1403] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1404] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1405] The storage 32 stores a data generation model 58 and an emotion identification model 59.

[1406] The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1407] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1408] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1409] "Example 1"

[1410] The system of the present invention includes a means for learning the user's routine work by time and season, a means for suggesting actions to the user based on the learned routine work, and a means for communicating the suggested content to the user. Specifically, the system is equipped with an AI engine that learns the user's daily life patterns and habits, and makes action suggestions to the user based on the learning results. The suggested content includes options such as "menu," "clothes to wear today," and "procedures that need to be done at this time of year." These suggestions are made based on the learned patterns and communicated to the user when the user asks questions such as "What should I do today?", "What should I eat today?", and "What should I wear today?".

[1411] "Example 2"

[1412] The system of the present invention includes a means for learning the user's routine work by time and season, a means for suggesting actions to the user based on the learned routine work, and a means for communicating the suggested content to the user. Specifically, the system is equipped with an AI engine that learns the user's daily life patterns and habits, and makes action suggestions to the user based on the learning results. The suggested content includes options such as "menu," "clothes to wear today," and "procedures that need to be done at this time of year." These suggestions are made based on the learned patterns and communicated to the user when the user asks questions such as "What should I do today?", "What should I eat today?", and "What should I wear today?".

[1413] The process flow of each embodiment will be described below.

[1414] "Example 1"

[1415] Step 1: Learn the user's daily patterns and habits The AI ​​engine observes the user's behavior and collects data. For example, it learns that the user wakes up at 7:00 every morning and has breakfast at 7:30.

[1416] Step 2: Based on the collected data, the AI ​​engine learns the user's routine work using machine learning techniques.

[1417] Step 3: Based on the learned routine work, the AI ​​engine suggests actions to the user. For example, when the user asks, "What should I do today?", the AI ​​engine suggests, "Wake up at 7:00 and have breakfast at 7:30."

[1418] Step 4: The proposal is communicated to the user. This is done through the notification function of the device (e.g. smartphone, PC) used by the user, or through a voice assistant.

[1419] Example 1

[1420] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1421] In today's busy lifestyles, it is difficult for users to efficiently manage their daily routines and seasonal activities. In particular, choosing daily meals, clothing, necessary procedures, etc. often takes time, which can be a source of stress. In addition, in order to optimize these choices, it is necessary to take into account the user's past behavioral data and current situation, but this is very difficult to do manually.

[1422] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1423] In this invention, the server includes a means for learning a user's routine work by time and season, a means for making action suggestions to the user based on the learned routine work, a means for transmitting the content of the suggestions to the user, a means for collecting user action data, a means for preprocessing the collected data, a means for training an AI engine using the preprocessed data, a means for accepting questions from the user, a means for generating optimal suggestions for the questions, and a means for transmitting the content of the suggestions to the user. This enables the user to efficiently manage daily routine work and seasonal actions and receive optimal suggestions.

[1424] "Means of learning the user's routine work by time and season" refers to technology that collects the user's daily life patterns and habits and uses this to predict the user's behavior.

[1425] The "means of suggesting actions to the user based on learned routine work" refers to technology that analyzes collected data and suggests optimal actions based on the user's past behavioral patterns.

[1426] The "means for communicating the content of the proposal to the user" refers to an interface or communication means for notifying the user of the generated proposal.

[1427] "Means of collecting user behavioral data" refers to technology for obtaining user location information, calendar information, past behavioral history, etc.

[1428] "Means of preprocessing collected data" refers to techniques for filling in missing values, removing outliers, converting data formats, etc. in collected data.

[1429] "Means for training an AI engine using preprocessed data" refers to a technology that uses preprocessed data as input to train an AI model and have it learn user behavior patterns.

[1430] The "means for accepting questions from the user" is an interface that enables the user to input questions to the system.

[1431] "Means for generating optimal suggestions in response to questions" refers to technology that generates optimal answers to questions from users based on a learned model.

[1432] The "means for communicating the content of the proposal to the user" refers to an interface or communication means for notifying the user of the generated proposal.

[1433] This invention relates to a system that learns a user's routine work by time and season and suggests optimal actions. This system includes a series of processes that collect user behavior data, learns using an AI engine, generates optimal suggestions in response to questions from the user, and communicates the suggested content to the user.

[1434] To collect user behavior data, the server obtains location information, calendar information, past behavior history, etc. from smartphones and wearable devices. For example, when a user enters an event such as "going to the gym" into the smartphone calendar, that information is sent to the server.

[1435] Next, the server preprocesses the collected data. Specifically, it complements missing values ​​in the data and removes outliers. It also converts the data into a format that is easy for the AI ​​engine to learn from. For example, it converts location information from latitude and longitude to an address.

[1436] Using the preprocessed data, the server trains the AI ​​engine. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to model the user's behavioral patterns. For example, if a user has a habit of going to the gym every Monday, it will learn that pattern.

[1437] The terminal provides an interface that accepts questions from the user. When the user inputs a question such as "What should I do today?", "What should I eat today?", or "What should I wear today?", the terminal sends the question to the server.

[1438] The server generates optimal suggestions for the received questions from the patterns it has learned. Specifically, it makes appropriate suggestions by taking into account the user's past behavioral history and current situation (weather, time of day, etc.). For example, if a user asks, "What should I eat today?" the server will suggest the optimal menu based on the user's past eating history, the current season, time of day, etc.

[1439] The device conveys the suggestions received from the server to the user. Specifically, it displays them in the form of text messages, voice messages, etc. For example, it displays a suggestion such as "Today is sunny and the temperature is 25 degrees, so it would be good to wear a light jacket and jeans."

[1440] Examples of prompts include:

[1441] "What should I do today?"

[1442] "What should I eat today?"

[1443] "What should I wear today?"

[1444] This enables users to efficiently manage their daily routine work and seasonal activities and receive optimal suggestions.

[1445] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1446] Step 1:

[1447] Collecting user behavior data

[1448] The server collects location information, calendar information, and past activity history from the user's smartphone or wearable device. For example, when a user enters an event such as "going to the gym" into the smartphone calendar, the information is sent to the server. The input data includes location information, calendar information, and activity history, which are then stored in a database.

[1449] Step 2:

[1450] Data Preprocessing

[1451] The server preprocesses the collected data. Specifically, it complements missing values ​​in the data and removes outliers. It also converts the data into a format that is easy for the AI ​​engine to learn from. For example, it converts location information from latitude and longitude to an address. The input data is the raw data collected in step 1, and the output data is the preprocessed, clean data.

[1452] Step 3:

[1453] AI engine learning

[1454] The server trains the AI ​​engine using the preprocessed data. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to model the user's behavioral patterns. For example, if a user has a habit of going to the gym every Monday, that pattern is learned. The input data is the preprocessed data, and the output data is the trained AI model.

[1455] Step 4:

[1456] Accepting questions from users

[1457] The terminal provides an interface that accepts questions from the user. When the user inputs a question such as "What should I do today?", "What should I eat today?", or "What should I wear today?", the terminal sends the question to the server. The input data is the user's question, and the output data is a request to the server.

[1458] Step 5:

[1459] Generate best-fit suggestions for your questions

[1460] The server generates optimal suggestions for the received questions from the patterns it has learned. Specifically, it makes appropriate suggestions by taking into account the user's past behavioral history and current situation (weather, time of day, etc.). For example, if a user asks, "What should I eat today?", the server will suggest the optimal menu based on the user's past meal history, the current season, time of day, etc. The input data is the user's question and the trained AI model, and the output data is the suggestions.

[1461] Step 6:

[1462] Communicating the proposal to users

[1463] The terminal conveys the suggestions received from the server to the user. Specifically, it displays them in the form of text messages, voice messages, etc. For example, it displays a suggestion such as "Today is sunny with a temperature of 25 degrees, so it would be good to wear a light jacket and jeans." The input data is the suggestion from the server, and the output data is a notification to the user.

[1464] (Application example 1)

[1465] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1466] Conventional systems can learn a user's daily life patterns and habits and make action suggestions, but they lack specific product suggestions and real-time notification functions to improve the shopping experience in physical stores. In addition, optimal product suggestions are not made taking into account the user's past purchase history and behavioral patterns, which may reduce user satisfaction. Furthermore, the system lacks the ability to provide appropriate answers to user questions.

[1467] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for learning the user's routine work by time and season, a means for suggesting actions to the user based on the learned routine work, a means for communicating the content of the suggestions to the user, a means for learning the user's past purchase history and behavioral patterns, a means for suggesting optimal products according to the current season and time period, a means for notifying the user of in-store special sale information and new product information in real time, and a means for providing answers to questions from the user. This makes it possible to improve the user's shopping experience and increase satisfaction.

[1468] "Means of learning the user's routine work by time and season" is a function for collecting and analyzing the user's daily life patterns and habits by time and season.

[1469] "A means of suggesting actions to the user based on learned routine work" is a function for suggesting optimal actions to the user based on collected user routine work data.

[1470] "Means of communicating the proposed content to the user" refers to a function for notifying the user of proposed actions or information.

[1471] "Means of learning about a user's past purchase history and behavioral patterns" refers to a function for collecting and analyzing a user's past purchase history and behavioral history.

[1472] "Means for suggesting optimal products according to the current season and time of day" refers to a function for selecting and suggesting optimal products to users based on the current season and time of day.

[1473] "Means of notifying users of in-store sales and new product information in real time" is a function for notifying users of sales and new product information in physical stores in real time.

[1474] "Means for providing answers to questions from users" is a function for providing appropriate answers to questions asked by users.

[1475] The system for carrying out this invention learns the user's routine work by time and season, suggests actions to the user based on the learned routine work, and communicates the contents of the suggestions to the user.Furthermore, it includes a function to learn the user's past purchase history and behavioral patterns, suggest optimal products according to the current season and time of day, and notify the user of in-store special sale information and new product information in real time.

[1476] System Configuration

[1477] Hardware

[1478] Server: A central processing unit for collecting data, analyzing it, and generating recommendations.

[1479] Device: The device used by a user, such as a smartphone or smart glasses.

[1480] Network: Infrastructure for data communication between servers and terminals.

[1481] software

[1482] AI engine: An algorithm that learns users' routines and purchase history to generate action suggestions.

[1483] Database: Storage for storing user behavior patterns, purchase history, and product information.

[1484] Notification system: Software that notifies users of offers, special offers, and new product information in real time.

[1485] Data processing and calculation

[1486] The server collects user behavior data and purchase history, and analyzes this data using an AI engine. Specifically, it processes and calculates the following data:

[1487] 1. Data collection: User behavior data and purchase history are sent from the device to the server.

[1488] 2. Data analysis: The AI ​​engine analyzes the collected data and learns the user's routine work and behavioral patterns.

[1489] 3. Action suggestion generation: Based on the learning results, optimal action suggestions are generated for the user.

[1490] 4. Real-time notifications: Proposals, sale information, and new product information are sent to users' devices in real time.

[1491] Examples

[1492] For example, if User A has a history of purchasing a "Hanami Bento" and a "Spring Jacket" in the spring, the system will learn this and suggest "Hay Fever Prevention Goods" and "Spring Jacket" in the spring. Also, when a user asks "What's recommended today?", the system will suggest the best products according to the current season and time of day.

[1493] Examples of prompt statements

[1494] When a user asks "What's the recommendation for today?", the system generates a prompt like this:

[1495] Build an assistant that learns the user's purchase history and suggests the best products based on the current season. If the user purchased "Hanami Bento" and "Spring Jacket" in spring, suggest recommended products for spring.

[1496] In this way, it is possible to realize a smart shopping assistant that learns the user's behavioral patterns and makes optimal product suggestions.

[1497] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1498] Step 1:

[1499] The user uses the terminal to input behavioral data and purchase history. The terminal sends this data to the server. The input data includes information on purchased products and timestamps of actions. The server receives this data and stores it in a database.

[1500] Step 2:

[1501] The server inputs the user's behavioral data and purchase history stored in the database into the AI ​​engine. The AI ​​engine analyzes this data and learns the user's routine work and behavioral patterns. Specifically, it performs time series analysis and clustering of the data to extract the user's behavioral patterns.

[1502] Step 3:

[1503] The server generates action suggestions based on the learning results. The AI ​​engine considers the user's behavioral patterns and the current season and time of day to generate optimal action suggestions. For example, in the spring season, it suggests "hay fever prevention goods" and "spring jackets." The generated suggestions are stored in a database.

[1504] Step 4:

[1505] A user uses a terminal to input a question such as "What's recommended today?" The terminal sends the question to a server, which analyzes the question and retrieves appropriate action suggestions from a database.

[1506] Step 5:

[1507] The server transmits the obtained action suggestions to the terminal. The terminal notifies the user of the received suggestions. The notification is made by a text message or a push notification. The user can confirm and execute the suggested action or product.

[1508] Step 6:

[1509] The server collects information on special sales and new products in the store in real time and stores it in a database. This information is obtained from the store's management system and external information services.

[1510] Step 7:

[1511] The server notifies the user's terminal of the sale information and new product information collected in real time. The terminal displays the received information to the user. The user can check the sale information and new product information and consider purchasing.

[1512] In this way, it is possible to realize a smart shopping assistant that learns the user's behavioral patterns and makes optimal product suggestions.

[1513] Example 2

[1514] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1515] In modern society, users lead busy lives and are required to efficiently manage their routine work according to time and season. However, conventional systems have difficulty fully learning users' behavioral patterns and habits and providing appropriate action suggestions. In addition, there was a lack of means to quickly provide optimal suggestions when users asked specific questions. This caused users to spend a lot of time and effort on daily choices and plans, preventing them from living an efficient life.

[1516] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1517] In this invention, the server includes: means for learning the user's routine work by time and season, means for suggesting actions to the user based on the learned routine work, means for transmitting the contents of the suggestions to the user, means for mounting an artificial intelligence engine that learns the user's daily life patterns and habits and for suggesting actions to the user based on the learning results, and means for making optimal suggestions based on the learned patterns and transmitting them to the user when the user asks questions such as "What should I do today?", "What should I eat today?", "What should I wear today?", etc. This enables the user to quickly receive optimal suggestions regarding daily choices and plans, enabling them to live an efficient life.

[1518] "User" refers to any individual or entity that uses the System.

[1519] "Routine work by time or season" refers to activities or tasks that a user performs regularly at specific times or seasons.

[1520] "Means of learning" refers to methods and technologies for analyzing user behavioral patterns and habits and accumulating them as data.

[1521] "Means for making action suggestions" refers to methods and technologies for presenting optimal actions and options to users based on learned data.

[1522] "Means for communicating the suggestions to the user" refers to the methods or technologies for notifying or displaying the generated suggestions to the user.

[1523] An "artificial intelligence engine" refers to software and algorithms for machine learning and data analysis.

[1524] "Question" refers to a request for information or inquiry made by a User to the System.

[1525] "Optimal suggestions" refer to the actions or options that are deemed most appropriate based on the user's past data and current situation.

[1526] The system of the present invention learns the user's routine work by time and season, and suggests actions to the user based on the results of that learning. Specifically, it is equipped with an artificial intelligence engine that learns the user's daily life patterns and habits, and suggests actions to the user based on the results of that learning. The suggestions include options such as "menu," "what to wear today," and "procedures that need to be done at this time of year."

[1527] Hardware and Software Used

[1528] Server: Collects, stores, analyzes data, and generates proposals. Specifically, it uses a database server and an AI engine (e.g., TensorFlow or PyTorch).

[1529] Terminal: The terminal where the user inputs data, sends questions, and displays the proposals. Specifically, a smartphone or PC is used.

[1530] User: Enters daily actions and choices and receives suggestions from the system.

[1531] Data collection

[1532] Users input their daily actions and choices into the device. For example, they record what they eat, what they wear, and what their plans are. The device collects this data and sends it to the server. For example, if a user enters "Today's breakfast is bread and coffee" into a smartphone app, that data is saved on the device. The device sends the collected data to the server periodically or in response to user operations.

[1533] Training the Data

[1534] The server receives the collected data and stores it in a database. For example, a user's dietary data and clothing data are stored in the server's database. The server uses an AI engine to learn from the stored data. Specifically, the server analyzes past data and models the user's behavioral patterns and seasonal routines. The server stores the learning results in a database.

[1535] Generate action suggestions

[1536] The user asks questions such as "What should I do today?", "What should I eat today?", and "What should I wear today?". For example, the user enters "What should I eat today?" into a smartphone app. The device sends this question to the server. Specifically, the user's question is sent from the device to the server as an HTTP request. The server generates optimal suggestions based on the learning results. For example, the server may suggest "It would be a good idea to make curry today" taking into account past eating patterns, the season, and the time of day. The server stores the generated suggestions in a database.

[1537] Communication of proposals

[1538] The server sends the generated suggestions to the device. Specifically, the server sends the suggestions to the device as an HTTP response. The device then displays the suggestions to the user. For example, the suggestions are displayed as notifications on the smartphone or messages within the app. When the user opens the app, the app displays the message, "It would be a good idea to make curry today."

[1539] Examples and prompts

[1540] Example 1: When a user asks, "What should I eat today?"

[1541] Prompt: "What should I eat today?"

[1542] Suggestion: "You might want to make curry today. Based on historical data, you often eat curry on Mondays."

[1543] Example 2: When a user asks, "What should I wear today?"

[1544] Prompt: "What should I wear today?"

[1545] Suggestion: "The temperature is cold today, so you should wear a sweater and a coat."

[1546] Example 3: When a user asks, "What steps do I need to take at this time?"

[1547] Prompt: "What procedures need to be completed at this time?"

[1548] Suggestion: "This month is tax return season, so get ready to go to the tax office."

[1549] In this way, the system learns the user's behavioral patterns and makes optimal suggestions.

[1550] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1551] Step 1: Data collection

[1552] The user inputs daily actions and choices into the terminal, for example recording what they eat, what they wear, what they plan to do, etc.

[1553] Input: A user types "Today's breakfast is bread and coffee" into an app on their smartphone.

[1554] The terminal collects these data and transmits them to a server. Specifically, the terminal uploads the collected data to the server periodically or in response to a user's operation.

[1555] Output: The collected data is sent to the server.

[1556] Step 2: Save your data

[1557] The server stores the received data in a database. For example, the user's food data and clothing data are stored in the server's database.

[1558] Input: User behavior data sent from the device.

[1559] The server uses a database management system (e.g., MySQL or PostgreSQL) to store the data.

[1560] Output: User behavior data stored in a database.

[1561] Step 3: Train the data

[1562] The server uses an AI engine (such as TensorFlow or PyTorch) to learn from the stored data. Specifically, the server analyzes past data and models the user's behavioral patterns and seasonal routines.

[1563] Input: User behavior data stored in the database.

[1564] The server applies machine learning algorithms to analyze the data and learn, for example, that users tend to eat curry on Mondays.

[1565] Output: A behavioral pattern model as a result of learning.

[1566] Step 4: Generate action suggestions

[1567] A user asks questions such as "What should I do today?", "What should I eat today?", "What should I wear today?", etc. For example, a user types "What should I eat today?" into a smartphone app.

[1568] Input: The user question (prompt).

[1569] The terminal sends this question to the server. Specifically, the user's question is sent from the terminal to the server as an HTTP request.

[1570] The server generates optimal suggestions based on the results of learning. For example, the server might suggest, "It would be good to make curry today," taking into account past eating patterns, the season, and the time of day.

[1571] Output: Generated action suggestions.

[1572] Step 5: Communicate your proposal

[1573] The server sends the generated proposal to the terminal. Specifically, the proposal content is sent from the server to the terminal as an HTTP response.

[1574] Input: Generated action suggestions.

[1575] The device will display the suggestions to the user. For example, it will appear as a smartphone notification or an in-app message. When the user opens the app, it will say, "It would be a good idea to make curry today."

[1576] Output: The suggestions displayed to the user.

[1577] (Application example 2)

[1578] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1579] Conventional behavior suggestion systems can learn a user's routine work and make suggestions, but there is a problem in that it is difficult to collect user behavior data in real time and make more accurate suggestions. In addition, the means by which users can receive suggestions are limited, and a method that allows them to understand the suggestions visually and intuitively is required.

[1580] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1581] In this invention, the server includes a means for learning the user's routine work by time and season, a means for making action suggestions to the user based on the learned routine work, a means for transmitting the content of the suggestions to the user, a wearable device equipped with a sensor for collecting the user's behavioral data, an artificial intelligence engine for analyzing the collected data and learning the user's routine, and a wearable device including a display for generating optimal suggestions based on the user's questions and visually displaying them. This makes it possible to collect the user's behavioral data in real time and visually provide highly accurate suggestions.

[1582] "User" means an individual or organization that uses the System.

[1583] "Routine work" refers to regular tasks and activities that users perform on a daily basis.

[1584] "Means of learning" are methods and technologies for collecting and analyzing user behavior data to recognize patterns.

[1585] "Action suggestions" are content that recommends actions to the user based on the learned routine work.

[1586] "Means of communicating the suggestions" refers to the methods or techniques used to notify the user of the suggested actions.

[1587] A "wearable device with sensors" is a wearable device equipped with sensors for collecting user behavioral data.

[1588] An "artificial intelligence engine" is a machine learning model or algorithm that analyzes collected data and learns user routines.

[1589] A "wearable device including a display" is a wearable device that has the ability to visually display information to a user.

[1590] The system for carrying out this invention learns the user's routine work by time and season, makes action suggestions based on the learned routine work, and communicates the suggested content to the user. Specifically, the system is implemented with the following configuration and procedure.

[1591] System Configuration

[1592] 1. Wearable devices equipped with sensors to collect user behavior data:

[1593] Wearable devices such as smart glasses and smart watches will be used.

[1594] These devices are equipped with cameras, microphones, GPS sensors, etc. to collect user behavior data in real time.

[1595] 2. An artificial intelligence engine that analyzes collected data and learns user routines:

[1596] An AI engine located on the server analyzes the collected data.

[1597] Use machine learning models to learn user behavior patterns and habits.

[1598] 3. A wearable device that includes a display that generates and visually displays optimal recommendations based on a user's questions:

[1599] Suggestions generated by the AI ​​engine are displayed on the display of the wearable device.

[1600] When the user asks a question using voice or gestures, suggestions appear on the display.

[1601] Program processing explanation

[1602] The server receives data from wearable devices equipped with sensors to collect user behavior data. These include smart glasses and smart watches. The collected data is analyzed by an artificial intelligence engine on the server. The AI ​​engine uses machine learning models to learn the user's routine and recognize behavioral patterns.

[1603] When a user asks a question such as "What should I eat today?", the AI ​​engine generates optimal suggestions based on the learned data. These suggestions are visually displayed on the display of the wearable device. For example, if a user asks "What should I eat today?", the display may show "I recommend curry for dinner tonight."

[1604] Examples of concrete examples and prompts

[1605] Examples

[1606] User Question: "What should I eat today?"

[1607] AI suggestion: "I recommend curry for dinner tonight."

[1608] Examples of prompt statements

[1609] Make suggestions for dinner today based on user behavior data. Users are asking, "What should I eat today?"

[1610] In this way, it is possible to collect user behavior data in real time and provide highly accurate visual suggestions.

[1611] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1612] Step 1:

[1613] A user puts on a wearable device. The wearable device is equipped with a camera, microphone, GPS sensor, etc., and these sensors collect the user's behavior data in real time. The input is the user's location information, audio data, video data, etc., and the output is the collection of this data.

[1614] Step 2:

[1615] The terminal transmits the collected behavioral data to the server. The input is the behavioral data transmitted from the wearable device, and the output is the data transmitted to the server. Specifically, the device uploads data to the server at regular intervals.

[1616] Step 3:

[1617] The server analyzes the behavioral data it receives. The input is the behavioral data sent from the device, and the output is the analysis results. In concrete terms, an artificial intelligence engine on the server analyzes the data and learns the user's behavioral patterns and routines.

[1618] Step 4:

[1619] A user asks a question to a wearable device. For example, the user asks by voice, "What should I eat today?" The input is the user's voice data, and the output is the recognition result of the voice data. In concrete terms, the device uses voice recognition technology to convert the user's question into text data.

[1620] Step 5:

[1621] The server generates optimal suggestions based on the user's question. The input is the user's question and analyzed behavioral data, and the output is the content of the suggestions. In concrete terms, the server generates optimal suggestions for the user's question based on the data learned by the artificial intelligence engine.

[1622] Step 6:

[1623] The server transmits the generated proposal to the terminal. The input is the generated proposal, and the output is the proposal transmitted to the terminal. In concrete terms, the server transmits the proposal to the wearable device.

[1624] Step 7:

[1625] The terminal visually displays the proposal. The input is the proposal sent from the server, and the output is the proposal displayed on the display. The specific operation is to display the proposal on the display of the wearable device and notify the user.

[1626] In this way, it is possible to collect user behavior data in real time and provide highly accurate visual suggestions.

[1627] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1628] "Example 1"

[1629] As one embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotions of a user. This system learns the user's routine work by time and season, makes action suggestions to the user based on the learned routine work, and transmits the suggested content to the user. Furthermore, the system recognizes the user's emotions and adjusts the action suggestions according to the emotions. Specifically, the emotion engine recognizes emotions from the user's tone of voice, facial expressions, behavior patterns, etc. For example, if the user speaks in a sad tone of voice, the emotion engine recognizes that the user is sad and transmits the information to the AI ​​engine. The AI ​​engine, after recognizing that the user is sad, makes action suggestions to the user. For example, it makes a suggestion such as "How about listening to relaxing music today?" In this way, the system of the present invention can make personalized suggestions that take the user's emotions into consideration.

[1630] "Example 2"

[1631] As one embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotions of a user. This system learns the user's routine work by time and season, makes action suggestions to the user based on the learned routine work, and transmits the suggested content to the user. Furthermore, the system recognizes the user's emotions and adjusts the action suggestions according to the emotions. Specifically, the emotion engine recognizes emotions from the user's tone of voice, facial expressions, behavior patterns, etc. For example, if the user speaks in a sad tone of voice, the emotion engine recognizes that the user is sad and transmits the information to the AI ​​engine. The AI ​​engine, after recognizing that the user is sad, makes action suggestions to the user. For example, it makes a suggestion such as "How about listening to relaxing music today?" In this way, the system of the present invention can make personalized suggestions that take the user's emotions into consideration.

[1632] The process flow of each embodiment will be described below.

[1633] "Example 1"

[1634] Step 1: The system learns the user's routine work by time and season. This learning is done from the user's behavioral history, selection history, etc.

[1635] Step 2: Based on the learned routine, the system will provide the user with action suggestions, which will vary according to the user's current time and season.

[1636] Step 3: The system recognizes the user's emotions using an emotion engine that recognizes the user's emotions from the user's tone of voice, facial expressions, behavioral patterns, etc.

[1637] Step 4: Based on the emotion engine's recognized emotion, the system will adjust the action suggestions. For example, if the user is sad, the system will suggest listening to relaxing music, etc., depending on the user's emotion.

[1638] Example 1

[1639] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1640] Conventional systems can learn a user's routine work by time and season and suggest actions based on that, but they cannot make personalized suggestions that take into account the user's emotions. This makes it difficult to provide optimal suggestions based on the user's emotional state.

[1641] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1642] In this invention, the server includes a means for learning the user's routine work by time and season, a means for making action suggestions to the user based on the learned routine work, a means for communicating the suggested content to the user, a means for recognizing the user's emotion, and a means for adjusting the action suggestions based on the recognized emotion, thereby enabling personalized suggestions according to the user's emotional state.

[1643] "User's routine work by time or season" refers to actions or habits that a user repeatedly performs at specific times of the day or during specific seasons.

[1644] "Learning means" refers to technologies and algorithms used to collect and analyze user behavior data to recognize patterns.

[1645] "Means for making action suggestions" refers to technologies and algorithms for suggesting optimal actions to users based on learned user routine work.

[1646] "Means for communicating suggestions to the user" refers to an interface or device for notifying the user of the generated action suggestions.

[1647] "Means of recognizing user emotions" refers to technologies and algorithms for analyzing and recognizing emotions from a user's tone of voice, facial expressions, behavioral patterns, etc.

[1648] "Means for tailoring action suggestions based on recognized emotions" refers to technologies or algorithms for appropriately modifying the content of action suggestions based on the user's emotional state.

[1649] The present invention is a system that learns a user's routine work by time and season, suggests actions based on the learned routine work, and further recognizes the user's emotions to adjust the suggested content. A specific embodiment of this system will be described below.

[1650] System Configuration

[1651] Hardware and Software

[1652] The server is equipped with an AI engine for collecting and analyzing user behavior data. This AI engine uses machine learning algorithms to learn the user's routine work. It also has an emotion engine that recognizes emotions from the user's tone of voice, facial expressions, behavioral patterns, etc. The terminal is a device such as a smartphone or tablet, and plays a role in conveying the suggestions from the server to the user.

[1653] Data collection and learning

[1654] The server collects data about the user's daily life. For example, the user inputs information such as "Wake up at 7am," "Eat breakfast at 8am," and "Get home at 6pm" through a smartphone app. This data is input into an AI engine, which analyzes the user's behavioral patterns and models routines by time and season.

[1655] Generate action suggestions

[1656] The server generates action suggestions for the user based on the learning results. For example, when a user asks, "What should I do today?", the server uses an AI engine to generate optimal suggestions. Specifically, it makes suggestions such as, "The weather is nice today, so how about going for a walk?"

[1657] Communicating the proposal

[1658] The device transmits the suggestions received from the server to the user. The user can receive the suggestions through a device such as a smartphone or tablet. For example, if the user asks "What should I eat today?" on their smartphone, the device will display "It would be good to eat a salad with lots of vegetables today."

[1659] Emotion recognition and suggestion adjustment

[1660] The server recognizes the user's emotions using an emotion engine. The emotion engine analyzes emotions from the user's tone of voice, facial expressions, behavioral patterns, etc. For example, if the user speaks in a sad tone of voice, the emotion engine recognizes the user as sad and transmits that information to the AI ​​engine. The server adjusts behavior suggestions based on the emotion information received from the emotion engine. For example, if the user is recognized as sad, the server makes a suggestion such as, "How about listening to some relaxing music today?"

[1661] Examples

[1662] Example 1: User questions and suggestions

[1663] When a user asks, "What should I eat today?", the server takes into account past data and the current season and suggests, "How about eating a salad with lots of vegetables today?" The device displays this suggestion to the user.

[1664] Example 2: Sentiment-based suggestions

[1665] When a user asks "What should I do today?" in a sad tone of voice, the emotion engine recognizes that the user is sad, and the server suggests "How about listening to some relaxing music today?" The device displays this suggestion to the user.

[1666] Examples of prompt statement...

Claims

1. A means for learning a routine work of a user at a predetermined timing; means for recognizing an emotion of the user; A means for suggesting an action to the user based on the learned routine work and emotion information indicating the recognized emotion of the user; A system including:

2. The means for making the action suggestion makes the action suggestion using a generative AI model based on a prompt sentence corresponding to the user's emotion. The system of claim 1 .

3. The predetermined timing is time or seasonal timing, The learning means collects and analyzes the user's daily life patterns. The system of claim 1 .

Images