System

Abstract

To provide a system.SOLUTION: A system includes: collection means for collecting data including an operation history, position information, and a communication history of a user; means for transmitting the collected data to a server; analysis means for analyzing the data received by the server to learn an action pattern of the user; prediction means for using a generative AI model to predict a next action on the basis of the learned action pattern; and support means for supporting task processing of the user on the basis of a prediction result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, people often check information on their smartphones and then use them to complete tasks. However, smartphones are not able to predict the user's next actions or mistakes from the information on the smartphone, which is a problem. [Means for solving the problem]

[0005] As a means of solving this problem, we propose a system in which AI in a smartphone collects, analyzes, and manages data in the smartphone, reducing the user's workload and preventing potential mistakes.Specifically, we provide a system that includes a means for collecting data in the smartphone, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis results, and a means for supporting the user in processing tasks based on the prediction results. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12]FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In one embodiment of the present invention, the AI ​​in the smartphone has a function of collecting data from the smartphone, such as a user's operation history, location information, and communication history. In addition, as a means for analyzing the collected data, the AI ​​has a function of learning the user's behavioral patterns and predicting the user's next action. Furthermore, as a means for supporting the user's task processing based on the prediction results, the AI ​​has a function of providing the user with suggestions for the next action and information on how to prevent mistakes.

[0029] "Example 2"

[0030] As a specific example, if the AI ​​learns that the user leaves for work at 8 a.m. every morning, the AI ​​will suggest that the user start preparing for work before 8 a.m. In addition, if the AI ​​learns that the user often makes the mistake of forgetting their phone during their commute, the AI ​​will notify the user before leaving to remember to take their phone with them.

[0031] The process flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: The AI ​​in the smartphone collects data such as the user's operation history, location information, and communication history.

[0034] Step 2: AI analyzes the collected data and learns user behavior patterns.

[0035] Step 3: Predict the user's next action based on the behavioral patterns the AI ​​has learned.

[0036] Step 4: Based on the prediction results, the AI ​​provides the user with suggestions for the next course of action and information on how to prevent mistakes.

[0037] "Example 2"

[0038] Step 1: The AI ​​learns the user's behavioral pattern: that they leave for work at 8am every morning.

[0039] Step 2: Just before 8am, the AI ​​suggests the user start getting ready for work.

[0040] Step 3: The AI ​​learns that users often make the mistake of forgetting their phone during their commute.

[0041] Step 4: Before leaving, the AI ​​notifies the user to remember to bring their phone.

[0042] Example 1

[0043] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0044] In conventional smartphone systems, it was difficult to effectively use data such as user operation history, location information, and communication history to predict user behavior and assist with task processing. In addition, there was a lack of appropriate means to learn the user's behavioral patterns and predict the next action, making it difficult to improve user convenience.

[0045] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0046] In this invention, the server includes a means for collecting data such as a user's operation history, location information, and communication history, a means for transmitting the collected data to the server, a means for the server to analyze the data received and learn the user's behavioral patterns, a means for predicting the user's next behavior based on the learned behavioral patterns, and a means for supporting the user's task processing based on the prediction results. This makes it possible to effectively predict the user's behavior and support the task processing at an appropriate time.

[0047] "User operation history" is a record of operations performed by a user on a smartphone.

[0048] "Location information" refers to information about a user's current location and movement history, obtained using a smartphone's GPS function or the like.

[0049] "Communication history" refers to records of calls and messages sent and received by a user via their smartphone.

[0050] A "server" is a computer system that collects, analyzes, stores, and transmits data over a network.

[0051] "Means of collecting data" refers to the function of obtaining data such as user operation history, location information, and communication history using sensors and software within the smartphone.

[0052] The "means for transmitting data to a server" is a function for transmitting collected data to a server via a network.

[0053] "Means for analyzing data" refers to a function for the server to process the data it receives and learn the user's behavioral patterns.

[0054] The "means for learning behavioral patterns" is a function that uses a machine learning algorithm to extract behavioral tendencies and patterns from the user's past data.

[0055] The "means for predicting the next action" is a function for predicting the next action that the user will take based on the learned behavioral patterns.

[0056] "Means to assist in task processing" is a function that provides the user with suggestions for the next course of action and information on how to prevent mistakes based on the prediction results.

[0057] The present invention relates to a system for assisting in task processing by collecting data in a smartphone, analyzing the data in a server, and predicting user behavior. A specific embodiment of this system will be described below.

[0058] First, the device collects data such as the user's operation history, location information, and communication history. This is done using the smartphone's built-in sensors, GPS, and communication module. For example, if a user frequently uses a particular app, its operation history is recorded.

[0059] Next, the terminal sends the collected data to the server. The data is encrypted and sent using a secure communication protocol (e.g., HTTPS). For example, the user's operation history and location information are sent to the server at regular intervals.

[0060] The server analyzes the received data. Specifically, it uses machine learning algorithms (e.g., TENSORFLOW (registered trademark) and PyTorch) to learn the user's behavioral patterns. For example, the server analyzes the user's operation history and finds a pattern of using a specific app at a specific time of day.

[0061] The server then predicts the user's next action based on the learned behavioral patterns. For example, the server predicts that the user will arrive at work at 8:00 every morning.

[0062] Finally, the server sends a notification to the device based on the prediction result. The device displays the user's next action suggestion and information on how to prevent mistakes. For example, the device displays a notification saying, "It's time to start getting ready for work."

[0063] Examples

[0064] Example 1: Collecting operation history

[0065] The device records the time when the user opens the SNS app. For example, if the user opens the SNS app at 10 p.m. every night, that time is recorded as the operation history.

[0066] Example 2: Sending data

[0067] The device sends the recorded operation history to the server. For example, data that indicates that a social networking app was opened at 10 p.m. every night is sent to the server.

[0068] Example 3: Data analysis

[0069] The server analyzes the operation history it receives and learns the pattern of how users use the SNS app every night at 10 p.m. For example, it models this pattern using TensorFlow.

[0070] Example 4: Behavioral prediction

[0071] Based on the patterns learned by the server, the server predicts the next time the user will open the SNS app. For example, the server predicts that "the user will next open the SNS app at 10 p.m."

[0072] Example 5: Task assistance

[0073] The server sends the prediction result to the device, and the device displays a notification to the user. For example, at 9:50 p.m., the device displays a notification saying, "SNS time is approaching."

[0074] Examples of prompt statements

[0075] "Design an AI system that collects user operation history, location information, and communication history, learns behavioral patterns, and predicts the next action. Specifically, implement a function that learns behavioral patterns using TensorFlow and sends a notification to the user based on the prediction results."

[0076] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0077] Step 1:

[0078] The device collects data such as the user's operation history, location information, and communication history.

[0079] Input: User operations, location information, communication history

[0080] Specific behavior: Using the smartphone's built-in sensors, GPS, and communication module, it records information such as which apps the user used and when, where they were, and who they communicated with.

[0081] Output: Collected data (operation history, location information, communication history)

[0082] Step 2:

[0083] The terminal transmits the collected data to the server.

[0084] Input: Collected data (operation history, location information, communication history)

[0085] What it does: Encrypts the data and sends it to the server using a secure communications protocol (e.g. HTTPS).

[0086] Output: Data sent to the server

[0087] Step 3:

[0088] The server parses the received data.

[0089] Input: Data sent to the server

[0090] Specific operation: Using machine learning algorithms (e.g. TensorFlow and PyTorch), the system learns user behavior patterns. For example, the server analyzes the user's operation history and finds a pattern of using a specific app at a specific time of day.

[0091] Output: Learned behavioral patterns

[0092] Step 4:

[0093] The server predicts the next action based on the behavioral patterns it has learned.

[0094] Input: Learned behavioral patterns

[0095] Specific behavior: Using a generative AI model, the server predicts what the user will do next. For example, the server predicts that the user will arrive at work at 8am every morning.

[0096] Output: Predicted next action

[0097] Step 5:

[0098] The server sends a notification to the terminal based on the prediction result.

[0099] Input: Predicted next action

[0100] Specific operation: The server sends the prediction result to the device, and the device displays the user's next action suggestion and information on how to prevent mistakes. For example, the device displays a notification saying, "It's time to start getting ready for work."

[0101] Output: The notification displayed to the user

[0102] (Application example 1)

[0103] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0104] Conventional food delivery systems do not fully utilize the user's order history and behavioral patterns, making it difficult to suggest menu items or send reminders to users at the right time. In addition, they lack the ability to learn the user's preferences and behavioral patterns, so there is a need to improve the user experience.

[0105] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0106] In this invention, the server includes means for collecting data in the smartphone, means for analyzing the collected data, means for predicting the user's next action based on the analysis result, means for supporting the user's task processing based on the prediction result, means for collecting the user's order history, location information, and communication history, and learning the user's behavioral patterns, means for predicting the next order based on the learned behavioral patterns, means for providing recommended menus and promotion information at appropriate times, and means for sending reminders at specific times, thereby enabling personalized suggestions and reminders to be sent based on the user's preferences and behavioral patterns.

[0107] A "smartphone" is a multi-function mobile device that, in addition to the functions of a mobile phone, can also connect to the Internet and use applications.

[0108] "Means for collecting data" refers to functions and devices for acquiring information such as a user's operation history, location information, and communication history.

[0109] "Means for analyzing data" refers to functions or devices for analyzing collected data and understanding user behavior patterns and trends.

[0110] The "means for predicting the user's next action" refers to a function or device for predicting the action that the user is likely to take next based on the analysis results.

[0111] The "means for supporting the user's task processing" refers to a function or device for supporting the user so that the user can efficiently carry out a task based on the prediction result.

[0112] "Order history" is a record of orders a user has placed in the past.

[0113] "Location information" is information relating to the user's current location and past movement history.

[0114] "Communication history" is a record of calls and messages made by a user.

[0115] A "means for learning behavioral patterns" is a function or device for learning the behavioral tendencies and habits of a user using collected data.

[0116] The "means for predicting the next order" refers to a function or device for predicting the next product that the user is likely to order based on the learned behavioral patterns.

[0117] The "means for providing recommended menu items and promotion information" refers to a function or device for presenting appropriate menu items and discount information to the user based on the prediction results.

[0118] A "means for sending a reminder" is a function or device for sending a notification to a user at a specific time.

[0119] As an embodiment of the present invention, a system is provided in which a user's order history, location information, and communication history in a food delivery system are collected, and an AI analyzes the data to learn the user's behavioral patterns. A specific embodiment of this system is described below.

[0120] System Configuration

[0121] The system includes a smartphone, a server, and an AI model. The smartphone is hardware for collecting users' operation history, location information, and communication history. The server runs software for analyzing the collected data and learning users' behavioral patterns. The AI ​​model is used to predict the user's next action and provide recommended menus and promotional information at the appropriate time.

[0122] Collecting data

[0123] Smartphones collect users' operation history, location information, and communication history in real time. This includes GPS sensors, communication modules, and application log data. The collected data is sent to a server.

[0124] Analyzing data

[0125] The server receives the collected data and analyzes it using an AI model. Specifically, it uses programming languages ​​such as Python to pre-process the data, extract features, and train the model. The AI ​​model learns the user's behavioral patterns and predicts the next order.

[0126] Predicting and suggesting actions

[0127] Based on the learned behavioral patterns, the AI ​​model predicts what the user is likely to order next. Based on the prediction results, the server sends menu recommendations and promotion information to the smartphone. It also includes a function to send reminders at specific times.

[0128] Examples

[0129] For example, if a user has ordered "pizza," "sushi," "ramen," and "curry" in the past, the AI ​​model might recommend "pizza" next. It also sends a reminder at 12 o'clock saying, "It's lunchtime! Order your favorite menu!"

[0130] Examples of prompt statements

[0131] Here is an example of a prompt to be input to the generative AI model:

[0132] Use the user's past orders, location, and communication history to predict what their next menu recommendation will be, and add the ability to send them a reminder at a specific time.

[0133] In this way, personalized offers and reminders can be sent based on the user's preferences and behavioral patterns.

[0134] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0135] Step 1:

[0136] The smartphone collects the user's operation history, location information, and communication history. Specifically, it obtains location information using a GPS sensor, collects operation history from application log data, and records communication history through a communication module. This data is sent to a server in real time.

[0137] Input: User operation history, location information, communication history

[0138] Output: Sending collected data to a server

[0139] Step 2:

[0140] The server receives the data sent from the smartphone. The received data is stored in a database. The database organizes and stores data for each user.

[0141] Input: Data sent from smartphone

[0142] Output: Data stored in the database

[0143] Step 3:

[0144] The server preprocesses the data stored in the database, specifically by imputing missing values, removing outliers, normalizing the data, etc. This improves the quality of the data and makes it easier for subsequent analysis.

[0145] Input: Raw data stored in a database

[0146] Output: Preprocessed data

[0147] Step 4:

[0148] The server uses the preprocessed data to train the AI ​​model. Specifically, it uses a programming language such as Python to perform feature extraction, model selection, and model training. The AI ​​model learns the user's behavioral patterns.

[0149] Input: Preprocessed data

[0150] Output: A trained AI model

[0151] Step 5:

[0152] The server uses the trained AI model to predict the user's next action, specifically, guessing what menu item the user is likely to order next based on their past ordering history and behavioral patterns.

[0153] Input: Trained AI model, user's past order history

[0154] Output: Next order prediction result

[0155] Step 6:

[0156] Based on the prediction results, the server sends recommended menus and promotion information to the smartphone, specifically providing menus tailored to the user's preferences and currently available promotion information.

[0157] Input: Predicted result for next order

[0158] Output: Menu recommendations and promotional information sent to your smartphone

[0159] Step 7:

[0160] The server sends a reminder at a specific time. Specifically, the server sends a reminder to the smartphone at 12 o'clock saying "It's lunchtime! Order your favorite menu!" so that the user does not forget to order at lunchtime.

[0161] Input: current time, user behavior pattern

[0162] Output: Reminders sent to your smartphone

[0163] These steps enable personalized suggestions and reminders based on the user's preferences and behavioral patterns.

[0164] Example 2

[0165] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0166] Conventional smartphones and wearable devices can collect user behavior data, but there are only a limited number of systems that effectively use that data to support user behavior. In particular, there is a demand for systems that can learn user behavior patterns and notify users at the appropriate time to reduce user errors.

[0167] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting user behavior data, a means for transmitting the collected data to the server, a means for analyzing the data in the server and learning the user behavior pattern using an AI model, a means for generating notification content based on the learned behavior pattern, a means for transmitting the generated notification content to the terminal, and a means for displaying the notification on the terminal. This enables notification at an appropriate time based on the user's behavior pattern, and reduces user errors.

[0168] "User behavioral data" refers to information related to a user's behavior, such as the user's daily behavior, movement patterns, and application usage history.

[0169] "Server" refers to a computer system that receives user behavioral data, analyzes it, and trains the AI ​​model.

[0170] "AI model" refers to a model that uses machine learning algorithms to learn user behavior patterns and generate predictions and notifications.

[0171] A "behavioral pattern" refers to a tendency for a user to repeatedly perform an action at a particular time or in a particular situation.

[0172] "Notification content" refers to messages or alerts that are generated to prompt the user to take action.

[0173] "Terminal" refers to a device that is directly operated by a user, such as a smartphone or tablet.

[0174] "Notification" means a message or alert that is displayed to the User via a Device.

[0175] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0176] The present invention is a system that collects user behavior data and analyzes it using an AI model to provide notifications based on the user's behavior patterns. A specific embodiment of this system will be described below.

[0177] Collecting user behavior data

[0178] Users collect daily behavioral data using smartphones and wearable devices. For example, the GPS function of a smartphone is used to record the user's movement patterns. Data from calendar apps and reminder apps is also collected.

[0179] Sending behavioral data to the server

[0180] The device periodically transmits the collected behavioral data to a server, where the data is encrypted and transmitted using a secure communication protocol (e.g. HTTPS).

[0181] Data analysis on the server and training of AI models

[0182] The server analyzes the received behavioral data and uses an AI model to learn the user's behavioral patterns. Specifically, it uses Python and leverages machine learning libraries such as TensorFlow and PyTorch. For example, it learns that a user leaves for work at 8am every morning.

[0183] Generate notification content

[0184] The server generates notifications based on learned behavioral patterns. For example, it generates a notification "Start getting ready for work" when the user should start getting ready for work (7:50). Also, if the user tends to forget their phone, it generates a notification "Don't forget to take your phone with you."

[0185] Sending notifications to devices

[0186] The server then sends the generated notification content to the terminal. The notification is sent in real time so that the user can receive it at the appropriate time.

[0187] Displaying notifications on your device

[0188] The device will display the received notifications to the user. Notifications will be displayed as pop-ups or push notifications so that the user can check them immediately.

[0189] Examples

[0190] As a concrete example, let's consider a case where the AI ​​has learned a user's behavioral pattern of leaving for work at 8:00 every morning. The AI ​​will notify the user at 7:50, "Please start preparing for your commute." Also, if the user has forgotten their phone many times in the past, the AI ​​will notify them, "Don't forget to take your phone with you."

[0191] Examples of prompt statements

[0192] An example of a prompt sentence might be:

[0193] Example prompt:

[0194] Design an AI system that learns user behavior patterns and notifies them at the right time. If a user leaves for work at 8am every morning, include a feature that notifies them to start getting ready for work at 7:50am and reminds users who tend to forget their phone to take it with them before leaving.

[0195] By inputting this prompt sentence into a generative AI model, a notification system design based on the user's behavioral patterns is generated.

[0196] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0197] Program processing flow

[0198] Step 1: Collect user behavior data

[0199] Users collect daily behavioral data using smartphones and wearable devices. Specifically, the GPS function of the smartphone is used to record the user's movement patterns. Data from calendar apps and reminder apps is also collected. The input is the user's behavioral data, and the output is the collected behavioral data.

[0200] Step 2: Sending behavioral data to the server

[0201] The device periodically transmits the collected behavioral data to the server. The data is encrypted and transmitted using a secure communication protocol (e.g., HTTPS). The input is the collected behavioral data, and the output is the behavioral data transmitted to the server.

[0202] Step 3: Data analysis on the server and training of the AI ​​model

[0203] The server analyzes the received behavioral data and uses an AI model to learn the user's behavioral patterns. Specifically, it uses Python to leverage machine learning libraries such as TensorFlow and PyTorch. The input is the behavioral data sent to the server, and the output is the learned behavioral pattern. For example, it learns a pattern that a user leaves for work at 8am every morning.

[0204] Step 4: Generate notification content

[0205] The server generates notification content based on the learned behavioral patterns. For example, it generates a notification "Start preparing for your commute" when the user should start preparing for their commute (7:50). Also, if the user tends to forget their phone, it generates a notification "Don't forget to take your phone with you." The input is the learned behavioral patterns, and the output is the generated notification content.

[0206] Step 5: Sending notifications to devices

[0207] The server sends the generated notification content to the terminal. The notification is sent in real time so that the user can receive it at the appropriate time. The input is the generated notification content, and the output is the notification content sent to the terminal.

[0208] Step 6: Display notifications on your device

[0209] The terminal displays the received notifications to the user. Notifications are displayed as popups or push notifications so that the user can see them immediately. The input is the notification content sent to the terminal, and the output is the notification displayed to the user.

[0210] Examples

[0211] As a concrete example, let's consider a case where the AI ​​has learned a user's behavioral pattern of leaving for work at 8:00 every morning. The AI ​​will notify the user at 7:50, "Please start preparing for your commute." Also, if the user has forgotten their phone many times in the past, the AI ​​will notify them, "Don't forget to take your phone with you."

[0212] (Application example 2)

[0213] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0214] Conventional user support systems using smart devices can learn the user's behavioral patterns and predict the user's next actions, but due to lack of coordination with the automobile, which is the user's means of transportation, they could not sufficiently improve the user's convenience. In addition, there was a lack of concrete measures to prevent users from forgetting things. This often caused inconvenience to users when commuting or traveling.

[0215] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0216] In this invention, the server includes a means for collecting data in the smart device, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for controlling the functions of the car based on the user's behavioral pattern, and a means for notifying the user of the user's forgotten items. This makes it possible to automatically control the functions of the car based on the user's behavioral pattern and prompt the user to make necessary preparations for commuting or traveling. In addition, by providing a notification function to prevent the user from forgetting things, the convenience of the user can be significantly improved.

[0217] A "smart device" is an electronic device that can be carried by a user and has functions such as data collection, analysis, and communication.

[0218] "Means of collecting data" refers to the function of obtaining information from various sensors and applications within smart devices.

[0219] "Means for analyzing data" refers to the function of processing collected data and analyzing user behavior patterns and trends.

[0220] "Means for predicting the user's next action" refers to a function for predicting the action that the user is likely to take next based on the analysis results.

[0221] "Means for supporting user task processing" refers to a function that supports the user to efficiently complete a task based on predicted behavior.

[0222] "Means for controlling the functions of an automobile" refers to a function for automatically operating functions such as starting the automobile engine and setting the air conditioning based on the user's behavioral patterns.

[0223] "Means for notifying a user of lost items" refers to a feature that notifies the user before departure to ensure that the user does not forget a particular item.

[0224] The system for carrying out this invention collects and analyzes data in smart devices, learns the user's behavioral patterns, predicts the user's next actions, and assists the user in processing tasks. It also has the function of controlling the functions of a car based on the user's behavioral patterns and notifying the user of lost items.

[0225] System Configuration

[0226] The system includes the following major hardware and software components:

[0227] Hardware: Smart devices (smartphones, smart glasses, head-mounted displays, etc.), infotainment systems for autonomous vehicles

[0228] Software: Python, smtplib (email sending library), data analysis libraries (e.g. Pandas, NumPy), machine learning libraries (e.g. scikit-learn)

[0229] Data collection and analysis

[0230] The server collects data from various sensors and applications in smart devices. The collected data is processed using a data analysis library to analyze user behavior patterns and trends. For example, a user's behavior pattern of commuting to work every morning at 8 o'clock is detected.

[0231] Behavioral prediction and task support

[0232] The server predicts the user's next action based on the analysis results. Based on the predicted action, the server supports the user to efficiently complete tasks. For example, when it is time to commute, the server notifies the user to prepare for commuting.

[0233] Automotive Function Control

[0234] The system automatically controls the functions of the car based on the user's behavioral patterns, for example starting the car's engine and setting the air conditioner to an appropriate temperature when it is time to commute to work.

[0235] Lost property notification

[0236] By registering items that a user often forgets in advance, the system will notify the user to bring these items before leaving. For example, if a user often forgets their phone and wallet, the system will notify the user to bring these items before leaving.

[0237] Examples

[0238] As a concrete example, consider a user who commutes to work every morning at 8am and often forgets their phone and wallet. In this case, the system will start the engine before 8am, set the air conditioner, and notify the user to prepare for work. It will also notify the user to not forget their phone and wallet.

[0239] Examples of prompt statements

[0240] Here is an example of a prompt to be input to the generative AI model:

[0241] Design an AI system that learns the user's behavioral pattern of commuting to work every morning at 8:00, and then starts the engine before 8:00, sets the air conditioner to an appropriate temperature, and notifies the user to prepare for the commute. Also, if the user often forgets his / her phone and wallet during the commute, add a feature that notifies the user to have these items with him / her before leaving.

[0242] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0243] Step 1:

[0244] The server collects data from various sensors and applications within the smart device.

[0245] Input: Sensor data from smart devices (location, time, application usage history, etc.)

[0246] Output: Raw data collected

[0247] Specific operation: The server communicates with the smart device, obtains location information from the GPS sensor, and collects application usage history.

[0248] Step 2:

[0249] The server processes the collected data using a data analysis library to analyze user behavior patterns.

[0250] Input: Raw data collected

[0251] Output: Analyzed behavioral patterns (e.g. commuting to work at 8am every morning)

[0252] Specific operation: The server uses data analysis libraries such as Pandas and NumPy to organize the collected data in chronological order and extract user behavior patterns.

[0253] Step 3:

[0254] The server predicts the user's next action based on the analysis results.

[0255] Input: Analyzed behavioral patterns

[0256] Output: Predicted next action (e.g., start commuting at 8am)

[0257] Specific operation: The server uses a machine learning model (e.g., scikit-learn) to predict the next action based on past behavior patterns.

[0258] Step 4:

[0259] The server assists the user in processing tasks based on the predicted behavior.

[0260] Input: Predicted next action

[0261] Output: Notification to the user (e.g., notification to prepare for commute)

[0262] Specific operation: The server sends a notification to the user's smart device and displays a message encouraging them to prepare for their commute.

[0263] Step 5:

[0264] The server controls the functions of the vehicle based on the user's behavioral patterns.

[0265] Input: Predicted next action

[0266] Output: Car control instructions (e.g. engine start, air conditioning settings)

[0267] What it does: The server communicates with the autonomous vehicle's infotainment system, starts the engine, and sets the air conditioning to the right temperature.

[0268] Step 6:

[0269] The server notifies the user of items that he or she often forgets.

[0270] Input: A list of lost items registered by the user (e.g. phone, wallet)

[0271] Output: Lost item notification content

[0272] Specific operation: The server sends a notification to the user's smart device and displays a message reminding the user to have their phone and wallet ready before leaving.

[0273] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0274] "Example 1"

[0275] In one embodiment of the present invention, the AI ​​in the smartphone collects data such as the user's operation history, location information, and communication history. Furthermore, the collected data is analyzed to learn the user's behavioral patterns. Then, based on the learned behavioral patterns, the AI ​​predicts the user's next action. Based on the prediction result, the AI ​​provides the user with suggestions for the next action and information on preventing mistakes. Furthermore, in one embodiment of the present invention, the AI ​​includes an emotion engine. The emotion engine recognizes emotions from the user's tone of voice and facial expressions. Based on the recognized emotions, the AI ​​more appropriately supports the user in processing tasks. For example, if the AI ​​recognizes that the user is feeling stressed, the AI ​​supports the user in reducing stress by playing relaxation music and providing a message encouraging deep breathing.

[0276] "Example 2"

[0277] In another embodiment of the present invention, the AI ​​in the smartphone learns a behavioral pattern that the user leaves for work at 8 o'clock every morning. Before 8 o'clock, the AI ​​suggests to the user to start preparing for work. Furthermore, the AI ​​learns information that the user often makes the mistake of forgetting his / her phone during the commute. Before leaving, the AI ​​notifies the user not to forget to take the phone. Furthermore, in one embodiment of the present invention, the AI ​​includes an emotion engine. The emotion engine recognizes emotions from the user's tone of voice and facial expressions. Based on the recognized emotions, the AI ​​better supports the user in processing tasks. For example, if the AI ​​recognizes that the user is feeling impatient or anxious, the AI ​​reduces the user's anxiety by speeding up the reminder notification time.

[0278] The process flow of each embodiment will be described below.

[0279] "Example 1"

[0280] Step 1: The AI ​​collects data such as the user's operation history, location information, communication history, etc. Step 2: The AI ​​analyzes the collected data and learns the user's behavioral patterns.

[0281] Step 3: Predict the user's next action based on learned behavioral patterns.

[0282] Step 4: Based on the prediction results, the AI ​​provides the user with suggestions for next steps and information on how to prevent mistakes.

[0283] Step 5: The emotion engine recognizes emotions from the user's tone of voice and facial expressions.

[0284] Step 6: Based on the recognized emotions, the AI ​​can better assist the user in completing tasks. For example, if the AI ​​recognizes that the user is feeling stressed, it can help the user reduce stress by playing relaxation music or providing messages encouraging deep breathing.

[0285] "Example 2"

[0286] Step 1: The AI ​​learns the user's behavioral pattern: that they leave for work at 8am every morning.

[0287] Step 2: Just before 8am, the AI ​​suggests the user start getting ready for work.

[0288] Step 3: The AI ​​learns that users often make the mistake of forgetting their phone during their commute. Before leaving, the AI ​​will notify the user to remember to take their phone with them.

[0289] Step 4: The emotion engine recognizes emotions from the user's tone of voice and facial expressions.

[0290] Step 5: Based on the recognized emotions, the AI ​​can better assist the user in completing the task. For example, if the AI ​​recognizes that the user is feeling anxious or impatient, it can speed up the reminder notification time to reduce the user's anxiety.

[0291] Example 1

[0292] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0293] Conventional smartphone systems can collect data such as user operation history, location information, and communication history, but it is difficult to effectively use this data to predict user behavior and provide appropriate task support. In addition, there was no system that recognized the user's emotions and provided task support based on them. This made it difficult to reduce user stress and mistakes, and improving the user experience was a challenge.

[0294] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0295] In this invention, the server includes a means for collecting data such as a user's operation history, location information, and communication history, a means for analyzing the collected data and learning the user's behavioral patterns, a means for predicting the user's next behavior based on the learned behavioral patterns, a means for supporting the user's task processing based on the prediction results, a means for recognizing emotions from the user's tone of voice and facial expressions, and a means for appropriately supporting the user's task processing based on the recognized emotions. This makes it possible to predict the user's behavior and provide appropriate task support, reducing user stress and mistakes and improving the user experience.

[0296] "User operation history" is a record of operations performed by a user on a smartphone, and is data including the time spent using an app and the location of taps.

[0297] "Location information" is data about a user's current location and movement history obtained using a smartphone's GPS function, etc.

[0298] "Communication history" refers to records of calls and messages sent and received by a user, and is data including call history and message contents.

[0299] "Means of collecting data" refers to the function of obtaining data such as user operation history, location information, and communication history using sensors and communication modules within the smartphone.

[0300] "Means for analyzing data" refers to functions that use machine learning algorithms and databases to analyze collected data and extract user behavior patterns.

[0301] "Means for learning behavioral patterns" refers to a function that uses a machine learning algorithm to extract patterns from a user's past behavioral data and generate a model for predicting the user's next behavior.

[0302] "Means for predicting the next action" refers to a function for predicting the next action that the user is likely to take based on learned behavioral patterns.

[0303] "Means to assist in task processing" refers to functions that suggest the user's next course of action based on prediction results or provide information to prevent mistakes.

[0304] "Means for recognizing emotions" refers to functionality that uses voice recognition software and a camera module to analyze the user's tone of voice and facial expressions to determine the user's emotional state.

[0305] "Means for appropriately supporting task processing based on emotions" refers to a function that provides assistance such as playing relaxation music or providing a message encouraging the user to take a deep breath, depending on the recognized emotions.

[0306] This invention is a system that uses AI in a smartphone to collect data such as a user's operation history, location information, and communication history, analyzes this data, learns the user's behavioral patterns, and predicts the user's next actions. Furthermore, it has a function to recognize the user's emotions from their tone of voice and facial expressions, and appropriately support the user's task processing based on the recognized emotions.

[0307] Data collection

[0308] When a user operates a smartphone, the device collects data such as the user's operation history, location information, and communication history. This data is collected using hardware such as the smartphone's built-in sensors, GPS, and communication modules. In addition, the smartphone's storage and dedicated database software are used to collect and store the data.

[0309] Analyzing data and learning behavioral patterns

[0310] The data collected by the device is sent to a server. The server stores the received data in a database and analyzes the data using machine learning algorithms. Specifically, frameworks such as TensorFlow and PyTorch are used to extract patterns from the user's past behavior data and generate a model to predict the user's next behavior.

[0311] Behavioral prediction and suggestions

[0312] The server sends the generated prediction model to the device. The device uses the prediction model to predict the user's next action. For example, if a user has the habit of opening a specific app at 8am every morning, the AI ​​will suggest automatically opening the app at that time. Also, if a user has the habit of calling a specific contact when they are in a specific location, the AI ​​will suggest making the call when they arrive at that location.

[0313] Emotion Recognition and Task Assistance

[0314] When a user is using a smartphone, the device analyzes the user's tone of voice and facial expressions using an emotion engine. Voice recognition software and a camera module are used for this emotion recognition. For example, Google (registered trademark) Cloud Speech-to-Text API and OpenCV are used. Based on the recognized emotion, the AI ​​assists the user in completing tasks. For example, if it recognizes that the user is feeling stressed, it plays relaxation music and provides a message encouraging deep breathing.

[0315] Examples and prompts

[0316] As a concrete example, suppose a user has the habit of opening the calendar app at 8 am every morning. The AI ​​learns this behavioral pattern and suggests opening the calendar app at 8 am every morning. Also, if the AI ​​recognizes that the user is feeling stressed, it will play relaxation music.

[0317] Examples of prompts include, "If the user has the habit of opening the calendar app at 8am every morning, the AI ​​will suggest opening the calendar app at that time. Also, if it recognizes that the user is feeling stressed, it will suggest playing relaxation music."

[0318] In this way, the AI ​​in the smartphone learns the user's behavioral patterns and predicts and suggests the next action to assist the user in completing tasks. In addition, it recognizes the user's emotions using an emotion engine and provides appropriate assistance.

[0319] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0320] Step 1:

[0321] The user operates the smartphone.

[0322] When a user opens an app or sends a message, data such as operation history, location information, and communication history is generated. This data becomes the input.

[0323] Step 2:

[0324] The device collects data such as the user's operation history, location information, and communication history.

[0325] The device collects data in real time using built-in sensors, GPS, and communication modules. The collected data is stored in the internal storage, which is the output.

[0326] Step 3:

[0327] The terminal transmits the collected data to the server.

[0328] The terminal transmits the collected data to the server at regular intervals using the HTTPS protocol. The transmitted data becomes the input.

[0329] Step 4:

[0330] The server stores the received data in a database.

[0331] The server stores the received data in a database (e.g., MySQL (registered trademark), PostgreSQL). The stored data becomes the output.

[0332] Step 5:

[0333] The server analyzes the data using machine learning algorithms.

[0334] The server uses frameworks such as TensorFlow and PyTorch to analyze the stored data. Specifically, it performs clustering and classification to extract user behavior patterns. The analysis results are output.

[0335] Step 6:

[0336] The server learns user behavior patterns and generates a predictive model.

[0337] Based on the analysis results, the server generates a model to predict the user's next action. This model is the output.

[0338] Step 7:

[0339] The server sends the generated predictive model to the terminal.

[0340] The server uses a secure communication protocol (e.g. SSL / TLS) to send the prediction model to the terminal. The sent model becomes the input.

[0341] Step 8:

[0342] The device uses the predictive model to predict the user's next action.

[0343] The device uses the received prediction model to predict the user's next action in real time. For example, if the user has the habit of opening the calendar app at 8:00 every morning, the device will suggest opening the calendar app before 8:00. The prediction result is the output.

[0344] Step 9:

[0345] The device suggests the user's next action based on the prediction results.

[0346] Based on the prediction results, the device will suggest the next action to the user through notifications or pop-up messages. For example, it may display a message such as "It's 8 o'clock. Would you like to open the calendar app?". The suggestions are the output.

[0347] Step 10:

[0348] When a user uses a smartphone, the device analyzes the user's tone of voice and facial expressions using an emotion engine.

[0349] The device uses a built-in camera and microphone to analyze the user's tone of voice and facial expressions in real time, and the analysis results become input.

[0350] Step 11:

[0351] The device recognizes the user's emotions based on the analysis results.

[0352] The device uses voice recognition software and a camera module (e.g., Google Cloud Speech-to-Text API, OpenCV) to recognize the user's emotions. The recognition results are the output.

[0353] Step 12:

[0354] The device provides appropriate task assistance based on the emotions it recognizes.

[0355] Depending on the emotion recognized, the device will play relaxation music or display a message encouraging deep breathing. For example, if it recognizes that the user is feeling stressed, it will play relaxation music. The support content is the output.

[0356] (Application example 1)

[0357] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0358] Conventional food delivery applications do not take into account the user's behavioral patterns or emotions when making suggestions, making it difficult to improve user satisfaction. In addition, if the user is feeling stressed, the application may not be able to respond appropriately, resulting in a poor user experience.

[0359] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0360] In this invention, the server includes a means for collecting data in the smartphone, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for recognizing the user's emotion, and a means for making an appropriate suggestion to the user based on the recognized emotion. This makes it possible to make a suggestion that takes into account the user's behavioral patterns and emotions, thereby improving the user's satisfaction. In addition, when the user is feeling stressed, a relaxation menu or a special offer can be provided to improve the user experience.

[0361] "Data within a smartphone" refers to various information collected through a smartphone, such as a user's operation history, location information, and communication history.

[0362] "Means of collection" refers to the functions and processes for acquiring and storing data within a smartphone.

[0363] "Means of analysis" refers to the functions and processes for analyzing collected data and understanding user behavior patterns and trends.

[0364] The "means for predicting the user's next action" refers to a function or process for estimating the action that the user is likely to take next based on the analysis results.

[0365] The "means for supporting user task processing" refers to functions and processes for supporting the user to efficiently perform tasks based on the prediction results.

[0366] "Means for recognizing the user's emotions" refers to a function or process for determining the emotions of the user from the user's tone of voice, facial expression, etc.

[0367] The "means for making appropriate suggestions to the user based on the recognized emotions" refers to a function or process for providing optimal actions or suggestions depending on the user's emotional state.

[0368] The system for realizing the present invention has a series of functions for collecting and analyzing data in a smartphone, predicting the user's next action, and making appropriate suggestions. A specific embodiment of this system will be described below.

[0369] 1. Data Collection

[0370] The server uses the means for collecting data within the smartphone to obtain data such as the user's operation history, location information, communication history, etc. This data collection is done automatically from various sensors and applications on the smartphone.

[0371] 2. Data Analysis

[0372] The collected data is analyzed by an analysis method in the server. This analysis method is implemented using a programming language such as Python and includes an algorithm for learning the user's behavioral patterns. Specifically, the method grasps the user's behavioral tendencies based on past operation history and location information.

[0373] 3. Behavioral Prediction

[0374] The server includes a means for predicting the user's next action based on the analysis results. This prediction means is implemented using a machine learning model to estimate the action that the user is likely to take next. For example, if past data shows that the user tends to visit a specific restaurant at a specific time, the server suggests that restaurant.

[0375] 4. Task processing support

[0376] The server has a means to assist the user in processing tasks based on the prediction results. Specifically, the server suggests appropriate restaurants and menus so that the user can perform tasks efficiently. If the user feels stressed, the server provides relaxation menus and special offers.

[0377] 5. Emotion recognition

[0378] The server is equipped with a means for recognizing the user's emotions. This emotion recognition is performed by analyzing the user's tone of voice and facial expressions. An emotion recognition engine (EmotionEngine) is used to determine whether the user is feeling stressed.

[0379] 6. Providing suggestions

[0380] The server has a means for making appropriate suggestions to the user based on the recognized emotion. For example, if the user feels stressed, the server can suggest a relaxation menu. As a concrete example, the following prompt sentence can be used:

[0381] Examples of prompt statements

[0382] "If the user is feeling stressed, suggest a relaxation menu."

[0383] In this way, the system can improve user satisfaction by making suggestions that take into account the user's behavioral patterns and emotions. Also, if the user is feeling stressed, the system can improve the user experience by providing relaxation menus and special offers.

[0384] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0385] Step 1:

[0386] The server collects data from the smartphone. Specifically, it automatically obtains data such as the user's operation history, location information, and communication history from various sensors and applications on the smartphone. The input is data from the smartphone, and the output is a set of collected data.

[0387] Step 2:

[0388] The server analyzes the collected data. Using a programming language such as Python, it executes an algorithm to learn the user's behavioral patterns. The input is the data collected in step 1, and the output is the analysis results on the user's behavioral patterns. Specifically, it analyzes the user's behavioral tendencies based on past operation history and location information.

[0389] Step 3:

[0390] The server predicts the user's next action based on the analysis results. A machine learning model is used to estimate the action the user is likely to take next. The input is the analysis result from step 2, and the output is the predicted next action. Specifically, if past data shows that the user has a tendency to use a specific restaurant at a specific time of day, the server suggests that restaurant.

[0391] Step 4:

[0392] The server assists the user in processing tasks based on the prediction results. Specifically, it suggests appropriate restaurants and menus so that the user can perform the task efficiently. The input is the prediction result from step 3, and the output is the suggestions to the user. Specifically, it notifies the user of recommended restaurants and menus.

[0393] Step 5:

[0394] The server recognizes the user's emotions. By analyzing the user's tone of voice and facial expressions, an emotion recognition engine (EmotionEngine) is used to determine whether the user is feeling stressed. The input is the user's tone of voice and facial expression data, and the output is the recognized emotional state. Specific operations include voice analysis and image analysis.

[0395] Step 6:

[0396] The server makes appropriate suggestions to the user based on the recognized emotions. If the user is feeling stressed, it will provide a relaxation menu or a special offer. The input is the emotion recognition result from step 5, and the output is the content of the suggestions to the user. Specifically, it will suggest a relaxation menu or notify the user of a special offer.

[0397] In this way, the server can make suggestions that take into account the user's behavioral patterns and emotions through a series of processing steps, thereby improving the user's satisfaction.

[0398] Example 2

[0399] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0400] Conventional smart devices have limited functionality for learning the user's behavioral patterns and predicting the user's next actions, and they do not provide appropriate notifications or support that take into account the user's emotional state. In particular, when the user is feeling impatient or anxious, there is a problem that notifications are not provided at the appropriate time, making it difficult to reduce the user's stress.

[0401] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0402] In this invention, the server includes a means for collecting data in the smart device, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for recognizing the user's emotion, and a means for adjusting the notification time based on the recognized emotion. This makes it possible to not only learn the user's behavioral pattern and predict the user's next action, but also to notify and support the user at an appropriate time taking into account the user's emotional state.

[0403] A "smart device" is an electronic device that can be carried by a user and has functions such as data collection, analysis, and notification.

[0404] "Means of collecting data" refers to the function of obtaining information about user behavior and the environment from various sensors and applications within smart devices.

[0405] "Means for analyzing data" refers to algorithms or software used to analyze collected data and identify user behavior patterns and trends.

[0406] The "means for predicting the user's next action" is a function for predicting the next action that the user should take based on the analysis results.

[0407] The "means for supporting the user in processing a task" is a function that supports the user to efficiently complete a task based on predicted behavior.

[0408] "Means for recognizing the user's emotions" refers to technology for detecting the user's emotional state from the user's tone of voice, facial expressions, etc.

[0409] The "means for adjusting notification time" is a function that optimizes the timing of notifications to the user based on the recognized emotional state.

[0410] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0411] The present invention relates to a notification system that combines learning of a user's behavioral patterns and emotion recognition in a smart device. A specific embodiment of this system will be described below.

[0412] 1. Generating the system program

[0413] The server generates an AI model to learn the user's behavioral patterns. This AI model is installed on the user's smart device. Machine learning algorithms are used to generate the AI ​​model. Specifically, programming languages ​​and libraries such as Python and TensorFlow are used.

[0414] 2. Explanation of program processing

[0415] The terminal (smart device) uses a built-in AI model to learn the user's behavioral patterns. This AI model collects the user's location information and schedule data, and analyzes the behavioral patterns based on this. Specifically, it collects data using the smart device's GPS function and calendar app.

[0416] Just before 8am, the device will notify the user to start getting ready for work, using the smart device's push notification function.

[0417] Additionally, the AI ​​model will learn from users' common mistake of forgetting their phone during their commute and will remind them to remember to take their phone with them before they leave.

[0418] In one embodiment of the present invention, the AI ​​in the terminal includes an emotion engine. The emotion engine recognizes emotions from the user's tone of voice and facial expressions. For example, the emotion engine uses the camera and microphone of the smart device to analyze the user's facial expressions and tone of voice. Based on the recognized emotions, the AI ​​better assists the user in processing tasks. For example, if it recognizes that the user is feeling impatient or anxious, it advances the reminder notification time.

[0419] 3. Adding concrete examples

[0420] As a concrete example, if the AI ​​has learned that a user leaves for work at 8am every morning, the following prompts can be input to the generative AI model:

[0421] Example prompt:

[0422] "An AI that has learned the user's behavioral pattern of leaving for work at 8am every morning will notify the user to start preparing for work before 8am. Also, an AI that has learned the user's common mistake of forgetting their phone during the commute will notify the user not to forget to take their phone with them before leaving."

[0423] By inputting this prompt text into a generative AI model, the AI ​​will generate appropriate notifications based on the user's behavioral patterns.

[0424] As described above, the present invention provides a notification system that takes into account a user's behavioral patterns and emotional state, making the user's life more efficient and comfortable.

[0425] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0426] Step 1:

[0427] Collecting user behavior data

[0428] The terminal collects user location information, schedule data, call history, and other behavioral data. Specifically, it uses the GPS function, calendar app, call log, etc. of the smart device.

[0429] Input: GPS data, calendar data, call logs

[0430] Data processing: recording location information, obtaining schedules, saving call history

[0431] Output: Collected behavioral data

[0432] Specific operation: The device records the user's location information every morning between 7:45 and 8:00 a.m. It also retrieves commute-related events from the user's calendar app.

[0433] Step 2:

[0434] Learning behavioral patterns

[0435] The server uses the collected data to train an AI model to learn user behavior patterns, using machine learning algorithms to identify when a user leaves for work at 8am every morning.

[0436] Input: Collected behavioral data

[0437] Data Computing: Data Analysis with Machine Learning Algorithms

[0438] Output: Learned behavioral patterns

[0439] What happens: The server analyzes data from the past 30 days and finds a pattern of users commuting from home to work every morning at 8am.

[0440] Step 3:

[0441] Commuting preparation notification

[0442] Based on the learned behavioral patterns, the device will notify the user to start preparing for their commute every morning before 8:00 a.m. This notification will be sent using the push notification function of the smart device.

[0443] Input: Learned behavioral patterns

[0444] Data processing: Generation of notification messages

[0445] Output: Notification to prepare for commute

[0446] Specific operation: At 7:45, the device sends the user a notification saying, "Start getting ready for your commute."

[0447] Step 4:

[0448] Notifications to prevent lost items

[0449] The device learns that users often make the mistake of forgetting their phone during their commute and notifies them before they leave to remember to take their phone with them.

[0450] Input: Learned mistake patterns

[0451] Data processing: Generation of notification messages

[0452] Output: Notification to prevent loss

[0453] Specific behavior: The device sends the user a notification before 8 o'clock saying, "Don't forget to take your phone with you."

[0454] Step 5:

[0455] Emotion recognition and response

[0456] The emotion engine in the device recognizes emotions from the user's tone of voice and facial expressions. Based on the recognized emotions, the AI ​​can better assist the user in completing tasks. For example, if it recognizes that the user is feeling impatient or anxious, it will advance the reminder notification time.

[0457] Input: User's voice tone, facial expression data

[0458] Data Computation: Analysis with Emotion Recognition Algorithms

[0459] Output: Adjusted notification time

[0460] Specific behavior: If the device recognizes that the user is in a hurry, it will send a notification at 7:30 saying, "Please start getting ready for your commute."

[0461] (Application example 2)

[0462] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0463] Although the reminder and notification functions of conventional smart devices can learn the user's behavioral patterns, they have the problem of not being able to notify the user at the appropriate time while taking into account the user's emotional state. In addition, the function of notifying the user to bring items that the user often forgets is also limited, and it has not been possible to sufficiently improve user convenience.

[0464] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0465] In this invention, the server includes a means for collecting data in the smart device, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for recognizing the user's emotion, and a means for adjusting the timing of notification based on the recognized emotion. This enables notification at an appropriate timing taking into account the user's behavioral pattern and emotional state, thereby improving user convenience.

[0466] A "smart device" is an electronic device with advanced functions, such as a smartphone, smart glasses, a head-mounted display, or a robot.

[0467] "Means of collecting data" refers to the functions of sensors and software for obtaining information about the user's behavior and environment.

[0468] "Means for analyzing data" refers to algorithms or software functions that analyze collected data and identify user behavior patterns and trends.

[0469] "Means for predicting behavior" refers to algorithms or software functions for predicting a user's next behavior based on the analysis results.

[0470] The "means for supporting task processing" is a function for providing appropriate reminders and notifications to the user based on the prediction results, thereby enabling the user to efficiently process tasks.

[0471] "Means for recognizing emotions" refers to algorithms or software functions for detecting and recognizing emotions from the user's tone of voice, facial expressions, etc.

[0472] "Means for adjusting notification timing" refers to algorithms or software features that optimize the timing of reminders or notifications based on perceived emotions.

[0473] A system for carrying out the present invention includes a program installed in a smart device. The smart device may be a smartphone, smart glasses, a head-mounted display, or a robot. The system has a function of learning a user's behavioral patterns and providing reminders and notifications at appropriate times. It also has an emotion engine that recognizes the user's emotions and provides appropriate support.

[0474] Program processing explanation

[0475] 1. Data collection:

[0476] Smart devices use sensors and software to collect data about users' behavior and the environment, including GPS data, calendar information, and call history.

[0477] 2. Data Analysis:

[0478] The collected data is analyzed using algorithms and software within the device (e.g., Python data analysis libraries) to understand user behavior patterns and trends.

[0479] 3. Behavioral Prediction:

[0480] Based on the analysis results, it predicts the user's next action. For example, it learns that the user commutes to work at 8am every morning and sets a reminder to get ready for work before 8am.

[0481] 4. Task processing assistance:

[0482] Based on the prediction results, the system provides appropriate reminders and notifications to the user, for example, notifying the user to bring items that the user often forgets (such as a smartphone or keys).

[0483] 5. Emotion recognition:

[0484] The emotion engine recognizes the user's emotions from their tone of voice and facial expressions. For example, if the user is feeling impatient or anxious, the emotion engine will recognize this as "anxious."

[0485] 6. Notification timing adjustment:

[0486] The timing of reminders and notifications is optimized based on the recognized emotions. For example, if the user is feeling anxious or impatient, notifications can be sent earlier to reduce the user's anxiety.

[0487] Examples

[0488] As a concrete example, consider a user who commutes to work every morning at 8:00. If the user feels anxious or impatient, the AI ​​will notify them to "start preparing to leave early" and remind them not to forget their smartphone or keys.

[0489] Examples of prompt statements

[0490] Example of a prompt to be input to a generative AI model:

[0491] Create an application that learns the user's behavioral pattern of commuting to work every morning at 8am and displays reminders to prepare for the commute before the user leaves. Include a function that notifies the user to bring things that the user often forgets (e.g. smartphone or keys) and includes an emotion engine that recognizes emotions from the user's tone of voice and facial expressions and notifies them at the appropriate time.

[0492] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0493] Step 1:

[0494] Data collection

[0495] The device collects data about the user's behavior and environment using sensors and software. Specifically, it acquires GPS data, calendar information, call history, etc. The input is data about the user's behavior and environment, and the output is a set of collected data.

[0496] Step 2:

[0497] Data analysis

[0498] The terminal analyzes the collected data. Specifically, it uses a Python data analysis library to understand the user's behavioral patterns and trends. The input is a set of collected data, and the output is the analysis results on the user's behavioral patterns and trends.

[0499] Step 3:

[0500] Behavioral Prediction

[0501] The device predicts the user's next action based on the analysis results. Specifically, it learns the user's commute pattern at 8 a.m. every morning and sets a reminder to prepare for work before 8 a.m. The input is the analysis results on the user's behavioral patterns and tendencies, and the output is the user's predicted next action.

[0502] Step 4:

[0503] Task processing assistance

[0504] The device sends appropriate reminders and notifications to the user based on the prediction results. Specifically, it notifies the user to bring items that the user often forgets (such as a smartphone or keys). The input is the predicted next action of the user, and the output is a reminder or notification for the user.

[0505] Step 5:

[0506] emotion recognition

[0507] The device uses an emotion engine to recognize emotions from the user's tone of voice and facial expressions. Specifically, if the user is feeling impatient or anxious, the emotion engine recognizes this as "anxious." The input is data related to the user's tone of voice and facial expressions, and the output is the recognized emotional state of the user.

[0508] Step 6:

[0509] Notification timing adjustment

[0510] The device optimizes the timing of reminders and notifications based on the recognized emotions. Specifically, if the user feels impatient or anxious, the device advances the timing of notifications to reduce the user's anxiety. The input is the recognized user's emotional state, and the output is the adjusted timing of notifications.

[0511] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0512] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0513] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0514] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0515] [Second embodiment]

[0516] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0517] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0518] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0519] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0520] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0521] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0522] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0523] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0524] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0525] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0526] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0527] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0528] "Example 1"

[0529] In one embodiment of the present invention, the AI ​​in the smartphone has a function of collecting data from the smartphone, such as a user's operation history, location information, and communication history. In addition, as a means for analyzing the collected data, the AI ​​has a function of learning the user's behavioral patterns and predicting the user's next action. Furthermore, as a means for supporting the user's task processing based on the prediction results, the AI ​​has a function of providing the user with suggestions for the next action and information on how to prevent mistakes.

[0530] "Example 2"

[0531] As a specific example, if the AI ​​learns that the user leaves for work at 8 a.m. every morning, the AI ​​will suggest that the user start preparing for work before 8 a.m. In addition, if the AI ​​learns that the user often makes the mistake of forgetting their phone during their commute, the AI ​​will notify the user before leaving to remember to take their phone with them.

[0532] The process flow of each embodiment will be described below.

[0533] "Example 1"

[0534] Step 1: The AI ​​in the smartphone collects data such as the user's operation history, location information, and communication history.

[0535] Step 2: AI analyzes the collected data and learns user behavior patterns.

[0536] Step 3: Predict the user's next action based on the behavioral patterns the AI ​​has learned.

[0537] Step 4: Based on the prediction results, the AI ​​provides the user with suggestions for the next course of action and information on how to prevent mistakes.

[0538] "Example 2"

[0539] Step 1: The AI ​​learns the user's behavioral pattern: that they leave for work at 8am every morning.

[0540] Step 2: Just before 8am, the AI ​​suggests the user start getting ready for work.

[0541] Step 3: The AI ​​learns that users often make the mistake of forgetting their phone during their commute.

[0542] Step 4: Before leaving, the AI ​​notifies the user to remember to bring their phone.

[0543] Example 1

[0544] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0545] In conventional smartphone systems, it was difficult to effectively use data such as user operation history, location information, and communication history to predict user behavior and assist with task processing. In addition, there was a lack of appropriate means to learn the user's behavioral patterns and predict the next action, making it difficult to improve user convenience.

[0546] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0547] In this invention, the server includes a means for collecting data such as a user's operation history, location information, and communication history, a means for transmitting the collected data to the server, a means for the server to analyze the data received and learn the user's behavioral patterns, a means for predicting the user's next behavior based on the learned behavioral patterns, and a means for supporting the user's task processing based on the prediction results. This makes it possible to effectively predict the user's behavior and support the task processing at an appropriate time.

[0548] "User operation history" is a record of operations performed by a user on a smartphone.

[0549] "Location information" refers to information about a user's current location and movement history, obtained using a smartphone's GPS function or the like.

[0550] "Communication history" refers to records of calls and messages sent and received by a user via their smartphone.

[0551] A "server" is a computer system that collects, analyzes, stores, and transmits data over a network.

[0552] "Means of collecting data" refers to the function of obtaining data such as user operation history, location information, and communication history using sensors and software within the smartphone.

[0553] The "means for transmitting data to a server" is a function for transmitting collected data to a server via a network.

[0554] "Means for analyzing data" refers to a function for the server to process the data it receives and learn the user's behavioral patterns.

[0555] The "means for learning behavioral patterns" is a function that uses a machine learning algorithm to extract behavioral tendencies and patterns from the user's past data.

[0556] The "means for predicting the next action" is a function for predicting the next action that the user will take based on the learned behavioral patterns.

[0557] "Means to assist in task processing" is a function that provides the user with suggestions for the next course of action and information on how to prevent mistakes based on the prediction results.

[0558] The present invention relates to a system for assisting in task processing by collecting data in a smartphone, analyzing the data in a server, and predicting user behavior. A specific embodiment of this system will be described below.

[0559] First, the device collects data such as the user's operation history, location information, and communication history. This is done using the smartphone's built-in sensors, GPS, and communication module. For example, if a user frequently uses a particular app, its operation history is recorded.

[0560] Next, the terminal sends the collected data to the server. The data is encrypted and sent using a secure communication protocol (e.g., HTTPS). For example, the user's operation history and location information are sent to the server at regular intervals.

[0561] The server analyzes the received data. Specifically, it uses machine learning algorithms (e.g., TensorFlow or PyTorch) to learn the user's behavioral patterns. For example, the server analyzes the user's operation history and finds a pattern of using a specific app at a specific time of day.

[0562] The server then predicts the user's next action based on the learned behavioral patterns. For example, the server predicts that the user will arrive at work at 8:00 every morning.

[0563] Finally, the server sends a notification to the device based on the prediction result. The device displays the user's next action suggestion and information on how to prevent mistakes. For example, the device displays a notification saying, "It's time to start getting ready for work."

[0564] Examples

[0565] Example 1: Collecting operation history

[0566] The device records the time when the user opens the SNS app. For example, if the user opens the SNS app at 10 p.m. every night, that time is recorded as the operation history.

[0567] Example 2: Sending data

[0568] The device sends the recorded operation history to the server. For example, data that indicates that a social networking app was opened at 10 p.m. every night is sent to the server.

[0569] Example 3: Data analysis

[0570] The server analyzes the operation history it receives and learns the pattern of how users use the SNS app every night at 10 p.m. For example, it models this pattern using TensorFlow.

[0571] Example 4: Behavioral prediction

[0572] Based on the patterns learned by the server, the server predicts the next time the user will open the SNS app. For example, the server predicts that "the user will next open the SNS app at 10 p.m."

[0573] Example 5: Task assistance

[0574] The server sends the prediction result to the device, and the device displays a notification to the user. For example, at 9:50 p.m., the device displays a notification saying, "SNS time is approaching."

[0575] Examples of prompt statements

[0576] "Design an AI system that collects user operation history, location information, and communication history, learns behavioral patterns, and predicts the next action. Specifically, implement a function that learns behavioral patterns using TensorFlow and sends a notification to the user based on the prediction results."

[0577] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0578] Step 1:

[0579] The device collects data such as the user's operation history, location information, and communication history.

[0580] Input: User operations, location information, communication history

[0581] Specific behavior: Using the smartphone's built-in sensors, GPS, and communication module, it records information such as which apps the user used and when, where they were, and who they communicated with.

[0582] Output: Collected data (operation history, location information, communication history)

[0583] Step 2:

[0584] The terminal transmits the collected data to the server.

[0585] Input: Collected data (operation history, location information, communication history)

[0586] What it does: Encrypts the data and sends it to the server using a secure communications protocol (e.g. HTTPS).

[0587] Output: Data sent to the server

[0588] Step 3:

[0589] The server parses the received data.

[0590] Input: Data sent to the server

[0591] Specific operation: Using machine learning algorithms (e.g. TensorFlow and PyTorch), the system learns user behavior patterns. For example, the server analyzes the user's operation history and finds a pattern of using a specific app at a specific time of day.

[0592] Output: Learned behavioral patterns

[0593] Step 4:

[0594] The server predicts the next action based on the behavioral patterns it has learned.

[0595] Input: Learned behavioral patterns

[0596] Specific behavior: Using a generative AI model, the server predicts what the user will do next. For example, the server predicts that the user will arrive at work at 8am every morning.

[0597] Output: Predicted next action

[0598] Step 5:

[0599] The server sends a notification to the terminal based on the prediction result.

[0600] Input: Predicted next action

[0601] Specific operation: The server sends the prediction result to the device, and the device displays the user's next action suggestion and information on how to prevent mistakes. For example, the device displays a notification saying, "It's time to start getting ready for work."

[0602] Output: The notification displayed to the user

[0603] (Application example 1)

[0604] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0605] Conventional food delivery systems do not fully utilize the user's order history and behavioral patterns, making it difficult to suggest menu items or send reminders to users at the right time. In addition, they lack the ability to learn the user's preferences and behavioral patterns, so there is a need to improve the user experience.

[0606] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0607] In this invention, the server includes means for collecting data in the smartphone, means for analyzing the collected data, means for predicting the user's next action based on the analysis result, means for supporting the user's task processing based on the prediction result, means for collecting the user's order history, location information, and communication history, and learning the user's behavioral patterns, means for predicting the next order based on the learned behavioral patterns, means for providing recommended menus and promotion information at appropriate times, and means for sending reminders at specific times, thereby enabling personalized suggestions and reminders to be sent based on the user's preferences and behavioral patterns.

[0608] A "smartphone" is a multi-function mobile device that, in addition to the functions of a mobile phone, can also connect to the Internet and use applications.

[0609] "Means for collecting data" refers to functions and devices for acquiring information such as a user's operation history, location information, and communication history.

[0610] "Means for analyzing data" refers to functions or devices for analyzing collected data and understanding user behavior patterns and trends.

[0611] The "means for predicting the user's next action" refers to a function or device for predicting the action that the user is likely to take next based on the analysis results.

[0612] The "means for supporting the user's task processing" refers to a function or device for supporting the user so that the user can efficiently carry out a task based on the prediction result.

[0613] "Order history" is a record of orders a user has placed in the past.

[0614] "Location information" is information relating to the user's current location and past movement history.

[0615] "Communication history" is a record of calls and messages made by a user.

[0616] A "means for learning behavioral patterns" is a function or device for learning the behavioral tendencies and habits of a user using collected data.

[0617] The "means for predicting the next order" refers to a function or device for predicting the next product that the user is likely to order based on the learned behavioral patterns.

[0618] The "means for providing recommended menu items and promotion information" refers to a function or device for presenting appropriate menu items and discount information to the user based on the prediction results.

[0619] A "means for sending a reminder" is a function or device for sending a notification to a user at a specific time.

[0620] As an embodiment of the present invention, a system is provided in which a user's order history, location information, and communication history in a food delivery system are collected, and an AI analyzes the data to learn the user's behavioral patterns. A specific embodiment of this system is described below.

[0621] System Configuration

[0622] The system includes a smartphone, a server, and an AI model. The smartphone is hardware for collecting users' operation history, location information, and communication history. The server runs software for analyzing the collected data and learning users' behavioral patterns. The AI ​​model is used to predict the user's next action and provide recommended menus and promotional information at the appropriate time.

[0623] Collecting data

[0624] Smartphones collect users' operation history, location information, and communication history in real time. This includes GPS sensors, communication modules, and application log data. The collected data is sent to a server.

[0625] Analyzing data

[0626] The server receives the collected data and analyzes it using an AI model. Specifically, it uses programming languages ​​such as Python to pre-process the data, extract features, and train the model. The AI ​​model learns the user's behavioral patterns and predicts the next order.

[0627] Predicting and suggesting actions

[0628] Based on the learned behavioral patterns, the AI ​​model predicts what the user is likely to order next. Based on the prediction results, the server sends menu recommendations and promotion information to the smartphone. It also includes a function to send reminders at specific times.

[0629] Examples

[0630] For example, if a user has ordered "pizza," "sushi," "ramen," and "curry" in the past, the AI ​​model might recommend "pizza" next. It also sends a reminder at 12 o'clock saying, "It's lunchtime! Order your favorite menu!"

[0631] Examples of prompt statements

[0632] Here is an example of a prompt to be input to the generative AI model:

[0633] Use the user's past orders, location, and communication history to predict what their next menu recommendation will be, and add the ability to send them a reminder at a specific time.

[0634] In this way, personalized offers and reminders can be sent based on the user's preferences and behavioral patterns.

[0635] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0636] Step 1:

[0637] The smartphone collects the user's operation history, location information, and communication history. Specifically, it obtains location information using a GPS sensor, collects operation history from application log data, and records communication history through a communication module. This data is sent to a server in real time.

[0638] Input: User operation history, location information, communication history

[0639] Output: Sending collected data to a server

[0640] Step 2:

[0641] The server receives the data sent from the smartphone. The received data is stored in a database. The database organizes and stores data for each user.

[0642] Input: Data sent from smartphone

[0643] Output: Data stored in the database

[0644] Step 3:

[0645] The server preprocesses the data stored in the database, specifically by imputing missing values, removing outliers, normalizing the data, etc. This improves the quality of the data and makes it easier for subsequent analysis.

[0646] Input: Raw data stored in a database

[0647] Output: Preprocessed data

[0648] Step 4:

[0649] The server uses the preprocessed data to train the AI ​​model. Specifically, it uses a programming language such as Python to perform feature extraction, model selection, and model training. The AI ​​model learns the user's behavioral patterns.

[0650] Input: Preprocessed data

[0651] Output: A trained AI model

[0652] Step 5:

[0653] The server uses the trained AI model to predict the user's next action, specifically, guessing what menu item the user is likely to order next based on their past ordering history and behavioral patterns.

[0654] Input: Trained AI model, user's past order history

[0655] Output: Next order prediction result

[0656] Step 6:

[0657] Based on the prediction results, the server sends recommended menus and promotion information to the smartphone, specifically providing menus tailored to the user's preferences and currently available promotion information.

[0658] Input: Predicted result for next order

[0659] Output: Menu recommendations and promotional information sent to your smartphone

[0660] Step 7:

[0661] The server sends a reminder at a specific time. Specifically, the server sends a reminder to the smartphone at 12 o'clock saying "It's lunchtime! Order your favorite menu!" so that the user does not forget to order at lunchtime.

[0662] Input: current time, user behavior pattern

[0663] Output: Reminders sent to your smartphone

[0664] These steps enable personalized suggestions and reminders based on the user's preferences and behavioral patterns.

[0665] Example 2

[0666] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0667] Conventional smartphones and wearable devices can collect user behavior data, but there are only a limited number of systems that effectively use that data to support user behavior. In particular, there is a demand for systems that can learn user behavior patterns and notify users at the appropriate time to reduce user errors.

[0668] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting user behavior data, a means for transmitting the collected data to the server, a means for analyzing the data in the server and learning the user behavior pattern using an AI model, a means for generating notification content based on the learned behavior pattern, a means for transmitting the generated notification content to the terminal, and a means for displaying the notification on the terminal. This enables notification at an appropriate time based on the user's behavior pattern, and reduces user errors.

[0669] "User behavioral data" refers to information related to a user's behavior, such as the user's daily behavior, movement patterns, and application usage history.

[0670] "Server" refers to a computer system that receives user behavioral data, analyzes it, and trains the AI ​​model.

[0671] "AI model" refers to a model that uses machine learning algorithms to learn user behavior patterns and generate predictions and notifications.

[0672] A "behavioral pattern" refers to a tendency for a user to repeatedly perform an action at a particular time or in a particular situation.

[0673] "Notification content" refers to messages or alerts that are generated to prompt the user to take action.

[0674] "Terminal" refers to a device that is directly operated by a user, such as a smartphone or tablet.

[0675] "Notification" means a message or alert that is displayed to the User via a Device.

[0676] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0677] The present invention is a system that collects user behavior data and analyzes it using an AI model to provide notifications based on the user's behavior patterns. A specific embodiment of this system will be described below.

[0678] Collecting user behavior data

[0679] Users use smartphones and wearable devices to collect daily behavioral data. For example, the GPS function of a smartphone is used to record the user's movement patterns. Data from calendar apps and reminder apps is also collected.

[0680] Sending behavioral data to the server

[0681] The device periodically transmits the collected behavioral data to a server, where the data is encrypted and transmitted using a secure communication protocol (e.g. HTTPS).

[0682] Data analysis on the server and training of AI models

[0683] The server analyzes the received behavioral data and uses an AI model to learn the user's behavioral patterns. Specifically, it uses Python and leverages machine learning libraries such as TensorFlow and PyTorch. For example, it learns that a user leaves for work at 8am every morning.

[0684] Generate notification content

[0685] The server generates notifications based on learned behavioral patterns. For example, it generates a notification "Start getting ready for work" when the user should start getting ready for work (7:50). Also, if the user tends to forget their phone, it generates a notification "Don't forget to take your phone with you."

[0686] Sending notifications to devices

[0687] The server sends the generated notification content to the terminal. The notification is sent in real time so that the user can receive it at the appropriate time.

[0688] Displaying notifications on your device

[0689] The device will display the received notifications to the user. Notifications will be displayed as pop-ups or push notifications so that the user can check them immediately.

[0690] Examples

[0691] As a concrete example, let's consider a case where the AI ​​has learned a user's behavioral pattern of leaving for work at 8:00 every morning. The AI ​​will notify the user at 7:50, "Please start preparing for your commute." Also, if the user has forgotten their phone many times in the past, the AI ​​will notify them, "Don't forget to take your phone with you."

[0692] Examples of prompt statements

[0693] An example of a prompt sentence might be:

[0694] Example prompt:

[0695] Design an AI system that learns user behavior patterns and notifies them at the right time. If a user leaves for work at 8am every morning, include a feature that notifies them to start getting ready for work at 7:50am and reminds users who tend to forget their phone to take it with them before leaving.

[0696] By inputting this prompt sentence into a generative AI model, a notification system design based on the user's behavioral patterns is generated.

[0697] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0698] Program processing flow

[0699] Step 1: Collect user behavior data

[0700] Users collect daily behavioral data using smartphones and wearable devices. Specifically, the GPS function of the smartphone is used to record the user's movement patterns. Data from calendar apps and reminder apps is also collected. The input is the user's behavioral data, and the output is the collected behavioral data.

[0701] Step 2: Sending behavioral data to the server

[0702] The device periodically transmits the collected behavioral data to the server. The data is encrypted and transmitted using a secure communication protocol (e.g., HTTPS). The input is the collected behavioral data, and the output is the behavioral data transmitted to the server.

[0703] Step 3: Data analysis on the server and training of the AI ​​model

[0704] The server analyzes the received behavioral data and uses an AI model to learn the user's behavioral patterns. Specifically, it uses Python to leverage machine learning libraries such as TensorFlow and PyTorch. The input is the behavioral data sent to the server, and the output is the learned behavioral pattern. For example, it learns a pattern that a user leaves for work at 8am every morning.

[0705] Step 4: Generate notification content

[0706] The server generates notification content based on the learned behavioral patterns. For example, it generates a notification "Start preparing for your commute" when the user should start preparing for their commute (7:50). Also, if the user tends to forget their phone, it generates a notification "Don't forget to take your phone with you." The input is the learned behavioral patterns, and the output is the generated notification content.

[0707] Step 5: Sending notifications to devices

[0708] The server sends the generated notification content to the terminal. The notification is sent in real time so that the user can receive it at the appropriate time. The input is the generated notification content, and the output is the notification content sent to the terminal.

[0709] Step 6: Display notifications on your device

[0710] The terminal displays the received notifications to the user. Notifications are displayed as popups or push notifications so that the user can see them immediately. The input is the notification content sent to the terminal, and the output is the notification displayed to the user.

[0711] Examples

[0712] As a concrete example, let's consider a case where the AI ​​has learned a user's behavioral pattern of leaving for work at 8:00 every morning. The AI ​​will notify the user at 7:50, "Please start preparing for your commute." Also, if the user has forgotten their phone many times in the past, the AI ​​will notify them, "Don't forget to take your phone with you."

[0713] (Application example 2)

[0714] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0715] Conventional user support systems using smart devices can learn the user's behavioral patterns and predict the user's next actions, but due to lack of coordination with the automobile, which is the user's means of transportation, they could not sufficiently improve the user's convenience. In addition, there was a lack of concrete measures to prevent users from forgetting things. This often caused inconvenience to users when commuting or traveling.

[0716] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0717] In this invention, the server includes a means for collecting data in the smart device, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for controlling the functions of the car based on the user's behavioral pattern, and a means for notifying the user of the user's forgotten items. This makes it possible to automatically control the functions of the car based on the user's behavioral pattern and prompt the user to make necessary preparations for commuting or traveling. In addition, by providing a notification function to prevent the user from forgetting things, the convenience of the user can be significantly improved.

[0718] A "smart device" is an electronic device that can be carried by a user and has functions such as data collection, analysis, and communication.

[0719] "Means of collecting data" refers to the function of obtaining information from various sensors and applications within smart devices.

[0720] "Means for analyzing data" refers to the function of processing collected data and analyzing user behavior patterns and trends.

[0721] "Means for predicting the user's next action" refers to a function for predicting the action that the user is likely to take next based on the analysis results.

[0722] "Means for supporting user task processing" refers to a function that supports the user to efficiently complete a task based on predicted behavior.

[0723] "Means for controlling the functions of an automobile" refers to a function for automatically operating functions such as starting the automobile engine and setting the air conditioning based on the user's behavioral patterns.

[0724] "Means for notifying a user of lost items" refers to a feature that notifies the user before departure to ensure that the user does not forget a particular item.

[0725] The system for carrying out this invention collects and analyzes data in smart devices, learns the user's behavioral patterns, predicts the user's next actions, and assists the user in processing tasks. It also has the function of controlling the functions of a car based on the user's behavioral patterns and notifying the user of lost items.

[0726] System Configuration

[0727] The system includes the following major hardware and software components:

[0728] Hardware: Smart devices (smartphones, smart glasses, head-mounted displays, etc.), infotainment systems for autonomous vehicles

[0729] Software: Python, smtplib (email sending library), data analysis libraries (e.g. Pandas, NumPy), machine learning libraries (e.g. scikit-learn)

[0730] Data collection and analysis

[0731] The server collects data from various sensors and applications in smart devices. The collected data is processed using a data analysis library to analyze user behavior patterns and trends. For example, a user's behavior pattern of commuting to work every morning at 8 o'clock is detected.

[0732] Behavioral prediction and task support

[0733] The server predicts the user's next action based on the analysis results. Based on the predicted action, the server supports the user to efficiently complete tasks. For example, when it is time to commute, the server notifies the user to prepare for commuting.

[0734] Automotive Function Control

[0735] The system automatically controls the functions of the car based on the user's behavioral patterns, for example starting the car's engine and setting the air conditioner to an appropriate temperature when it is time to commute to work.

[0736] Lost property notification

[0737] By registering items that a user often forgets in advance, the system will notify the user to bring these items before leaving. For example, if a user often forgets their phone and wallet, the system will notify the user to bring these items before leaving.

[0738] Examples

[0739] As a concrete example, consider a user who commutes to work every morning at 8am and often forgets their phone and wallet. In this case, the system will start the engine before 8am, set the air conditioner, and notify the user to prepare for work. It will also notify the user to not forget their phone and wallet.

[0740] Examples of prompt statements

[0741] Here is an example of a prompt to be input to the generative AI model:

[0742] Design an AI system that learns the user's behavioral pattern of commuting to work every morning at 8:00, and then starts the engine before 8:00, sets the air conditioner to an appropriate temperature, and notifies the user to prepare for the commute. Also, if the user often forgets his / her phone and wallet during the commute, add a feature that notifies the user to have these items with him / her before leaving.

[0743] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0744] Step 1:

[0745] The server collects data from various sensors and applications within the smart device.

[0746] Input: Sensor data from smart devices (location, time, application usage history, etc.)

[0747] Output: Raw data collected

[0748] Specific operation: The server communicates with the smart device, obtains location information from the GPS sensor, and collects application usage history.

[0749] Step 2:

[0750] The server processes the collected data using a data analysis library to analyze user behavior patterns.

[0751] Input: Raw data collected

[0752] Output: Analyzed behavioral patterns (e.g. commuting to work at 8am every morning)

[0753] Specific operation: The server uses data analysis libraries such as Pandas and NumPy to organize the collected data in chronological order and extract user behavior patterns.

[0754] Step 3:

[0755] The server predicts the user's next action based on the analysis results.

[0756] Input: Analyzed behavioral patterns

[0757] Output: Predicted next action (e.g., start commuting at 8am)

[0758] Specific operation: The server uses a machine learning model (e.g., scikit-learn) to predict the next action based on past behavior patterns.

[0759] Step 4:

[0760] The server assists the user in processing tasks based on the predicted behavior.

[0761] Input: Predicted next action

[0762] Output: Notification to the user (e.g., notification to prepare for commute)

[0763] Specific operation: The server sends a notification to the user's smart device and displays a message encouraging them to prepare for their commute.

[0764] Step 5:

[0765] The server controls the functions of the vehicle based on the user's behavioral patterns.

[0766] Input: Predicted next action

[0767] Output: Car control instructions (e.g. engine start, air conditioning settings)

[0768] What it does: The server communicates with the autonomous vehicle's infotainment system, starts the engine, and sets the air conditioning to the right temperature.

[0769] Step 6:

[0770] The server notifies the user of items that he or she often forgets.

[0771] Input: A list of lost items registered by the user (e.g. phone, wallet)

[0772] Output: Lost item notification content

[0773] Specific operation: The server sends a notification to the user's smart device and displays a message reminding the user to have their phone and wallet ready before leaving.

[0774] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0775] "Example 1"

[0776] In one embodiment of the present invention, the AI ​​in the smartphone collects data such as the user's operation history, location information, and communication history. Furthermore, the collected data is analyzed to learn the user's behavioral patterns. Then, based on the learned behavioral patterns, the AI ​​predicts the user's next action. Based on the prediction result, the AI ​​provides the user with suggestions for the next action and information on preventing mistakes. Furthermore, in one embodiment of the present invention, the AI ​​includes an emotion engine. The emotion engine recognizes emotions from the user's tone of voice and facial expressions. Based on the recognized emotions, the AI ​​more appropriately supports the user in processing tasks. For example, if the AI ​​recognizes that the user is feeling stressed, the AI ​​supports the user in reducing stress by playing relaxation music and providing a message encouraging deep breathing.

[0777] "Example 2"

[0778] In another embodiment of the present invention, the AI ​​in the smartphone learns a behavioral pattern that the user leaves for work at 8 o'clock every morning. Before 8 o'clock, the AI ​​suggests to the user to start preparing for work. Furthermore, the AI ​​learns information that the user often makes the mistake of forgetting his / her phone during the commute. Before leaving, the AI ​​notifies the user not to forget to take the phone. Furthermore, in one embodiment of the present invention, the AI ​​includes an emotion engine. The emotion engine recognizes emotions from the user's tone of voice and facial expressions. Based on the recognized emotions, the AI ​​better supports the user in processing tasks. For example, if the AI ​​recognizes that the user is feeling impatient or anxious, the AI ​​reduces the user's anxiety by speeding up the reminder notification time.

[0779] The process flow of each embodiment will be described below.

[0780] "Example 1"

[0781] Step 1: The AI ​​collects data such as the user's operation history, location information, communication history, etc. Step 2: The AI ​​analyzes the collected data and learns the user's behavioral patterns.

[0782] Step 3: Predict the user's next action based on learned behavioral patterns.

[0783] Step 4: Based on the prediction results, the AI ​​provides the user with suggestions for next steps and information on how to prevent mistakes.

[0784] Step 5: The emotion engine recognizes emotions from the user's tone of voice and facial expressions.

[0785] Step 6: Based on the recognized emotions, the AI ​​will better assist the user in completing tasks. For example, if the AI ​​recognizes that the user is feeling stressed, it will help the user reduce stress by playing relaxation music or providing a message encouraging deep breathing. "Example 2"

[0786] Step 1: The AI ​​learns the user's behavioral pattern: that they leave for work at 8am every morning.

[0787] Step 2: Just before 8am, the AI ​​suggests the user start getting ready for work.

[0788] Step 3: The AI ​​learns that users often make the mistake of forgetting their phone during their commute. Before leaving, the AI ​​will notify the user to remember to take their phone with them.

[0789] Step 4: The emotion engine recognizes emotions from the user's tone of voice and facial expressions.

[0790] Step 5: Based on the recognized emotions, the AI ​​can better assist the user in completing the task. For example, if the AI ​​recognizes that the user is feeling anxious or impatient, it can speed up the reminder notification time to reduce the user's anxiety.

[0791] Example 1

[0792] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0793] Conventional smartphone systems can collect data such as user operation history, location information, and communication history, but it is difficult to effectively use this data to predict user behavior and provide appropriate task support. In addition, there was no system that recognized the user's emotions and provided task support based on them. This made it difficult to reduce user stress and mistakes, and improving the user experience was a challenge.

[0794] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0795] In this invention, the server includes a means for collecting data such as a user's operation history, location information, and communication history, a means for analyzing the collected data and learning the user's behavioral patterns, a means for predicting the user's next behavior based on the learned behavioral patterns, a means for supporting the user's task processing based on the prediction results, a means for recognizing emotions from the user's tone of voice and facial expressions, and a means for appropriately supporting the user's task processing based on the recognized emotions. This makes it possible to predict the user's behavior and provide appropriate task support, reducing user stress and mistakes and improving the user experience.

[0796] "User operation history" is a record of operations performed by a user on a smartphone, and is data including the time spent using an app and the location of taps.

[0797] "Location information" is data about a user's current location and movement history obtained using a smartphone's GPS function, etc.

[0798] "Communication history" refers to records of calls and messages sent and received by a user, and is data including call history and message contents.

[0799] "Means of collecting data" refers to the function of obtaining data such as user operation history, location information, and communication history using sensors and communication modules within the smartphone.

[0800] "Means for analyzing data" refers to functions that use machine learning algorithms and databases to analyze collected data and extract user behavior patterns.

[0801] "Means for learning behavioral patterns" refers to a function that uses a machine learning algorithm to extract patterns from a user's past behavioral data and generate a model for predicting the user's next behavior.

[0802] "Means for predicting the next action" refers to a function for predicting the next action that the user is likely to take based on learned behavioral patterns.

[0803] "Means to assist in task processing" refers to functions that suggest the user's next course of action based on prediction results or provide information to prevent mistakes.

[0804] "Means for recognizing emotions" refers to functionality that uses voice recognition software and a camera module to analyze the user's tone of voice and facial expressions to determine the user's emotional state.

[0805] "Means for appropriately supporting task processing based on emotions" refers to a function that provides assistance such as playing relaxation music or providing a message encouraging the user to take a deep breath, depending on the recognized emotions.

[0806] This invention is a system that uses AI in a smartphone to collect data such as a user's operation history, location information, and communication history, analyzes this data, learns the user's behavioral patterns, and predicts the user's next actions. Furthermore, it has a function to recognize the user's emotions from their tone of voice and facial expressions, and appropriately support the user's task processing based on the recognized emotions.

[0807] Data collection

[0808] When a user operates a smartphone, the device collects data such as the user's operation history, location information, and communication history. This data is collected using hardware such as the smartphone's built-in sensors, GPS, and communication modules. In addition, the smartphone's storage and dedicated database software are used to collect and store the data.

[0809] Analyzing data and learning behavioral patterns

[0810] The data collected by the device is sent to a server. The server stores the received data in a database and analyzes the data using machine learning algorithms. Specifically, frameworks such as TensorFlow and PyTorch are used to extract patterns from the user's past behavior data and generate a model to predict the user's next behavior.

[0811] Behavioral prediction and suggestions

[0812] The server sends the generated prediction model to the device. The device uses the prediction model to predict the user's next action. For example, if a user has the habit of opening a specific app at 8am every morning, the AI ​​will suggest automatically opening the app at that time. Also, if a user has the habit of calling a specific contact when they are in a specific location, the AI ​​will suggest making the call when they arrive at that location.

[0813] Emotion Recognition and Task Assistance

[0814] When a user is using a smartphone, the device analyzes the user's tone of voice and facial expressions with an emotion engine. Voice recognition software and a camera module are used for this emotion recognition. For example, Google Cloud Speech-to-Text API and OpenCV are used. Based on the recognized emotion, the AI ​​assists the user in completing tasks. For example, if it recognizes that the user is feeling stressed, it will play relaxation music and provide a message encouraging deep breathing.

[0815] Examples and prompts

[0816] As a concrete example, suppose a user has the habit of opening the calendar app at 8 am every morning. The AI ​​learns this behavioral pattern and suggests opening the calendar app at 8 am every morning. Also, if the AI ​​recognizes that the user is feeling stressed, it will play relaxation music.

[0817] Examples of prompts include, "If the user has the habit of opening the calendar app at 8am every morning, the AI ​​will suggest opening the calendar app at that time. Also, if it recognizes that the user is feeling stressed, it will suggest playing relaxation music."

[0818] In this way, the AI ​​in the smartphone learns the user's behavioral patterns and predicts and suggests the next action to assist the user in completing tasks. In addition, it uses an emotion engine to recognize the user's emotions and provide appropriate assistance.

[0819] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0820] Step 1:

[0821] The user operates the smartphone.

[0822] When a user opens an app or sends a message, data such as operation history, location information, and communication history is generated. This data becomes the input.

[0823] Step 2:

[0824] The device collects data such as the user's operation history, location information, and communication history.

[0825] The device collects data in real time using built-in sensors, GPS, and communication modules. The collected data is stored in the internal storage, which is the output.

[0826] Step 3:

[0827] The terminal transmits the collected data to the server.

[0828] The terminal transmits the collected data to the server at regular intervals using the HTTPS protocol. The transmitted data becomes the input.

[0829] Step 4:

[0830] The server stores the received data in a database.

[0831] The server stores the received data in a database (e.g. MySQL, PostgreSQL). The stored data becomes the output.

[0832] Step 5:

[0833] The server analyzes the data using machine learning algorithms.

[0834] The server uses frameworks such as TensorFlow and PyTorch to analyze the stored data. Specifically, it performs clustering and classification to extract user behavior patterns. The analysis results are output.

[0835] Step 6:

[0836] The server learns user behavior patterns and generates a predictive model.

[0837] Based on the analysis results, the server generates a model to predict the user's next action. This model is the output.

[0838] Step 7:

[0839] The server sends the generated predictive model to the terminal.

[0840] The server uses a secure communication protocol (e.g. SSL / TLS) to send the prediction model to the terminal. The sent model becomes the input.

[0841] Step 8:

[0842] The device uses the predictive model to predict the user's next action.

[0843] The device uses the received prediction model to predict the user's next action in real time. For example, if the user has the habit of opening the calendar app at 8:00 every morning, the device will suggest opening the calendar app before 8:00. The prediction result is the output.

[0844] Step 9:

[0845] The device suggests the user's next action based on the prediction results.

[0846] Based on the prediction results, the device will suggest the next action to the user through notifications or pop-up messages. For example, it may display a message such as "It's 8 o'clock. Would you like to open the calendar app?". The suggestions are the output.

[0847] Step 10:

[0848] When a user uses a smartphone, the device analyzes the user's tone of voice and facial expressions using an emotion engine.

[0849] The device uses a built-in camera and microphone to analyze the user's tone of voice and facial expressions in real time, and the analysis results become input.

[0850] Step 11:

[0851] The device recognizes the user's emotions based on the analysis results.

[0852] The device uses voice recognition software and a camera module (e.g., Google Cloud Speech-to-Text API, OpenCV) to recognize the user's emotions. The recognition results are the output.

[0853] Step 12:

[0854] The device provides appropriate task assistance based on the emotions it recognizes.

[0855] Depending on the emotion recognized, the device will play relaxation music or display a message encouraging deep breathing. For example, if it recognizes that the user is feeling stressed, it will play relaxation music. The support content is the output.

[0856] (Application example 1)

[0857] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0858] Conventional food delivery applications do not take into account the user's behavioral patterns or emotions when making suggestions, making it difficult to improve user satisfaction. In addition, if the user is stressed, the application may not be able to respond appropriately, resulting in a poor user experience.

[0859] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0860] In this invention, the server includes a means for collecting data in the smartphone, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for recognizing the user's emotion, and a means for making an appropriate suggestion to the user based on the recognized emotion. This makes it possible to make a suggestion that takes into account the user's behavioral patterns and emotions, thereby improving the user's satisfaction. In addition, when the user is feeling stressed, a relaxation menu or a special offer can be provided to improve the user experience.

[0861] "Data within a smartphone" refers to various information collected through a smartphone, such as a user's operation history, location information, and communication history.

[0862] "Means of collection" refers to the functions and processes for acquiring and storing data within a smartphone.

[0863] "Means of analysis" refers to the functions and processes for analyzing collected data and understanding user behavior patterns and trends.

[0864] The "means for predicting the user's next action" refers to a function or process for estimating the action that the user is likely to take next based on the analysis results.

[0865] The "means for supporting user task processing" refers to functions and processes for supporting the user to efficiently perform tasks based on the prediction results.

[0866] "Means for recognizing the user's emotions" refers to a function or process for determining the emotions of the user from the user's tone of voice, facial expression, etc.

[0867] The "means for making appropriate suggestions to the user based on the recognized emotions" refers to a function or process for providing optimal actions or suggestions depending on the user's emotional state.

[0868] The system for realizing the present invention has a series of functions for collecting and analyzing data in a smartphone, predicting the user's next action, and making appropriate suggestions. A specific embodiment of this system will be described below.

[0869] 1. Data Collection

[0870] The server uses the means for collecting data within the smartphone to obtain data such as the user's operation history, location information, communication history, etc. This data collection is done automatically from various sensors and applications on the smartphone.

[0871] 2. Data Analysis

[0872] The collected data is analyzed by an analysis method in the server. This analysis method is implemented using a programming language such as Python and includes an algorithm for learning the user's behavioral patterns. Specifically, the method grasps the user's behavioral tendencies based on past operation history and location information.

[0873] 3. Behavioral Prediction

[0874] The server includes a means for predicting the user's next action based on the analysis results. This prediction means is implemented using a machine learning model to estimate the action that the user is likely to take next. For example, if past data shows that the user tends to visit a specific restaurant at a specific time, the server suggests that restaurant.

[0875] 4. Task processing support

[0876] The server has a means to assist the user in processing tasks based on the prediction results. Specifically, the server suggests appropriate restaurants and menus so that the user can perform tasks efficiently. If the user feels stressed, the server provides relaxation menus and special offers.

[0877] 5. Emotion recognition

[0878] The server is equipped with a means for recognizing the user's emotions. This emotion recognition is performed by analyzing the user's tone of voice and facial expressions. An emotion recognition engine (EmotionEngine) is used to determine whether the user is feeling stressed.

[0879] 6. Providing suggestions

[0880] The server has a means for making appropriate suggestions to the user based on the recognized emotion. For example, if the user feels stressed, the server can suggest a relaxation menu. As a concrete example, the following prompt sentence can be used:

[0881] Examples of prompt statements

[0882] "If the user is feeling stressed, suggest a relaxation menu."

[0883] In this way, the system can improve user satisfaction by making suggestions that take into account the user's behavioral patterns and emotions. Also, if the user is feeling stressed, the system can improve the user experience by providing relaxation menus and special offers.

[0884] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0885] Step 1:

[0886] The server collects data from the smartphone. Specifically, it automatically obtains data such as the user's operation history, location information, and communication history from various sensors and applications on the smartphone. The input is data from the smartphone, and the output is a set of collected data.

[0887] Step 2:

[0888] The server analyzes the collected data. Using a programming language such as Python, it executes an algorithm to learn the user's behavioral patterns. The input is the data collected in step 1, and the output is the analysis results on the user's behavioral patterns. Specifically, it analyzes the user's behavioral tendencies based on past operation history and location information.

[0889] Step 3:

[0890] The server predicts the user's next action based on the analysis results. A machine learning model is used to estimate the action the user is likely to take next. The input is the analysis result from step 2, and the output is the predicted next action. Specifically, if past data shows that the user has a tendency to use a specific restaurant at a specific time of day, the server suggests that restaurant.

[0891] Step 4:

[0892] The server assists the user in processing tasks based on the prediction results. Specifically, it suggests appropriate restaurants and menus so that the user can perform the task efficiently. The input is the prediction result from step 3, and the output is the suggestions to the user. Specifically, it notifies the user of recommended restaurants and menus.

[0893] Step 5:

[0894] The server recognizes the user's emotions. By analyzing the user's tone of voice and facial expressions, an emotion recognition engine (EmotionEngine) is used to determine whether the user is feeling stressed. The input is the user's tone of voice and facial expression data, and the output is the recognized emotional state. Specific operations include voice analysis and image analysis.

[0895] Step 6:

[0896] The server makes appropriate suggestions to the user based on the recognized emotions. If the user is feeling stressed, it will provide a relaxation menu or a special offer. The input is the emotion recognition result from step 5, and the output is the content of the suggestions to the user. Specifically, it will suggest a relaxation menu or notify the user of a special offer.

[0897] In this way, the server can make suggestions that take into account the user's behavioral patterns and emotions through a series of processing steps, thereby improving the user's satisfaction.

[0898] Example 2

[0899] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0900] Conventional smart devices have limited functionality for learning the user's behavioral patterns and predicting the user's next actions, and they do not provide appropriate notifications or support that take into account the user's emotional state. In particular, when the user is feeling impatient or anxious, there is a problem that notifications are not provided at the appropriate time, making it difficult to reduce the user's stress.

[0901] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0902] In this invention, the server includes a means for collecting data in the smart device, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for recognizing the user's emotion, and a means for adjusting the notification time based on the recognized emotion. This makes it possible to not only learn the user's behavioral pattern and predict the user's next action, but also to notify and support the user at an appropriate time taking into account the user's emotional state.

[0903] A "smart device" is an electronic device that can be carried by a user and has functions such as data collection, analysis, and notification.

[0904] "Means of collecting data" refers to the function of obtaining information about user behavior and the environment from various sensors and applications within smart devices.

[0905] "Means for analyzing data" refers to algorithms or software used to analyze collected data and identify user behavior patterns and trends.

[0906] The "means for predicting the user's next action" is a function for predicting the next action that the user should take based on the analysis results.

[0907] The "means for supporting the user in processing a task" is a function that supports the user to efficiently complete a task based on predicted behavior.

[0908] "Means for recognizing the user's emotions" refers to technology for detecting the user's emotional state from the user's tone of voice, facial expressions, etc.

[0909] The "means for adjusting notification time" is a function that optimizes the timing of notifications to the user based on the recognized emotional state.

[0910] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0911] The present invention relates to a notification system that combines learning of a user's behavioral patterns and emotion recognition in a smart device. A specific embodiment of this system will be described below.

[0912] 1. Generating the system program

[0913] The server generates an AI model to learn the user's behavioral patterns. This AI model is installed on the user's smart device. Machine learning algorithms are used to generate the AI ​​model. Specifically, programming languages ​​and libraries such as Python and TensorFlow are used.

[0914] 2. Explanation of program processing

[0915] The terminal (smart device) uses a built-in AI model to learn the user's behavioral patterns. This AI model collects the user's location information and schedule data, and analyzes the behavioral patterns based on this. Specifically, it collects data using the smart device's GPS function and calendar app.

[0916] Just before 8am, the device will notify the user to start getting ready for work, using the smart device's push notification feature.

[0917] Additionally, the AI ​​model will learn from users' common mistake of forgetting their phone during their commute and will remind them to remember to take their phone with them before they leave.

[0918] In one embodiment of the present invention, the AI ​​in the terminal includes an emotion engine. The emotion engine recognizes emotions from the user's tone of voice and facial expressions. For example, the emotion engine uses the camera and microphone of the smart device to analyze the user's facial expressions and tone of voice. Based on the recognized emotions, the AI ​​better assists the user in processing tasks. For example, if it recognizes that the user is feeling impatient or anxious, it advances the reminder notification time.

[0919] 3. Adding concrete examples

[0920] As a concrete example, if the AI ​​has learned that a user leaves for work at 8am every morning, the following prompts can be input to the generative AI model:

[0921] Example prompt:

[0922] "An AI that has learned the user's behavioral pattern of leaving for work at 8am every morning will notify the user to start preparing for work before 8am. Also, an AI that has learned the user's common mistake of forgetting their phone during the commute will notify the user not to forget to take their phone with them before leaving."

[0923] By inputting this prompt text into a generative AI model, the AI ​​will generate appropriate notifications based on the user's behavioral patterns.

[0924] As described above, the present invention provides a notification system that takes into account a user's behavioral patterns and emotional state, making the user's life more efficient and comfortable.

[0925] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0926] Step 1:

[0927] Collecting user behavior data

[0928] The terminal collects user location information, schedule data, call history, and other behavioral data. Specifically, it uses the smart device's GPS function, calendar app, call log, etc.

[0929] Input: GPS data, calendar data, call logs

[0930] Data processing: recording location information, obtaining schedules, saving call history

[0931] Output: Collected behavioral data

[0932] Specific operation: The device records the user's location information every morning between 7:45 and 8:00 a.m. It also retrieves commute-related events from the user's calendar app.

[0933] Step 2:

[0934] Learning behavioral patterns

[0935] The server uses the collected data to train an AI model to learn user behavior patterns, using machine learning algorithms to identify when a user leaves for work at 8am every morning.

[0936] Input: Collected behavioral data

[0937] Data Computing: Data Analysis with Machine Learning Algorithms

[0938] Output: Learned behavioral patterns

[0939] What happens: The server analyzes data from the past 30 days and finds a pattern of users commuting from home to work every morning at 8am.

[0940] Step 3:

[0941] Commuting preparation notification

[0942] Based on the learned behavioral patterns, the device will notify the user to start preparing for their commute every morning before 8:00 a.m. This notification will be sent using the push notification function of the smart device.

[0943] Input: Learned behavioral patterns

[0944] Data processing: Generation of notification messages

[0945] Output: Notification to prepare for commute

[0946] Specific operation: At 7:45, the device sends the user a notification saying, "Start getting ready for your commute."

[0947] Step 4:

[0948] Notifications to prevent lost items

[0949] The device learns that users often make the mistake of forgetting their phone during their commute and notifies them before they leave to remember to take their phone with them.

[0950] Input: Learned mistake patterns

[0951] Data processing: Generation of notification messages

[0952] Output: Notification to prevent loss

[0953] Specific behavior: The device sends the user a notification before 8 o'clock saying, "Don't forget to take your phone with you."

[0954] Step 5:

[0955] Emotion recognition and response

[0956] The emotion engine in the device recognizes emotions from the user's tone of voice and facial expressions. Based on the recognized emotions, the AI ​​can better assist the user in completing tasks. For example, if it recognizes that the user is feeling impatient or anxious, it will advance the reminder notification time.

[0957] Input: User's voice tone, facial expression data

[0958] Data Computation: Analysis with Emotion Recognition Algorithms

[0959] Output: Adjusted notification time

[0960] Specific behavior: If the device recognizes that the user is in a hurry, it will send a notification at 7:30 saying, "Please start getting ready for your commute."

[0961] (Application example 2)

[0962] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0963] Although the reminder and notification functions of conventional smart devices can learn the user's behavioral patterns, they have the problem of not being able to notify the user at the appropriate time while taking into account the user's emotional state. In addition, the function of notifying the user to bring items that the user often forgets is also limited, and it has not been possible to sufficiently improve user convenience.

[0964] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0965] In this invention, the server includes a means for collecting data in the smart device, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for recognizing the user's emotion, and a means for adjusting the timing of notification based on the recognized emotion. This enables notification at an appropriate timing taking into account the user's behavioral pattern and emotional state, thereby improving user convenience.

[0966] A "smart device" is an electronic device with advanced functions, such as a smartphone, smart glasses, a head-mounted display, or a robot.

[0967] "Means of collecting data" refers to the functions of sensors and software for obtaining information about the user's behavior and environment.

[0968] "Means for analyzing data" refers to algorithms or software functions that analyze collected data and identify user behavior patterns and trends.

[0969] "Means for predicting behavior" refers to algorithms or software functions for predicting a user's next behavior based on the analysis results.

[0970] The "means for supporting task processing" is a function for providing appropriate reminders and notifications to the user based on the prediction results, thereby enabling the user to efficiently process tasks.

[0971] "Means for recognizing emotions" refers to algorithms or software functions for detecting and recognizing emotions from the user's tone of voice, facial expressions, etc.

[0972] "Means for adjusting notification timing" refers to algorithms or software features that optimize the timing of reminders or notifications based on perceived emotions.

[0973] A system for carrying out the present invention includes a program installed in a smart device. The smart device may be a smartphone, smart glasses, a head-mounted display, or a robot. The system has a function of learning a user's behavioral patterns and providing reminders and notifications at appropriate times. It also has an emotion engine that recognizes the user's emotions and provides appropriate support.

[0974] Program processing explanation

[0975] 1. Data collection:

[0976] Smart devices use sensors and software to collect data about users' behavior and the environment, including GPS data, calendar information, and call history.

[0977] 2. Data Analysis:

[0978] The collected data is analyzed using algorithms and software within the device (e.g., Python data analysis libraries) to understand user behavior patterns and trends.

[0979] 3. Behavioral Prediction:

[0980] Based on the analysis results, it predicts the user's next action. For example, it learns that the user commutes to work at 8am every morning and sets a reminder to get ready for work before 8am.

[0981] 4. Task processing assistance:

[0982] Based on the prediction results, the system provides appropriate reminders and notifications to the user, for example, notifying the user to bring items that the user often forgets (such as a smartphone or keys).

[0983] 5. Emotion recognition:

[0984] The emotion engine recognizes the user's emotions from their tone of voice and facial expressions. For example, if the user is feeling impatient or anxious, the emotion engine will recognize this as "anxious."

[0985] 6. Notification timing adjustment:

[0986] The timing of reminders and notifications is optimized based on the recognized emotions. For example, if the user is feeling anxious or impatient, notifications can be sent earlier to reduce the user's anxiety.

[0987] Examples

[0988] As a concrete example, consider a user who commutes to work every morning at 8:00. If the user feels anxious or impatient, the AI ​​will notify them to "start preparing to leave early" and remind them not to forget their smartphone or keys.

[0989] Examples of prompt statements

[0990] Example of a prompt to be input to a generative AI model:

[0991] Create an application that learns the user's behavioral pattern of commuting to work every morning at 8am and displays reminders to prepare for the commute before the user leaves. Include a function that notifies the user to bring things that the user often forgets (e.g. smartphone or keys) and includes an emotion engine that recognizes emotions from the user's tone of voice and facial expressions and notifies them at the appropriate time.

[0992] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0993] Step 1:

[0994] Data collection

[0995] The device collects data about the user's behavior and environment using sensors and software. Specifically, it acquires GPS data, calendar information, call history, etc. The input is data about the user's behavior and environment, and the output is a set of collected data.

[0996] Step 2:

[0997] Data analysis

[0998] The terminal analyzes the collected data. Specifically, it uses a Python data analysis library to understand the user's behavioral patterns and trends. The input is a set of collected data, and the output is the analysis results on the user's behavioral patterns and trends.

[0999] Step 3:

[1000] Behavioral Prediction

[1001] The device predicts the user's next action based on the analysis results. Specifically, it learns the user's commute pattern at 8 a.m. every morning and sets a reminder to prepare for work before 8 a.m. The input is the analysis results on the user's behavioral patterns and tendencies, and the output is the user's predicted next action.

[1002] Step 4:

[1003] Task processing assistance

[1004] The device sends appropriate reminders and notifications to the user based on the prediction results. Specifically, it notifies the user to bring items that the user often forgets (such as a smartphone or keys). The input is the predicted next action of the user, and the output is a reminder or notification for the user.

[1005] Step 5:

[1006] emotion recognition

[1007] The device uses an emotion engine to recognize emotions from the user's tone of voice and facial expressions. Specifically, if the user is feeling impatient or anxious, the emotion engine recognizes this as "anxious." The input is data related to the user's tone of voice and facial expressions, and the output is the recognized emotional state of the user.

[1008] Step 6:

[1009] Notification timing adjustment

[1010] The device optimizes the timing of reminders and notifications based on the recognized emotions. Specifically, if the user feels impatient or anxious, the device advances the timing of notifications to reduce the user's anxiety. The input is the recognized user's emotional state, and the output is the adjusted timing of notifications.

[1011] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1012] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1013] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1014] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1015] [Third embodiment]

[1016] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1017] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1019] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1020] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1021] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1023] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1024] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1026] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1027] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1028] "Example 1"

[1029] In one embodiment of the present invention, the AI ​​in the smartphone has a function of collecting data from the smartphone, such as a user's operation history, location information, and communication history. In addition, as a means for analyzing the collected data, the AI ​​has a function of learning the user's behavioral patterns and predicting the user's next action. Furthermore, as a means for supporting the user's task processing based on the prediction results, the AI ​​has a function of providing the user with suggestions for the next action and information on how to prevent mistakes.

[1030] "Example 2"

[1031] As a specific example, if the AI ​​learns that the user leaves for work at 8 a.m. every morning, the AI ​​will suggest that the user start preparing for work before 8 a.m. In addition, if the AI ​​learns that the user often makes the mistake of forgetting their phone during their commute, the AI ​​will notify the user before leaving to remember to take their phone with them.

[1032] The process flow of each embodiment will be described below.

[1033] "Example 1"

[1034] Step 1: The AI ​​in the smartphone collects data such as the user's operation history, location information, and communication history.

[1035] Step 2: AI analyzes the collected data and learns user behavior patterns.

[1036] Step 3: Predict the user's next action based on the behavioral patterns the AI ​​has learned.

[1037] Step 4: Based on the prediction results, the AI ​​provides the user with suggestions for the next course of action and information on how to prevent mistakes.

[1038] "Example 2"

[1039] Step 1: The AI ​​learns the user's behavioral pattern: that they leave for work at 8am every morning.

[1040] Step 2: Just before 8am, the AI ​​suggests the user start getting ready for work.

[1041] Step 3: The AI ​​learns that users often make the mistake of forgetting their phone during their commute.

[1042] Step 4: Before leaving, the AI ​​notifies the user to remember to bring their phone.

[1043] Example 1

[1044] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1045] In conventional smartphone systems, it was difficult to effectively use data such as user operation history, location information, and communication history to predict user behavior and assist with task processing. In addition, there was a lack of appropriate means to learn the user's behavioral patterns and predict the next action, making it difficult to improve user convenience.

[1046] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1047] In this invention, the server includes a means for collecting data such as a user's operation history, location information, and communication history, a means for transmitting the collected data to the server, a means for the server to analyze the data received and learn the user's behavioral patterns, a means for predicting the user's next behavior based on the learned behavioral patterns, and a means for supporting the user's task processing based on the prediction results. This makes it possible to effectively predict the user's behavior and support the task processing at an appropriate time.

[1048] "User operation history" is a record of operations performed by a user on a smartphone.

[1049] "Location information" refers to information about a user's current location and movement history, obtained using a smartphone's GPS function or the like.

[1050] "Communication history" refers to records of calls and messages sent and received by a user via their smartphone.

[1051] A "server" is a computer system that collects, analyzes, stores, and transmits data over a network.

[1052] "Means of collecting data" refers to the function of obtaining data such as user operation history, location information, and communication history using sensors and software within the smartphone.

[1053] The "means for transmitting data to a server" is a function for transmitting collected data to a server via a network.

[1054] "Means for analyzing data" refers to a function for the server to process the data it receives and learn the user's behavioral patterns.

[1055] The "means for learning behavioral patterns" is a function that uses a machine learning algorithm to extract behavioral tendencies and patterns from the user's past data.

[1056] The "means for predicting the next action" is a function for predicting the next action that the user will take based on the learned behavioral patterns.

[1057] "Means to assist in task processing" is a function that provides the user with suggestions for the next course of action and information on how to prevent mistakes based on the prediction results.

[1058] The present invention relates to a system for assisting in task processing by collecting data in a smartphone, analyzing the data in a server, and predicting user behavior. A specific embodiment of this system will be described below.

[1059] First, the device collects data such as the user's operation history, location information, and communication history. This is done using the smartphone's built-in sensors, GPS, and communication module. For example, if a user frequently uses a particular app, its operation history is recorded.

[1060] Next, the terminal sends the collected data to the server. The data is encrypted and sent using a secure communication protocol (e.g., HTTPS). For example, the user's operation history and location information are sent to the server at regular intervals.

[1061] The server analyzes the received data. Specifically, it uses machine learning algorithms (e.g., TensorFlow or PyTorch) to learn the user's behavioral patterns. For example, the server analyzes the user's operation history and finds a pattern of using a specific app at a specific time of day.

[1062] The server then predicts the user's next action based on the learned behavioral patterns. For example, the server predicts that the user will arrive at work at 8:00 every morning.

[1063] Finally, the server sends a notification to the device based on the prediction result. The device displays the user's next action suggestion and information on how to prevent mistakes. For example, the device displays a notification saying, "It's time to start getting ready for work."

[1064] Examples

[1065] Example 1: Collecting operation history

[1066] The device records the time when the user opens the SNS app. For example, if the user opens the SNS app at 10 p.m. every night, that time is recorded as the operation history.

[1067] Example 2: Sending data

[1068] The device sends the recorded operation history to the server. For example, data that indicates that a social networking app was opened at 10 p.m. every night is sent to the server.

[1069] Example 3: Data analysis

[1070] The server analyzes the operation history it receives and learns the pattern of how users use the SNS app every night at 10 p.m. For example, it models this pattern using TensorFlow.

[1071] Example 4: Behavioral prediction

[1072] Based on the patterns learned by the server, the server predicts the next time the user will open the SNS app. For example, the server predicts that "the user will next open the SNS app at 10 p.m."

[1073] Example 5: Task assistance

[1074] The server sends the prediction result to the device, and the device displays a notification to the user. For example, at 9:50 p.m., the device displays a notification saying, "SNS time is approaching."

[1075] Examples of prompt statements

[1076] "Design an AI system that collects user operation history, location information, and communication history, learns behavioral patterns, and predicts the next action. Specifically, implement a function that learns behavioral patterns using TensorFlow and sends a notification to the user based on the prediction results."

[1077] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1078] Step 1:

[1079] The device collects data such as the user's operation history, location information, and communication history.

[1080] Input: User operations, location information, communication history

[1081] Specific behavior: Using the smartphone's built-in sensors, GPS, and communication module, it records information such as which apps the user used and when, where they were, and who they communicated with.

[1082] Output: Collected data (operation history, location information, communication history)

[1083] Step 2:

[1084] The terminal transmits the collected data to the server.

[1085] Input: Collected data (operation history, location information, communication history)

[1086] What it does: Encrypts the data and sends it to the server using a secure communications protocol (e.g. HTTPS).

[1087] Output: Data sent to the server

[1088] Step 3:

[1089] The server parses the received data.

[1090] Input: Data sent to the server

[1091] Specific operation: Using machine learning algorithms (e.g. TensorFlow and PyTorch), the system learns user behavior patterns. For example, the server analyzes the user's operation history and finds a pattern of using a specific app at a specific time of day.

[1092] Output: Learned behavioral patterns

[1093] Step 4:

[1094] The server predicts the next action based on the behavioral patterns it has learned.

[1095] Input: Learned behavioral patterns

[1096] Specific behavior: Using a generative AI model, the server predicts what the user will do next. For example, the server predicts that the user will arrive at work at 8am every morning.

[1097] Output: Predicted next action

[1098] Step 5:

[1099] The server sends a notification to the terminal based on the prediction result.

[1100] Input: Predicted next action

[1101] Specific operation: The server sends the prediction result to the device, and the device displays the user's next action suggestion and information on how to prevent mistakes. For example, the device displays a notification saying, "It's time to start getting ready for work."

[1102] Output: The notification displayed to the user

[1103] (Application example 1)

[1104] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1105] Conventional food delivery systems do not fully utilize the user's order history and behavioral patterns, making it difficult to suggest menu items or send reminders to users at the right time. In addition, they lack the ability to learn the user's preferences and behavioral patterns, so there is a need to improve the user experience.

[1106] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1107] In this invention, the server includes means for collecting data in the smartphone, means for analyzing the collected data, means for predicting the user's next action based on the analysis result, means for supporting the user's task processing based on the prediction result, means for collecting the user's order history, location information, and communication history, and learning the user's behavioral patterns, means for predicting the next order based on the learned behavioral patterns, means for providing recommended menus and promotion information at appropriate times, and means for sending reminders at specific times, thereby enabling personalized suggestions and reminders to be sent based on the user's preferences and behavioral patterns.

[1108] A "smartphone" is a multi-function mobile device that, in addition to the functions of a mobile phone, can also connect to the Internet and use applications.

[1109] "Means for collecting data" refers to functions and devices for acquiring information such as a user's operation history, location information, and communication history.

[1110] "Means for analyzing data" refers to functions or devices for analyzing collected data and understanding user behavior patterns and trends.

[1111] The "means for predicting the user's next action" refers to a function or device for predicting the action that the user is likely to take next based on the analysis results.

[1112] The "means for supporting the user's task processing" refers to a function or device for supporting the user so that the user can efficiently carry out a task based on the prediction result.

[1113] "Order history" is a record of orders a user has placed in the past.

[1114] "Location information" is information relating to the user's current location and past movement history.

[1115] "Communication history" is a record of calls and messages made by a user.

[1116] A "means for learning behavioral patterns" is a function or device for learning the behavioral tendencies and habits of a user using collected data.

[1117] The "means for predicting the next order" refers to a function or device for predicting the next product that the user is likely to order based on the learned behavioral patterns.

[1118] The "means for providing recommended menu items and promotion information" refers to a function or device for presenting appropriate menu items and discount information to the user based on the prediction results.

[1119] A "means for sending a reminder" is a function or device for sending a notification to a user at a specific time.

[1120] As an embodiment of the present invention, a system is provided in which a user's order history, location information, and communication history in a food delivery system are collected, and an AI analyzes the data to learn the user's behavioral patterns. A specific embodiment of this system is described below.

[1121] System Configuration

[1122] The system includes a smartphone, a server, and an AI model. The smartphone is hardware for collecting users' operation history, location information, and communication history. The server runs software for analyzing the collected data and learning users' behavioral patterns. The AI ​​model is used to predict the user's next action and provide recommended menus and promotional information at the appropriate time.

[1123] Collecting data

[1124] Smartphones collect users' operation history, location information, and communication history in real time. This includes GPS sensors, communication modules, and application log data. The collected data is sent to a server.

[1125] Analyzing data

[1126] The server receives the collected data and analyzes it using an AI model. Specifically, it uses programming languages ​​such as Python to pre-process the data, extract features, and train the model. The AI ​​model learns the user's behavioral patterns and predicts the next order.

[1127] Predicting and suggesting actions

[1128] Based on the learned behavioral patterns, the AI ​​model predicts what the user is likely to order next. Based on the prediction results, the server sends menu recommendations and promotion information to the smartphone. It also includes a function to send reminders at specific times.

[1129] Examples

[1130] For example, if a user has ordered "pizza," "sushi," "ramen," and "curry" in the past, the AI ​​model might recommend "pizza" next. It also sends a reminder at 12 o'clock saying, "It's lunchtime! Order your favorite menu!"

[1131] Examples of prompt statements

[1132] Here is an example of a prompt to be input to the generative AI model:

[1133] Use the user's past orders, location, and communication history to predict what their next menu recommendation will be, and add the ability to send them a reminder at a specific time.

[1134] In this way, personalized offers and reminders can be sent based on the user's preferences and behavioral patterns.

[1135] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1136] Step 1:

[1137] The smartphone collects the user's operation history, location information, and communication history. Specifically, it obtains location information using a GPS sensor, collects operation history from application log data, and records communication history through a communication module. This data is sent to a server in real time.

[1138] Input: User operation history, location information, communication history

[1139] Output: Sending collected data to a server

[1140] Step 2:

[1141] The server receives the data sent from the smartphone. The received data is stored in a database. The database organizes and stores data for each user.

[1142] Input: Data sent from smartphone

[1143] Output: Data stored in the database

[1144] Step 3:

[1145] The server preprocesses the data stored in the database, specifically by imputing missing values, removing outliers, normalizing the data, etc. This improves the quality of the data and makes it easier for subsequent analysis.

[1146] Input: Raw data stored in a database

[1147] Output: Preprocessed data

[1148] Step 4:

[1149] The server uses the preprocessed data to train the AI ​​model. Specifically, it uses a programming language such as Python to perform feature extraction, model selection, and model training. The AI ​​model learns the user's behavioral patterns.

[1150] Input: Preprocessed data

[1151] Output: A trained AI model

[1152] Step 5:

[1153] The server uses the trained AI model to predict the user's next action, specifically, guessing what menu item the user is likely to order next based on their past ordering history and behavioral patterns.

[1154] Input: Trained AI model, user's past order history

[1155] Output: Next order prediction result

[1156] Step 6:

[1157] Based on the prediction results, the server sends recommended menus and promotion information to the smartphone, specifically providing menus tailored to the user's preferences and currently available promotion information.

[1158] Input: Predicted result for next order

[1159] Output: Menu recommendations and promotional information sent to your smartphone

[1160] Step 7:

[1161] The server sends a reminder at a specific time. Specifically, the server sends a reminder to the smartphone at 12 o'clock saying "It's lunchtime! Order your favorite menu!" so that the user does not forget to order at lunchtime.

[1162] Input: current time, user behavior pattern

[1163] Output: Reminders sent to your smartphone

[1164] These steps enable personalized suggestions and reminders based on the user's preferences and behavioral patterns.

[1165] Example 2

[1166] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1167] Conventional smartphones and wearable devices can collect user behavior data, but there are only a limited number of systems that effectively use that data to support user behavior. In particular, there is a demand for systems that can learn user behavior patterns and notify users at the appropriate time to reduce user errors.

[1168] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting user behavior data, a means for transmitting the collected data to the server, a means for analyzing the data in the server and learning the user behavior pattern using an AI model, a means for generating notification content based on the learned behavior pattern, a means for transmitting the generated notification content to the terminal, and a means for displaying the notification on the terminal. This enables notification at an appropriate time based on the user's behavior pattern, and reduces user errors.

[1169] "User behavioral data" refers to information related to a user's behavior, such as the user's daily behavior, movement patterns, and application usage history.

[1170] "Server" refers to a computer system that receives user behavioral data, analyzes it, and trains the AI ​​model.

[1171] "AI model" refers to a model that uses machine learning algorithms to learn user behavior patterns and generate predictions and notifications.

[1172] A "behavioral pattern" refers to a tendency for a user to repeatedly perform an action at a particular time or in a particular situation.

[1173] "Notification content" refers to messages or alerts that are generated to prompt the user to take action.

[1174] "Terminal" refers to a device that is directly operated by a user, such as a smartphone or tablet.

[1175] "Notification" means a message or alert that is displayed to the User via a Device.

[1176] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1177] The present invention is a system that collects user behavior data and analyzes it using an AI model to provide notifications based on the user's behavior patterns. A specific embodiment of this system will be described below.

[1178] Collecting user behavior data

[1179] Users collect daily behavioral data using smartphones and wearable devices. For example, the GPS function of a smartphone is used to record the user's movement patterns. Data from calendar apps and reminder apps is also collected.

[1180] Sending behavioral data to the server

[1181] The device periodically transmits the collected behavioral data to a server, where the data is encrypted and transmitted using a secure communication protocol (e.g. HTTPS).

[1182] Data analysis on the server and training of AI models

[1183] The server analyzes the received behavioral data and uses an AI model to learn the user's behavioral patterns. Specifically, it uses Python and leverages machine learning libraries such as TensorFlow and PyTorch. For example, it learns that a user leaves for work at 8am every morning.

[1184] Generate notification content

[1185] The server generates notifications based on learned behavioral patterns. For example, it generates a notification "Start getting ready for work" when the user should start getting ready for work (7:50). Also, if the user tends to forget their phone, it generates a notification "Don't forget to take your phone with you."

[1186] Sending notifications to devices

[1187] The server sends the generated notification content to the terminal. The notification is sent in real time so that the user can receive it at the appropriate time.

[1188] Displaying notifications on your device

[1189] The device will display the received notifications to the user. Notifications will be displayed as pop-ups or push notifications so that the user can check them immediately.

[1190] Examples

[1191] As a concrete example, let's consider a case where the AI ​​has learned a user's behavioral pattern of leaving for work at 8:00 every morning. The AI ​​will notify the user at 7:50, "Please start preparing for your commute." Also, if the user has forgotten their phone many times in the past, the AI ​​will notify them, "Don't forget to take your phone with you."

[1192] Examples of prompt statements

[1193] An example of a prompt sentence might be:

[1194] Example prompt:

[1195] Design an AI system that learns user behavior patterns and notifies them at the right time. If a user leaves for work at 8am every morning, include a feature that notifies them to start getting ready for work at 7:50am and reminds users who tend to forget their phone to take it with them before leaving.

[1196] By inputting this prompt sentence into a generative AI model, a notification system design based on the user's behavioral patterns is generated.

[1197] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1198] Program processing flow

[1199] Step 1: Collect user behavior data

[1200] Users collect daily behavioral data using smartphones and wearable devices. Specifically, the GPS function of the smartphone is used to record the user's movement patterns. Data from calendar apps and reminder apps is also collected. The input is the user's behavioral data, and the output is the collected behavioral data.

[1201] Step 2: Sending behavioral data to the server

[1202] The device periodically transmits the collected behavioral data to the server. The data is encrypted and transmitted using a secure communication protocol (e.g., HTTPS). The input is the collected behavioral data, and the output is the behavioral data transmitted to the server.

[1203] Step 3: Data analysis on the server and training of the AI ​​model

[1204] The server analyzes the received behavioral data and uses an AI model to learn the user's behavioral patterns. Specifically, it uses Python to leverage machine learning libraries such as TensorFlow and PyTorch. The input is the behavioral data sent to the server, and the output is the learned behavioral pattern. For example, it learns a pattern that a user leaves for work at 8am every morning.

[1205] Step 4: Generate notification content

[1206] The server generates notification content based on the learned behavioral patterns. For example, it generates a notification "Start preparing for your commute" when the user should start preparing for their commute (7:50). Also, if the user tends to forget their phone, it generates a notification "Don't forget to take your phone with you." The input is the learned behavioral patterns, and the output is the generated notification content.

[1207] Step 5: Sending notifications to devices

[1208] The server sends the generated notification content to the terminal. The notification is sent in real time so that the user can receive it at the appropriate time. The input is the generated notification content, and the output is the notification content sent to the terminal.

[1209] Step 6: Display notifications on your device

[1210] The terminal displays the received notifications to the user. Notifications are displayed as popups or push notifications so that the user can see them immediately. The input is the notification content sent to the terminal, and the output is the notification displayed to the user.

[1211] Examples

[1212] As a concrete example, let's consider a case where the AI ​​has learned a user's behavioral pattern of leaving for work at 8:00 every morning. The AI ​​will notify the user at 7:50, "Please start preparing for your commute." Also, if the user has forgotten their phone many times in the past, the AI ​​will notify them, "Don't forget to take your phone with you."

[1213] (Application example 2)

[1214] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1215] Conventional user support systems using smart devices can learn the user's behavioral patterns and predict the user's next actions, but due to lack of coordination with the automobile, which is the user's means of transportation, they could not sufficiently improve the user's convenience. In addition, there was a lack of concrete measures to prevent users from forgetting things. This often caused inconvenience to users when commuting or traveling.

[1216] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1217] In this invention, the server includes a means for collecting data in the smart device, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for controlling the functions of the car based on the user's behavioral pattern, and a means for notifying the user of the user's forgotten items. This makes it possible to automatically control the functions of the car based on the user's behavioral pattern and prompt the user to make necessary preparations for commuting or traveling. In addition, by providing a notification function to prevent the user from forgetting things, the convenience of the user can be significantly improved.

[1218] A "smart device" is an electronic device that can be carried by a user and has functions such as data collection, analysis, and communication.

[1219] "Means of collecting data" refers to the function of obtaining information from various sensors and applications within smart devices.

[1220] "Means for analyzing data" refers to the function of processing collected data and analyzing user behavior patterns and trends.

[1221] "Means for predicting the user's next action" refers to a function for predicting the action that the user is likely to take next based on the analysis results.

[1222] "Means for supporting user task processing" refers to a function that supports the user to efficiently complete a task based on predicted behavior.

[1223] "Means for controlling the functions of an automobile" refers to a function for automatically operating functions such as starting the automobile engine and setting the air conditioning based on the user's behavioral patterns.

[1224] "Means for notifying a user of lost items" refers to a feature that notifies the user before departure to ensure that the user does not forget a particular item.

[1225] The system for carrying out this invention collects and analyzes data in smart devices, learns the user's behavioral patterns, predicts the user's next actions, and assists the user in processing tasks. It also has the function of controlling the functions of a car based on the user's behavioral patterns and notifying the user of lost items.

[1226] System Configuration

[1227] The system includes the following major hardware and software components:

[1228] Hardware: Smart devices (smartphones, smart glasses, head-mounted displays, etc.), infotainment systems for autonomous vehicles

[1229] Software: Python, smtplib (email sending library), data analysis libraries (e.g. Pandas, NumPy), machine learning libraries (e.g. scikit-learn)

[1230] Data collection and analysis

[1231] The server collects data from various sensors and applications in smart devices. The collected data is processed using a data analysis library to analyze user behavior patterns and trends. For example, a user's behavior pattern of commuting to work every morning at 8 o'clock is detected.

[1232] Behavioral prediction and task support

[1233] The server predicts the user's next action based on the analysis results. Based on the predicted action, the server supports the user to efficiently complete tasks. For example, when it is approaching commute time, the server notifies the user to prepare for commute.

[1234] Automotive Function Control

[1235] The system automatically controls the functions of the car based on the user's behavioral patterns, for example starting the car's engine and setting the air conditioner to an appropriate temperature when it is time to commute to work.

[1236] Lost property notification

[1237] By registering items that a user often forgets in advance, the system will notify the user to bring these items before leaving. For example, if a user often forgets their phone and wallet, the system will notify the user to bring these items before leaving.

[1238] Examples

[1239] As a concrete example, consider a user who commutes to work every morning at 8am and often forgets their phone and wallet. In this case, the system will start the engine before 8am, set the air conditioner, and notify the user to prepare for work. It will also notify the user to not forget their phone and wallet.

[1240] Examples of prompt statements

[1241] Here is an example of a prompt to be input to the generative AI model:

[1242] Design an AI system that learns the user's behavioral pattern of commuting to work every morning at 8:00, and then starts the engine before 8:00, sets the air conditioner to an appropriate temperature, and notifies the user to prepare for the commute. Also, if the user often forgets his / her phone and wallet during the commute, add a feature that notifies the user to have these items with him / her before leaving.

[1243] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1244] Step 1:

[1245] The server collects data from various sensors and applications within the smart device.

[1246] Input: Sensor data from smart devices (location, time, application usage history, etc.)

[1247] Output: Raw data collected

[1248] Specific operation: The server communicates with the smart device, obtains location information from the GPS sensor, and collects application usage history.

[1249] Step 2:

[1250] The server processes the collected data using a data analysis library to analyze user behavior patterns.

[1251] Input: Raw data collected

[1252] Output: Analyzed behavioral patterns (e.g. commuting to work at 8am every morning)

[1253] Specific operation: The server uses data analysis libraries such as Pandas and NumPy to organize the collected data in chronological order and extract user behavior patterns.

[1254] Step 3:

[1255] The server predicts the user's next action based on the analysis results.

[1256] Input: Analyzed behavioral patterns

[1257] Output: Predicted next action (e.g., start commuting at 8am)

[1258] Specific operation: The server uses a machine learning model (e.g., scikit-learn) to predict the next action based on past behavior patterns.

[1259] Step 4:

[1260] The server assists the user in processing tasks based on the predicted behavior.

[1261] Input: Predicted next action

[1262] Output: Notification to the user (e.g., notification to prepare for commute)

[1263] Specific operation: The server sends a notification to the user's smart device and displays a message encouraging them to prepare for their commute.

[1264] Step 5:

[1265] The server controls the functions of the vehicle based on the user's behavioral patterns.

[1266] Input: Predicted next action

[1267] Output: Car control instructions (e.g. engine start, air conditioning settings)

[1268] What it does: The server communicates with the autonomous vehicle's infotainment system, starts the engine, and sets the air conditioning to the right temperature.

[1269] Step 6:

[1270] The server notifies the user of items that he or she often forgets.

[1271] Input: A list of lost items registered by the user (e.g. phone, wallet)

[1272] Output: Lost item notification content

[1273] Specific operation: The server sends a notification to the user's smart device and displays a message reminding the user to have their phone and wallet ready before leaving.

[1274] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1275] "Example 1"

[1276] In one embodiment of the present invention, the AI ​​in the smartphone collects data such as the user's operation history, location information, and communication history. Furthermore, the collected data is analyzed to learn the user's behavioral patterns. Then, based on the learned behavioral patterns, the AI ​​predicts the user's next action. Based on the prediction result, the AI ​​provides the user with suggestions for the next action and information on preventing mistakes. Furthermore, in one embodiment of the present invention, the AI ​​includes an emotion engine. The emotion engine recognizes emotions from the user's tone of voice and facial expressions. Based on the recognized emotions, the AI ​​more appropriately supports the user in processing tasks. For example, if the AI ​​recognizes that the user is feeling stressed, the AI ​​supports the user in reducing stress by playing relaxation music and providing a message encouraging deep breathing.

[1277] "Example 2"

[1278] In another embodiment of the present invention, the AI ​​in the smartphone learns a behavioral pattern that the user leaves for work at 8 o'clock every morning. Before 8 o'clock, the AI ​​suggests to the user to start preparing for work. Furthermore, the AI ​​learns information that the user often makes the mistake of forgetting his / her phone during the commute. Before leaving, the AI ​​notifies the user not to forget to take the phone. Furthermore, in one embodiment of the present invention, the AI ​​includes an emotion engine. The emotion engine recognizes emotions from the user's tone of voice and facial expressions. Based on the recognized emotions, the AI ​​better supports the user in processing tasks. For example, if the AI ​​recognizes that the user is feeling impatient or anxious, the AI ​​reduces the user's anxiety by speeding up the reminder notification time.

[1279] The process flow of each embodiment will be described below.

[1280] "Example 1"

[1281] Step 1: The AI ​​collects data such as the user's operation history, location information, communication history, etc. Step 2: The AI ​​analyzes the collected data and learns the user's behavioral patterns.

[1282] Step 3: Predict the user's next action based on learned behavioral patterns.

[1283] Step 4: Based on the prediction results, the AI ​​provides the user with suggestions for next steps and information on how to prevent mistakes.

[1284] Step 5: The emotion engine recognizes emotions from the user's tone of voice and facial expressions.

[1285] Step 6: Based on the recognized emotions, the AI ​​will better assist the user in completing tasks. For example, if the AI ​​recognizes that the user is feeling stressed, it will help the user reduce stress by playing relaxation music or providing a message encouraging deep breathing. "Example 2"

[1286] Step 1: The AI ​​learns the user's behavioral pattern: that they leave for work at 8am every morning.

[1287] Step 2: Just before 8am, the AI ​​suggests the user start getting ready for work.

[1288] Step 3: The AI ​​learns that users often make the mistake of forgetting their phone during their commute. Before leaving, the AI ​​will notify the user to remember to take their phone with them.

[1289] Step 4: The emotion engine recognizes emotions from the user's tone of voice and facial expressions.

[1290] Step 5: Based on the recognized emotions, the AI ​​can better assist the user in completing the task. For example, if the AI ​​recognizes that the user is feeling anxious or impatient, it can speed up the reminder notification time to reduce the user's anxiety.

[1291] Example 1

[1292] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1293] Conventional smartphone systems can collect data such as user operation history, location information, and communication history, but it is difficult to effectively use this data to predict user behavior and provide appropriate task support. In addition, there was no system that recognized the user's emotions and provided task support based on them. This made it difficult to reduce user stress and mistakes, and improving the user experience was a challenge.

[1294] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1295] In this invention, the server includes a means for collecting data such as a user's operation history, location information, and communication history, a means for analyzing the collected data and learning the user's behavioral patterns, a means for predicting the user's next behavior based on the learned behavioral patterns, a means for supporting the user's task processing based on the prediction results, a means for recognizing emotions from the user's tone of voice and facial expressions, and a means for appropriately supporting the user's task processing based on the recognized emotions. This makes it possible to predict the user's behavior and provide appropriate task support, reducing user stress and mistakes and improving the user experience.

[1296] "User operation history" is a record of operations performed by a user on a smartphone, and is data including the time spent using an app and the location of taps.

[1297] "Location information" is data about a user's current location and movement history obtained using a smartphone's GPS function, etc.

[1298] "Communication history" refers to records of calls and messages sent and received by a user, and is data including call history and message contents.

[1299] "Means of collecting data" refers to the function of obtaining data such as user operation history, location information, and communication history using sensors and communication modules within the smartphone.

[1300] "Means for analyzing data" refers to functions that use machine learning algorithms and databases to analyze collected data and extract user behavior patterns.

[1301] "Means for learning behavioral patterns" refers to a function that uses a machine learning algorithm to extract patterns from a user's past behavioral data and generate a model for predicting the user's next behavior.

[1302] "Means for predicting the next action" refers to a function for predicting the next action that the user is likely to take based on learned behavioral patterns.

[1303] "Means to assist in task processing" refers to functions that suggest the user's next course of action based on prediction results or provide information to prevent mistakes.

[1304] "Means for recognizing emotions" refers to functionality that uses voice recognition software and a camera module to analyze the user's tone of voice and facial expressions to determine the user's emotional state.

[1305] "Means for appropriately supporting task processing based on emotions" refers to a function that provides assistance such as playing relaxation music or providing a message encouraging the user to take a deep breath, depending on the recognized emotions.

[1306] This invention is a system that uses AI in a smartphone to collect data such as a user's operation history, location information, and communication history, analyzes this data, learns the user's behavioral patterns, and predicts the user's next actions. Furthermore, it has a function to recognize the user's emotions from their tone of voice and facial expressions, and appropriately support the user's task processing based on the recognized emotions.

[1307] Data collection

[1308] When a user operates a smartphone, the device collects data such as the user's operation history, location information, and communication history. This data is collected using hardware such as the smartphone's built-in sensors, GPS, and communication modules. In addition, the smartphone's storage and dedicated database software are used to collect and store the data.

[1309] Analyzing data and learning behavioral patterns

[1310] The data collected by the device is sent to a server. The server stores the received data in a database and analyzes the data using machine learning algorithms. Specifically, frameworks such as TensorFlow and PyTorch are used to extract patterns from the user's past behavior data and generate a model to predict the user's next behavior.

[1311] Behavioral prediction and suggestions

[1312] The server sends the generated prediction model to the device. The device uses the prediction model to predict the user's next action. For example, if a user has the habit of opening a specific app at 8am every morning, the AI ​​will suggest automatically opening the app at that time. Also, if a user has the habit of calling a specific contact when they are in a specific location, the AI ​​will suggest making the call when they arrive at that location.

[1313] Emotion Recognition and Task Assistance

[1314] When a user is using a smartphone, the device analyzes the user's tone of voice and facial expressions with an emotion engine. Voice recognition software and a camera module are used for this emotion recognition. For example, Google Cloud Speech-to-Text API and OpenCV are used. Based on the recognized emotion, the AI ​​assists the user in completing tasks. For example, if it recognizes that the user is feeling stressed, it will play relaxation music and provide a message encouraging deep breathing.

[1315] Examples and prompts

[1316] As a concrete example, suppose a user has the habit of opening the calendar app at 8 am every morning. The AI ​​learns this behavioral pattern and suggests opening the calendar app at 8 am every morning. Also, if the AI ​​recognizes that the user is feeling stressed, it will play relaxation music.

[1317] Examples of prompts include, "If the user has the habit of opening the calendar app at 8am every morning, the AI ​​will suggest opening the calendar app at that time. Also, if it recognizes that the user is feeling stressed, it will suggest playing relaxation music."

[1318] In this way, the AI ​​in the smartphone learns the user's behavioral patterns and predicts and suggests the next action to assist the user in completing tasks. In addition, it recognizes the user's emotions using an emotion engine and provides appropriate assistance.

[1319] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1320] Step 1:

[1321] The user operates the smartphone.

[1322] When a user opens an app or sends a message, data such as operation history, location information, and communication history is generated. This data becomes the input.

[1323] Step 2:

[1324] The device collects data such as the user's operation history, location information, and communication history.

[1325] The device collects data in real time using built-in sensors, GPS, and communication modules. The collected data is stored in the internal storage, which is the output.

[1326] Step 3:

[1327] The terminal transmits the collected data to the server.

[1328] The terminal transmits the collected data to the server at regular intervals using the HTTPS protocol. The transmitted data becomes the input.

[1329] Step 4:

[1330] The server stores the received data in a database.

[1331] The server stores the received data in a database (e.g. MySQL, PostgreSQL). The stored data becomes the output.

[1332] Step 5:

[1333] The server analyzes the data using machine learning algorithms.

[1334] The server uses frameworks such as TensorFlow and PyTorch to analyze the stored data. Specifically, it performs clustering and classification to extract user behavior patterns. The analysis results are output.

[1335] Step 6:

[1336] The server learns user behavior patterns and generates a predictive model.

[1337] Based on the analysis results, the server generates a model to predict the user's next action. This model is the output.

[1338] Step 7:

[1339] The server sends the generated predictive model to the terminal.

[1340] The server uses a secure communication protocol (e.g. SSL / TLS) to send the prediction model to the terminal. The sent model becomes the input.

[1341] Step 8:

[1342] The device uses the predictive model to predict the user's next action.

[1343] The device uses the received prediction model to predict the user's next action in real time. For example, if the user has the habit of opening the calendar app at 8:00 every morning, the device will suggest opening the calendar app before 8:00. The prediction result is the output.

[1344] Step 9:

[1345] The device suggests the user's next action based on the prediction results.

[1346] Based on the prediction results, the device will suggest the next action to the user through notifications or pop-up messages. For example, it may display a message such as "It's 8 o'clock. Would you like to open the calendar app?". The suggestions are the output.

[1347] Step 10:

[1348] When a user uses a smartphone, the device analyzes the user's tone of voice and facial expressions using an emotion engine.

[1349] The device uses a built-in camera and microphone to analyze the user's tone of voice and facial expressions in real time, and the analysis results become input.

[1350] Step 11:

[1351] The device recognizes the user's emotions based on the analysis results.

[1352] The device uses voice recognition software and a camera module (e.g., Google Cloud Speech-to-Text API, OpenCV) to recognize the user's emotions. The recognition results are the output.

[1353] Step 12:

[1354] The device provides appropriate task assistance based on the emotions it recognizes.

[1355] Depending on the emotion recognized, the device will play relaxation music or display a message encouraging deep breathing. For example, if it recognizes that the user is feeling stressed, it will play relaxation music. The support content is the output.

[1356] (Application example 1)

[1357] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1358] Conventional food delivery applications do not take into account the user's behavioral patterns or emotions when making suggestions, making it difficult to improve user satisfaction. In addition, if the user is feeling stressed, the application may not be able to respond appropriately, resulting in a poor user experience.

[1359] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1360] In this invention, the server includes a means for collecting data in the smartphone, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for recognizing the user's emotion, and a means for making an appropriate suggestion to the user based on the recognized emotion. This makes it possible to make a suggestion that takes into account the user's behavioral patterns and emotions, thereby improving the user's satisfaction. In addition, when the user is feeling stressed, a relaxation menu or a special offer can be provided to improve the user experience.

[1361] "Data within a smartphone" refers to various information collected through a smartphone, such as a user's operation history, location information, and communication history.

[1362] "Means of collection" refers to the functions and processes for acquiring and storing data within a smartphone.

[1363] "Means of analysis" refers to the functions and processes for analyzing collected data and understanding user behavior patterns and trends.

[1364] The "means for predicting the user's next action" refers to a function or process for estimating the action that the user is likely to take next based on the analysis results.

[1365] The "means for supporting user task processing" refers to functions and processes for supporting the user to efficiently perform tasks based on the prediction results.

[1366] "Means for recognizing the user's emotions" refers to a function or process for determining the emotions of the user from the user's tone of voice, facial expression, etc.

[1367] The "means for making appropriate suggestions to the user based on the recognized emotions" refers to a function or process for providing optimal actions or suggestions depending on the user's emotional state.

[1368] The system for realizing the present invention has a series of functions for collecting and analyzing data in a smartphone, predicting the user's next action, and making appropriate suggestions. A specific embodiment of this system will be described below.

[1369] 1. Data Collection

[1370] The server uses the means for collecting data within the smartphone to obtain data such as the user's operation history, location information, communication history, etc. This data collection is done automatically from various sensors and applications on the smartphone.

[1371] 2. Data Analysis

[1372] The collected data is analyzed by an analysis method in the server. This analysis method is implemented using a programming language such as Python and includes an algorithm for learning the user's behavioral patterns. Specifically, the method grasps the user's behavioral tendencies based on past operation history and location information.

[1373] 3. Behavioral Prediction

[1374] The server includes a means for predicting the user's next action based on the analysis results. This prediction means is implemented using a machine learning model to estimate the action that the user is likely to take next. For example, if past data shows that the user tends to visit a specific restaurant at a specific time, the server suggests that restaurant.

[1375] 4. Task processing support

[1376] The server has a means to assist the user in processing tasks based on the prediction results. Specifically, the server suggests appropriate restaurants and menus so that the user can perform tasks efficiently. If the user feels stressed, the server provides relaxation menus and special offers.

[1377] 5. Emotion recognition

[1378] The server is equipped with a means for recognizing the user's emotions. This emotion recognition is performed by analyzing the user's tone of voice and facial expressions. An emotion recognition engine (EmotionEngine) is used to determine whether the user is feeling stressed.

[1379] 6. Providing suggestions

[1380] The server has a means for making appropriate suggestions to the user based on the recognized emotion. For example, if the user feels stressed, the server can suggest a relaxation menu. As a concrete example, the following prompt sentence can be used:

[1381] Examples of prompt statements

[1382] "If the user is feeling stressed, suggest a relaxation menu."

[1383] In this way, the system can improve user satisfaction by making suggestions that take into account the user's behavioral patterns and emotions. Also, if the user is feeling stressed, the system can improve the user experience by providing relaxation menus and special offers.

[1384] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1385] Step 1:

[1386] The server collects data from the smartphone. Specifically, it automatically obtains data such as the user's operation history, location information, and communication history from various sensors and applications on the smartphone. The input is data from the smartphone, and the output is a set of collected data.

[1387] Step 2:

[1388] The server analyzes the collected data. Using a programming language such as Python, it executes an algorithm to learn the user's behavioral patterns. The input is the data collected in step 1, and the output is the analysis results on the user's behavioral patterns. Specifically, it analyzes the user's behavioral tendencies based on past operation history and location information.

[1389] Step 3:

[1390] The server predicts the user's next action based on the analysis results. A machine learning model is used to estimate the action the user is likely to take next. The input is the analysis result from step 2, and the output is the predicted next action. Specifically, if past data shows that the user has a tendency to use a specific restaurant at a specific time of day, the server suggests that restaurant.

[1391] Step 4:

[1392] The server assists the user in processing tasks based on the prediction results. Specifically, it suggests appropriate restaurants and menus so that the user can perform the task efficiently. The input is the prediction result from step 3, and the output is the suggestions to the user. Specifically, it notifies the user of recommended restaurants and menus.

[1393] Step 5:

[1394] The server recognizes the user's emotions. By analyzing the user's tone of voice and facial expressions, an emotion recognition engine (EmotionEngine) is used to determine whether the user is feeling stressed. The input is the user's tone of voice and facial expression data, and the output is the recognized emotional state. Specific operations include voice analysis and image analysis.

[1395] Step 6:

[1396] The server makes appropriate suggestions to the user based on the recognized emotions. If the user is feeling stressed, it will provide a relaxation menu or a special offer. The input is the emotion recognition result from step 5, and the output is the content of the suggestions to the user. Specifically, it will suggest a relaxation menu or notify the user of a special offer.

[1397] In this way, the server can make suggestions that take into account the user's behavioral patterns and emotions through a series of processing steps, thereby improving the user's satisfaction.

[1398] Example 2

[1399] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1400] Conventional smart devices have limited functionality for learning the user's behavioral patterns and predicting the user's next actions, and they do not provide appropriate notifications or support that take into account the user's emotional state. In particular, when the user is feeling impatient or anxious, there is a problem that notifications are not provided at the appropriate time, making it difficult to reduce the user's stress.

[1401] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1402] In this invention, the server includes a means for collecting data in the smart device, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for recognizing the user's emotion, and a means for adjusting the notification time based on the recognized emotion. This makes it possible to not only learn the user's behavioral pattern and predict the user's next action, but also to notify and support the user at an appropriate time taking into account the user's emotional state.

[1403] A "smart device" is an electronic device that can be carried by a user and has functions such as data collection, analysis, and notification.

[1404] "Means of collecting data" refers to the function of obtaining information about user behavior and the environment from various sensors and applications within smart devices.

[1405] "Means for analyzing data" refers to algorithms or software used to analyze collected data and identify user behavior patterns and trends.

[1406] The "means for predicting the user's next action" is a function for predicting the next action that the user should take based on the analysis results.

[1407] The "means for supporting the user in processing a task" is a function that supports the user to efficiently complete a task based on predicted behavior.

[1408] "Means for recognizing the user's emotions" refers to technology for detecting the user's emotional state from the user's tone of voice, facial expressions, etc.

[1409] The "means for adjusting notification time" is a function that optimizes the timing of notifications to the user based on the recognized emotional state.

[1410] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1411] The present invention relates to a notification system that combines learning of a user's behavioral patterns and emotion recognition in a smart device. A specific embodiment of this system will be described below.

[1412] 1. Generating the system program

[1413] The server generates an AI model to learn the user's behavioral patterns. This AI model is installed on the user's smart device. Machine learning algorithms are used to generate the AI ​​model. Specifically, programming languages ​​and libraries such as Python and TensorFlow are used.

[1414] 2. Explanation of program processing

[1415] The terminal (smart device) uses a built-in AI model to learn the user's behavioral patterns. This AI model collects the user's location information and schedule data, and analyzes the behavioral patterns based on this. Specifically, it collects data using the smart device's GPS function and calendar app.

[1416] Just before 8am, the device will notify the user to start getting ready for work, using the smart device's push notification function.

[1417] Additionally, the AI ​​model will learn from users' common mistake of forgetting their phone during their commute and will remind them to remember to take their phone with them before they leave.

[1418] In one embodiment of the present invention, the AI ​​in the terminal includes an emotion engine. The emotion engine recognizes emotions from the user's tone of voice and facial expressions. For example, the emotion engine uses the camera and microphone of the smart device to analyze the user's facial expressions and tone of voice. Based on the recognized emotions, the AI ​​better assists the user in processing tasks. For example, if it recognizes that the user is feeling impatient or anxious, it advances the reminder notification time.

[1419] 3. Adding concrete examples

[1420] As a concrete example, if the AI ​​has learned that a user leaves for work at 8am every morning, the following prompts can be input to the generative AI model:

[1421] Example prompt:

[1422] "An AI that has learned the user's behavioral pattern of leaving for work at 8am every morning will notify the user to start preparing for work before 8am. Also, an AI that has learned the user's common mistake of forgetting their phone during the commute will notify the user not to forget to take their phone with them before leaving."

[1423] By inputting this prompt text into a generative AI model, the AI ​​will generate appropriate notifications based on the user's behavioral patterns.

[1424] As described above, the present invention provides a notification system that takes into account a user's behavioral patterns and emotional state, making the user's life more efficient and comfortable.

[1425] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1426] Step 1:

[1427] Collecting user behavior data

[1428] The terminal collects user location information, schedule data, call history, and other behavioral data. Specifically, it uses the GPS function, calendar app, call log, etc. of the smart device.

[1429] Input: GPS data, calendar data, call logs

[1430] Data processing: recording location information, obtaining schedules, saving call history

[1431] Output: Collected behavioral data

[1432] Specific operation: The device records the user's location information every morning between 7:45 and 8:00 a.m. It also retrieves commute-related events from the user's calendar app.

[1433] Step 2:

[1434] Learning behavioral patterns

[1435] The server uses the collected data to train an AI model to learn user behavior patterns, using machine learning algorithms to identify when a user leaves for work at 8am every morning.

[1436] Input: Collected behavioral data

[1437] Data Computing: Data Analysis with Machine Learning Algorithms

[1438] Output: Learned behavioral patterns

[1439] What happens: The server analyzes data from the past 30 days and finds a pattern of users commuting from home to work every morning at 8am.

[1440] Step 3:

[1441] Commuting preparation notification

[1442] Based on the learned behavioral patterns, the device will notify the user to start preparing for their commute every morning before 8:00 a.m. This notification will be sent using the push notification function of the smart device.

[1443] Input: Learned behavioral patterns

[1444] Data processing: Generation of notification messages

[1445] Output: Notification to prepare for commute

[1446] Specific operation: At 7:45, the device sends the user a notification saying, "Start getting ready for your commute."

[1447] Step 4:

[1448] Notifications to prevent lost items

[1449] The device learns that users often make the mistake of forgetting their phone during their commute and notifies them before they leave to remember to take their phone with them.

[1450] Input: Learned mistake patterns

[1451] Data processing: Generation of notification messages

[1452] Output: Notification to prevent loss

[1453] Specific behavior: The device sends the user a notification before 8 o'clock saying, "Don't forget to take your phone with you."

[1454] Step 5:

[1455] Emotion recognition and response

[1456] The emotion engine in the device recognizes emotions from the user's tone of voice and facial expressions. Based on the recognized emotions, the AI ​​can better assist the user in completing tasks. For example, if it recognizes that the user is feeling impatient or anxious, it will advance the reminder notification time.

[1457] Input: User's voice tone, facial expression data

[1458] Data Computation: Analysis with Emotion Recognition Algorithms

[1459] Output: Adjusted notification time

[1460] Specific behavior: If the device recognizes that the user is in a hurry, it will send a notification at 7:30 saying, "Please start getting ready for your commute."

[1461] (Application example 2)

[1462] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1463] Although the reminder and notification functions of conventional smart devices can learn the user's behavioral patterns, they have the problem of not being able to notify the user at the appropriate time while taking into account the user's emotional state. In addition, the function of notifying the user to bring items that the user often forgets is also limited, and it has not been possible to sufficiently improve user convenience.

[1464] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1465] In this invention, the server includes a means for collecting data in the smart device, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for recognizing the user's emotion, and a means for adjusting the timing of notification based on the recognized emotion. This enables notification at an appropriate timing taking into account the user's behavioral pattern and emotional state, thereby improving user convenience.

[1466] A "smart device" is an electronic device with advanced functions, such as a smartphone, smart glasses, a head-mounted display, or a robot.

[1467] "Means of collecting data" refers to the functions of sensors and software for obtaining information about the user's behavior and environment.

[1468] "Means for analyzing data" refers to algorithms or software functions that analyze collected data and identify user behavior patterns and trends.

[1469] "Means for predicting behavior" refers to algorithms or software functions for predicting a user's next behavior based on the analysis results.

[1470] The "means for supporting task processing" is a function for providing appropriate reminders and notifications to the user based on the prediction results, thereby enabling the user to efficiently process tasks.

[1471] "Means for recognizing emotions" refers to algorithms or software functions for detecting and recognizing emotions from the user's tone of voice, facial expressions, etc.

[1472] "Means for adjusting notification timing" refers to algorithms or software features that optimize the timing of reminders or notifications based on perceived emotions.

[1473] A system for carrying out the present invention includes a program installed in a smart device. The smart device may be a smartphone, smart glasses, a head-mounted display, or a robot. The system has a function of learning a user's behavioral patterns and providing reminders and notifications at appropriate times. It also has an emotion engine that recognizes the user's emotions and provides appropriate support.

[1474] Program processing explanation

[1475] 1. Data collection:

[1476] Smart devices use sensors and software to collect data about users' behavior and the environment, including GPS data, calendar information, and call history.

[1477] 2. Data Analysis:

[1478] The collected data is analyzed using algorithms and software within the device (e.g., Python data analysis libraries) to understand user behavior patterns and trends.

[1479] 3. Behavioral Prediction:

[1480] Based on the analysis results, it predicts the user's next action. For example, it learns that the user commutes to work at 8am every morning and sets a reminder to get ready for work before 8am.

[1481] 4. Task processing assistance:

[1482] Based on the prediction results, the system provides appropriate reminders and notifications to the user, for example, notifying the user to bring items that the user often forgets (such as a smartphone or keys).

[1483] 5. Emotion recognition:

[1484] The emotion engine recognizes the user's emotions from their tone of voice and facial expressions. For example, if the user is feeling impatient or anxious, the emotion engine will recognize this as "anxious."

[1485] 6. Notification timing adjustment:

[1486] The timing of reminders and notifications is optimized based on the recognized emotions. For example, if the user is feeling anxious or impatient, notifications can be sent earlier to reduce the user's anxiety.

[1487] Examples

[1488] As a concrete example, consider a user who commutes to work every morning at 8:00. If the user feels anxious or impatient, the AI ​​will notify them to "start preparing to leave early" and remind them not to forget their smartphone or keys.

[1489] Examples of prompt statements

[1490] Example of a prompt to be input to a generative AI model:

[1491] Create an application that learns the user's behavioral pattern of commuting to work every morning at 8am and displays reminders to prepare for the commute before the user leaves. Include a function that notifies the user to bring things that the user often forgets (e.g. smartphone or keys) and includes an emotion engine that recognizes emotions from the user's tone of voice and facial expressions and notifies them at the appropriate time.

[1492] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1493] Step 1:

[1494] Data collection

[1495] The device collects data about the user's behavior and environment using sensors and software. Specifically, it acquires GPS data, calendar information, call history, etc. The input is data about the user's behavior and environment, and the output is a set of collected data.

[1496] Step 2:

[1497] Data analysis

[1498] The terminal analyzes the collected data. Specifically, it uses a Python data analysis library to understand the user's behavioral patterns and trends. The input is a set of collected data, and the output is the analysis results on the user's behavioral patterns and trends.

[1499] Step 3:

[1500] Behavioral Prediction

[1501] The device predicts the user's next action based on the analysis results. Specifically, it learns the user's commute pattern at 8 a.m. every morning and sets a reminder to prepare for work before 8 a.m. The input is the analysis results on the user's behavioral patterns and tendencies, and the output is the user's predicted next action.

[1502] Step 4:

[1503] Task processing assistance

[1504] The device sends appropriate reminders and notifications to the user based on the prediction results. Specifically, it notifies the user to bring items that the user often forgets (such as a smartphone or keys). The input is the predicted next action of the user, and the output is a reminder or notification for the user.

[1505] Step 5:

[1506] emotion recognition

[1507] The device uses an emotion engine to recognize emotions from the user's tone of voice and facial expressions. Specifically, if the user is feeling impatient or anxious, the emotion engine recognizes this as "anxious." The input is data related to the user's tone of voice and facial expressions, and the output is the recognized emotional state of the user.

[1508] Step 6:

[1509] Notification timing adjustment

[1510] The device optimizes the timing of reminders and notifications based on the recognized emotions. Specifically, if the user feels impatient or anxious, the device advances the timing of notifications to reduce the user's anxiety. The input is the recognized user's emotional state, and the output is the adjusted timing of notifications.

[1511] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1512] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1513] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1514] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1515] [Fourth embodiment]

[1516] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1517] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1518] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1519] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1520] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1521] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1522] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1523] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1524] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1525] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1526] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1527] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1528] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1529] "Example 1"

[1530] In one embodiment of the present invention, the AI ​​in the smartphone has a function of collecting data from the smartphone, such as a user's operation history, location information, and communication history. In addition, as a means for analyzing the collected data, the AI ​​has a function of learning the user's behavioral patterns and predicting the user's next action. Furthermore, as a means for supporting the user's task processing based on the prediction results, the AI ​​has a function of providing the user with suggestions for the next action and information on how to prevent mistakes.

[1531] "Example 2"

[1532] As a specific example, if the AI ​​learns that the user leaves for work at 8 a.m. every morning, the AI ​​will suggest that the user start preparing for work before 8 a.m. In addition, if the AI ​​learns that the user often makes the mistake of forgetting their phone during their commute, the AI ​​will notify the user before leaving to remember to take their phone with them.

[1533] The process flow of each embodiment will be described below.

[1534] "Example 1"

[1535] Step 1: The AI ​​in the smartphone collects data such as the user's operation history, location information, and communication history.

[1536] Step 2: AI analyzes the collected data and learns user behavior patterns.

[1537] Step 3: Predict the user's next action based on the behavioral patterns the AI ​​has learned.

[1538] Step 4: Based on the prediction results, the AI ​​provides the user with suggestions for the next course of action and information on how to prevent mistakes.

[1539] "Example 2"

[1540] Step 1: The AI ​​learns the user's behavioral pattern: that they leave for work at 8am every morning.

[1541] Step 2: Just before 8am, the AI ​​suggests the user start getting ready for work.

[1542] Step 3: The AI ​​learns that users often make the mistake of forgetting their phone during their commute.

[1543] Step 4: Before leaving, the AI ​​notifies the user to remember to bring their phone.

[1544] Example 1

[1545] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1546] In conventional smartphone systems, it was difficult to effectively use data such as user operation history, location information, and communication history to predict user behavior and assist with task processing. In addition, there was a lack of appropriate means to learn the user's behavioral patterns and predict the next action, making it difficult to improve user convenience.

[1547] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1548] In this invention, the server includes a means for collecting data such as a user's operation history, location information, and communication history, a means for transmitting the collected data to the server, a means for the server to analyze the data received and learn the user's behavioral patterns, a means for predicting the user's next behavior based on the learned behavioral patterns, and a means for supporting the user's task processing based on the prediction results. This makes it possible to effectively predict the user's behavior and support the task processing at an appropriate time.

[1549] "User operation history" is a record of operations performed by a user on a smartphone.

[1550] "Location information" refers to information about a user's current location and movement history, obtained using a smartphone's GPS function or the like.

[1551] "Communication history" refers to records of calls and messages sent and received by a user via their smartphone.

[1552] A "server" is a computer system that collects, analyzes, stores, and transmits data over a network.

[1553] "Means of collecting data" refers to the function of obtaining data such as user operation history, location information, and communication history using sensors and software within the smartphone.

[1554] The "means for transmitting data to a server" is a function for transmitting collected data to a server via a network.

[1555] "Means for analyzing data" refers to a function for the server to process the data it receives and learn the user's behavioral patterns.

[1556] The "means for learning behavioral patterns" is a function that uses a machine learning algorithm to extract behavioral tendencies and patterns from the user's past data.

[1557] The "means for predicting the next action" is a function for predicting the next action that the user will take based on the learned behavioral patterns.

[1558] "Means to assist in task processing" is a function that provides the user with suggestions for the next course of action and information on how to prevent mistakes based on the prediction results.

[1559] The present invention relates to a system for assisting in task processing by collecting data in a smartphone, analyzing the data in a server, and predicting user behavior. A specific embodiment of this system will be described below.

[1560] First, the device collects data such as the user's operation history, location information, and communication history. This is done using the smartphone's built-in sensors, GPS, and communication module. For example, if a user frequently uses a particular app, its operation history is recorded.

[1561] Next, the terminal sends the collected data to the server. The data is encrypted and sent using a secure communication protocol (e.g., HTTPS). For example, the user's operation history and location information are sent to the server at regular intervals.

[1562] The server analyzes the received data. Specifically, it uses machine learning algorithms (e.g., TensorFlow or PyTorch) to learn the user's behavioral patterns. For example, the server analyzes the user's operation history and finds a pattern of using a specific app at a specific time of day.

[1563] The server then predicts the user's next action based on the learned behavioral patterns. For example, the server predicts that the user will arrive at work at 8:00 every morning.

[1564] Finally, the server sends a notification to the device based on the prediction result. The device displays the user's next action suggestion and information on how to prevent mistakes. For example, the device displays a notification saying, "It's time to start getting ready for work."

[1565] Examples

[1566] Example 1: Collecting operation history

[1567] The device records the time when the user opens the SNS app. For example, if the user opens the SNS app at 10 p.m. every night, that time is recorded as the operation history.

[1568] Example 2: Sending data

[1569] The device sends the recorded operation history to the server. For example, data that indicates that a social networking app was opened at 10 p.m. every night is sent to the server.

[1570] Example 3: Data analysis

[1571] The server analyzes the operation history it receives and learns the pattern of how users use the SNS app every night at 10 p.m. For example, it models this pattern using TensorFlow.

[1572] Example 4: Behavioral prediction

[1573] Based on the patterns learned by the server, the server predicts the next time the user will open the SNS app. For example, the server predicts that "the user will next open the SNS app at 10 p.m."

[1574] Example 5: Task assistance

[1575] The server sends the prediction result to the device, and the device displays a notification to the user. For example, at 9:50 p.m., the device displays a notification saying, "SNS time is approaching."

[1576] Examples of prompt statements

[1577] "Design an AI system that collects user operation history, location information, and communication history, learns behavioral patterns, and predicts the next action. Specifically, implement a function that learns behavioral patterns using TensorFlow and sends a notification to the user based on the prediction results."

[1578] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1579] Step 1:

[1580] The device collects data such as the user's operation history, location information, and communication history.

[1581] Input: User operations, location information, communication history

[1582] Specific behavior: Using the smartphone's built-in sensors, GPS, and communication module, it records information such as which apps the user used and when, where they were, and who they communicated with.

[1583] Output: Collected data (operation history, location information, communication history)

[1584] Step 2:

[1585] The terminal transmits the collected data to the server.

[1586] Input: Collected data (operation history, location information, communication history)

[1587] What it does: Encrypts the data and sends it to the server using a secure communications protocol (e.g. HTTPS).

[1588] Output: Data sent to the server

[1589] Step 3:

[1590] The server parses the received data.

[1591] Input: Data sent to the server

[1592] Specific operation: Using machine learning algorithms (e.g. TensorFlow and PyTorch), the system learns user behavior patterns. For example, the server analyzes the user's operation history and finds a pattern of using a specific app at a specific time of day.

[1593] Output: Learned behavioral patterns

[1594] Step 4:

[1595] The server predicts the next action based on the behavioral patterns it has learned.

[1596] Input: Learned behavioral patterns

[1597] Specific behavior: Using a generative AI model, the server predicts what the user will do next. For example, the server predicts that the user will arrive at work at 8am every morning.

[1598] Output: Predicted next action

[1599] Step 5:

[1600] The server sends a notification to the terminal based on the prediction result.

[1601] Input: Predicted next action

[1602] Specific operation: The server sends the prediction result to the device, and the device displays the user's next action suggestion and information on how to prevent mistakes. For example, the device displays a notification saying, "It's time to start getting ready for work."

[1603] Output: The notification displayed to the user

[1604] (Application example 1)

[1605] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1606] Conventional food delivery systems do not fully utilize the user's order history and behavioral patterns, making it difficult to suggest menu items or send reminders to users at the right time. In addition, they lack the ability to learn the user's preferences and behavioral patterns, so there is a need to improve the user experience.

[1607] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1608] In this invention, the server includes means for collecting data in the smartphone, means for analyzing the collected data, means for predicting the user's next action based on the analysis result, means for supporting the user's task processing based on the prediction result, means for collecting the user's order history, location information, and communication history, and learning the user's behavioral patterns, means for predicting the next order based on the learned behavioral patterns, means for providing recommended menus and promotion information at appropriate times, and means for sending reminders at specific times, thereby enabling personalized suggestions and reminders to be sent based on the user's preferences and behavioral patterns.

[1609] A "smartphone" is a multi-function mobile device that, in addition to the functions of a mobile phone, can also connect to the Internet and use applications.

[1610] "Means for collecting data" refers to functions and devices for acquiring information such as a user's operation history, location information, and communication history.

[1611] "Means for analyzing data" refers to functions or devices for analyzing collected data and understanding user behavior patterns and trends.

[1612] The "means for predicting the user's next action" refers to a function or device for predicting the action that the user is likely to take next based on the analysis results.

[1613] The "means for supporting the user's task processing" refers to a function or device for supporting the user so that the user can efficiently carry out a task based on the prediction result.

[1614] "Order history" is a record of orders a user has placed in the past.

[1615] "Location information" is information relating to the user's current location and past movement history.

[1616] "Communication history" is a record of calls and messages made by a user.

[1617] A "means for learning behavioral patterns" is a function or device for learning the behavioral tendencies and habits of a user using collected data.

[1618] The "means for predicting the next order" refers to a function or device for predicting the next product that the user is likely to order based on the learned behavioral patterns.

[1619] The "means for providing recommended menu items and promotion information" refers to a function or device for presenting appropriate menu items and discount information to the user based on the prediction results.

[1620] A "means for sending a reminder" is a function or device for sending a notification to a user at a specific time.

[1621] As an embodiment of the present invention, a system is provided in which a user's order history, location information, and communication history in a food delivery system are collected, and an AI analyzes the data to learn the user's behavioral patterns. A specific embodiment of this system is described below.

[1622] System Configuration

[1623] The system includes a smartphone, a server, and an AI model. The smartphone is hardware for collecting users' operation history, location information, and communication history. The server runs software for analyzing the collected data and learning users' behavioral patterns. The AI ​​model is used to predict the user's next action and provide recommended menus and promotional information at the appropriate time.

[1624] Collecting data

[1625] Smartphones collect users' operation history, location information, and communication history in real time. This includes GPS sensors, communication modules, and application log data. The collected data is sent to a server.

[1626] Analyzing data

[1627] The server receives the collected data and analyzes it using an AI model. Specifically, it uses programming languages ​​such as Python to pre-process the data, extract features, and train the model. The AI ​​model learns the user's behavioral patterns and predicts the next order.

[1628] Predicting and suggesting actions

[1629] Based on the learned behavioral patterns, the AI ​​model predicts what the user is likely to order next. Based on the prediction results, the server sends menu recommendations and promotion information to the smartphone. It also includes a function to send reminders at specific times.

[1630] Examples

[1631] For example, if a user has ordered "pizza," "sushi," "ramen," and "curry" in the past, the AI ​​model might recommend "pizza" next. It also sends a reminder at 12 o'clock saying, "It's lunchtime! Order your favorite menu!"

[1632] Examples of prompt statements

[1633] Here is an example of a prompt to be input to the generative AI model:

[1634] Use the user's past orders, location, and communication history to predict what their next menu recommendation will be, and add the ability to send them a reminder at a specific time.

[1635] In this way, personalized offers and reminders can be sent based on the user's preferences and behavioral patterns.

[1636] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1637] Step 1:

[1638] The smartphone collects the user's operation history, location information, and communication history. Specifically, it obtains location information using a GPS sensor, collects operation history from application log data, and records communication history through a communication module. This data is sent to a server in real time.

[1639] Input: User operation history, location information, communication history

[1640] Output: Sending collected data to a server

[1641] Step 2:

[1642] The server receives the data sent from the smartphone. The received data is stored in a database. The database organizes and stores data for each user.

[1643] Input: Data sent from smartphone

[1644] Output: Data stored in the database

[1645] Step 3:

[1646] The server preprocesses the data stored in the database, specifically by imputing missing values, removing outliers, normalizing the data, etc. This improves the quality of the data and makes it easier for subsequent analysis.

[1647] Input: Raw data stored in a database

[1648] Output: Preprocessed data

[1649] Step 4:

[1650] The server uses the preprocessed data to train the AI ​​model. Specifically, it uses a programming language such as Python to perform feature extraction, model selection, and model training. The AI ​​model learns the user's behavioral patterns.

[1651] Input: Preprocessed data

[1652] Output: A trained AI model

[1653] Step 5:

[1654] The server uses the trained AI model to predict the user's next action, specifically, guessing what menu item the user is likely to order next based on their past ordering history and behavioral patterns.

[1655] Input: Trained AI model, user's past order history

[1656] Output: Next order prediction result

[1657] Step 6:

[1658] Based on the prediction results, the server sends recommended menus and promotion information to the smartphone, specifically providing menus tailored to the user's preferences and currently available promotion information.

[1659] Input: Predicted result for next order

[1660] Output: Menu recommendations and promotional information sent to your smartphone

[1661] Step 7:

[1662] The server sends a reminder at a specific time. Specifically, the server sends a reminder to the smartphone at 12 o'clock saying "It's lunchtime! Order your favorite menu!" so that the user does not forget to order at lunchtime.

[1663] Input: current time, user behavior pattern

[1664] Output: Reminders sent to your smartphone

[1665] These steps enable personalized suggestions and reminders based on the user's preferences and behavioral patterns.

[1666] Example 2

[1667] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1668] Conventional smartphones and wearable devices can collect user behavior data, but there are only a limited number of systems that effectively use that data to support user behavior. In particular, there is a demand for systems that can learn user behavior patterns and notify users at the appropriate time to reduce user errors.

[1669] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting user behavior data, a means for transmitting the collected data to the server, a means for analyzing the data in the server and learning the user behavior pattern using an AI model, a means for generating notification content based on the learned behavior pattern, a means for transmitting the generated notification content to the terminal, and a means for displaying the notification on the terminal. This enables notification at an appropriate time based on the user's behavior pattern, and reduces user errors.

[1670] "User behavioral data" refers to information related to a user's behavior, such as the user's daily behavior, movement patterns, and application usage history.

[1671] "Server" refers to a computer system that receives user behavioral data, analyzes it, and trains the AI ​​model.

[1672] "AI model" refers to a model that uses machine learning algorithms to learn user behavior patterns and generate predictions and notifications.

[1673] A "behavioral pattern" refers to a tendency for a user to repeatedly perform an action at a particular time or in a particular situation.

[1674] "Notification content" refers to messages or alerts that are generated to prompt the user to take action.

[1675] "Terminal" refers to a device that is directly operated by a user, such as a smartphone or tablet.

[1676] "Notification" means a message or alert that is displayed to the User via a Device.

[1677] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1678] The present invention is a system that collects user behavior data and analyzes it using an AI model to provide notifications based on the user's behavior patterns. A specific embodiment of this system will be described below.

[1679] Collecting user behavior data

[1680] Users collect daily behavioral data using smartphones and wearable devices. For example, the GPS function of a smartphone is used to record the user's movement patterns. Data from calendar apps and reminder apps is also collected.

[1681] Sending behavioral data to the server

[1682] The device periodically transmits the collected behavioral data to a server, where the data is encrypted and transmitted using a secure communication protocol (e.g. HTTPS).

[1683] Data analysis on the server and training of AI models

[1684] The server analyzes the received behavioral data and uses an AI model to learn the user's behavioral patterns. Specifically, it uses Python and leverages machine learning libraries such as TensorFlow and PyTorch. For example, it learns that a user leaves for work at 8am every morning.

[1685] Generate notification content

[1686] The server generates notifications based on learned behavioral patterns. For example, it generates a notification "Start getting ready for work" when the user should start getting ready for work (7:50). Also, if the user tends to forget their phone, it generates a notification "Don't forget to take your phone with you."

[1687] Sending notifications to devices

[1688] The server sends the generated notification content to the terminal. The notification is sent in real time so that the user can receive it at the appropriate time.

[1689] Displaying notifications on your device

[1690] The device will display the received notifications to the user. Notifications will be displayed as pop-ups or push notifications so that the user can check them immediately.

[1691] Examples

[1692] As a concrete example, let's consider a case where the AI ​​has learned a user's behavioral pattern of leaving for work at 8:00 every morning. The AI ​​will notify the user at 7:50, "Please start preparing for your commute." Also, if the user has forgotten their phone many times in the past, the AI ​​will notify them, "Don't forget to take your phone with you."

[1693] Examples of prompt statements

[1694] An example of a prompt sentence might be:

[1695] Example prompt:

[1696] Design an AI system that learns user behavior patterns and notifies them at the right time. If a user leaves for work at 8am every morning, include a feature that notifies them to start getting ready for work at 7:50am and reminds users who tend to forget their phone to take it with them before leaving.

[1697] By inputting this prompt sentence into a generative AI model, a notification system design based on the user's behavioral patterns is generated.

[1698] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1699] Program processing flow

[1700] Step 1: Collect user behavior data

[1701] Users collect daily behavioral data using smartphones and wearable devices. Specifically, the GPS function of the smartphone is used to record the user's movement patterns. Data from calendar apps and reminder apps is also collected. The input is the user's behavioral data, and the output is the collected behavioral data.

[1702] Step 2: Sending behavioral data to the server

[1703] The device periodically transmits the collected behavioral data to the server. The data is encrypted and transmitted using a secure communication protocol (e.g., HTTPS). The input is the collected behavioral data, and the output is the behavioral data transmitted to the server.

[1704] Step 3: Data analysis on the server and training of the AI ​​model

[1705] The server analyzes the received behavioral data and uses an AI model to learn the user's behavioral patterns. Specifically, it uses Python to leverage machine learning libraries such as TensorFlow and PyTorch. The input is the behavioral data sent to the server, and the output is the learned behavioral pattern. For example, it learns a pattern that a user leaves for work at 8am every morning.

[1706] Step 4: Generate notification content

[1707] The server generates notification content based on the learned behavioral patterns. For example, it generates a notification "Start preparing for your commute" when the user should start preparing for their commute (7:50). Also, if the user tends to forget their phone, it generates a notification "Don't forget to take your phone with you." The input is the learned behavioral patterns, and the output is the generated notification content.

[1708] Step 5: Sending notifications to devices

[1709] The server sends the generated notification content to the terminal. The notification is sent in real time so that the user can receive it at the appropriate time. The input is the generated notification content, and the output is the notification content sent to the terminal.

[1710] Step 6: Display notifications on your device

[1711] The terminal displays the received notifications to the user. Notifications are displayed as popups or push notifications so that the user can see them immediately. The input is the notification content sent to the terminal, and the output is the notification displayed to the user.

[1712] Examples

[1713] As a concrete example, let's consider a case where the AI ​​has learned a user's behavioral pattern of leaving for work at 8:00 every morning. The AI ​​will notify the user at 7:50, "Please start preparing for your commute." Also, if the user has forgotten their phone many times in the past, the AI ​​will notify them, "Don't forget to take your phone with you."

[1714] (Application example 2)

[1715] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1716] Conventional user support systems using smart devices can learn the user's behavioral patterns and predict the user's next actions, but due to lack of coordination with the automobile, which is the user's means of transportation, they could not sufficiently improve the user's convenience. In addition, there was a lack of concrete measures to prevent users from forgetting things. This often caused inconvenience to users when commuting or traveling.

[1717] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1718] In this invention, the server includes a means for collecting data in the smart device, a means for analyzing the collected data, a means for predicting the user's next action based on the analysis result, a means for supporting the user's task processing based on the prediction result, a means for controlling the functions of the car based on the user's behavioral pattern, and a means for notifying the user of the user's forgotten items. This makes it possible to automatically control the functions of the car based on the user's behavioral pattern and prompt the user to make necessary preparations for commuting or traveling. In addition, by providing a notification function to prevent the user from forgetting things, the convenience of the user can be significantly improved.

[1719] A "smart device" is an electronic device that can be carried by a user and has functions such as data collection, analysis, and communication.

[1720] "Means of collecting data" refers to the function of obtaining information from various sensors and applications within smart devices.

[1721] "Means for analyzing data" refers to the function of processing collected data and analyzing user behavior patterns and trends.

[1722] "Means for predicting the user's next action" refers to a function for predicting the action that the user is likely to take next based on the analysis results.

[1723] "Means for supporting user task processing" refers to a function that supports the user to efficiently complete a task based on predicted behavior.

[1724] "Means for controlling the functions of an automobile" refers to a function for automatically operating functions such as starting the automobile engine and setting the air conditioning based on the user's behavioral patterns.

[1725] "Means for notifying a user of lost items" refers to a feature that notifies the user before departure to ensure that the user does not forget a particular item.

[1726] The system for carrying out this invention collects and analyzes data in smart devices, learns the user's behavioral patterns, predicts the user's next actions, and assists the user in processing tasks. It also has the function of controlling the functions of a car based on the user's behavioral patterns and notifying the user of lost items.

[1727] System Configuration

[1728] The system includes the following major hardware and software components:

[1729] Hardware: Smart devices (smartphones, smart glasses, head-mounted displays, etc.), infotainment systems for autonomous vehicles

[1730] Software: Python, smtplib (email sending library), data analysis libraries (e.g. Pandas, NumPy), machine learning libraries (e.g. scikit-learn)

[1731] Data collection and analysis

[1732] The server collects data from various sensors and applications in smart devices. The collected data is processed using a data analysis library to analyze user behavior patterns and trends. For example, a user's behavior pattern of commuting to work every morning at 8 o'clock is detected.

[1733] Behavioral prediction and task support

[1734] The server predicts the user's next action based on the analysis results. Based on the predicted action, the server supports the user to efficiently complete tasks. For example, when it is time to commute, the server notifies the user to prepare for commuting.

[1735] Automotive Function Control

[1736] The system automatically controls the functions of the car based on the user's behavioral patterns, for example starting the car's engine and setting the air conditioner to an appropriate temperature when it is time to commute to work.

[1737] Lost property notification

[1738] By registering items that a user often forgets in advance, the system will notify the user to bring these items before leaving. For example, if a user often forgets their phone and wallet, the system will notify the user to bring these items before leaving.

[1739] Examples

[1740] As a concrete example, consider a user who commutes to work every morning at 8am and often forgets their phone and wallet. In this case, the system will start the engine before 8am, set the air conditioner, and notify the user to prepare for work. It will also notify the user to not forget their phone and wallet.

[1741] Examples of prompt statements

[1742] Here is an example of a prompt to be input to the generative AI model:

[1743] Design an AI system that learns the user's behavioral pattern of commuting to work every morning at 8:00, and then starts the engine before 8:00, sets the air conditioner to an appropriate temperature, and notifies the user to prepare for the commute. Also, if the user often forgets his / her phone and wallet during the commute, add a feature that notifies the user to have these items with him / her before leaving.

[1744] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1745] Step 1:

[1746] The server collects data from various sensors and applications within the smart device.

[1747] Input: Sensor data from smart devices (location, time, application usage history, etc.)

[1748] Output: Raw data collected

[1749] Specific operation: The server communicates with the smart device, obtains location information from the GPS sensor, and collects application usage history.

[1750] Step 2:

[1751] The server processes the collected data using a data analysis library to analyze user behavior patterns.

[1752] Input: Raw data collected

[1753] Output: Analyzed behavioral patterns (e.g. commuting to work at 8am every morning)

[1754] Specific operation: The server uses data analysis libraries such as Pandas and NumPy to organize the collected data in chronological order and extract user behavior patterns.

[1755] Step 3:

[1756] The server predicts the user's next action based on the analysis results.

[1757] Input: Analyzed behavioral patterns

[1758] Output: Predicted next action (e.g., start commuting at 8am)

[1759] Specific operation: The server uses a machine learning model (e.g., scikit-learn) to predict the next action based on past behavior patterns.

[1760] Step 4:

[1761] The server assists the user in processing tasks based on the predicted behavior.

[1762] Input: Predicted next action

[1763] Output: Notification to the user (e.g., notification to prepare for commute)

[1764] Specific operation: The server sends a notification to the user's smart device and displays a message encouraging them to prepare for their commute.

[1765] Step 5:

[1766] The server controls the functions of the vehicle based on the user's behavioral patterns.

[1767] Input: Predicted next action

[1768] Output: Car control instructions (e.g. engine start, air conditioning settings)

[1769] What it does: The server communicates with the autonomous vehicle's infotainment system, starts the engine, and sets the air conditioning to the right temperature.

[1770] Step 6:

[1771] The server notifies the user of items that he or she often forgets.

[1772] Input: A list of lost items registered by the user (e.g. phone, wallet)

[1773] Output: Lost item notification content

[1774] Specific operation: The server sends a notification to the user's smart device and displays a message reminding the user to have their phone and wallet ready before leaving.

[1775] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1776] "Example 1"

[1777] In one embodiment of the present invention, the AI ​​in the smartphone collects data such as the user's operation history, location information, and communication history. Furthermore, the collected data is analyzed to learn the user's behavioral patterns. Then, based on the learned behavioral patterns, the AI ​​predicts the user's next action. Based on the prediction result, the AI ​​provides the user with suggestions for the next action and information on preventing mistakes. Furthermore, in one embodiment of the present invention, the AI ​​includes an emotion engine. The emotion engine recognizes emotions from the user's tone of voice and facial expressions. Based on the recognized emotions, the AI ​​more appropriately supports the user in processing tasks. For example, if the AI ​​recognizes that the user is feeling stressed, the AI ​​supports the user in reducing stress by playing relaxation music and providing a message encouraging deep breathing.

[1778] "Example 2"

[1779] In another embodiment of the present invention, the AI ​​in the smartphone learns a behavioral pattern that the user leaves for work at 8 o'clock every morning. Before 8 o'clock, the AI ​​suggests to the user to start preparing for work. Furthermore, the AI ​​learns information that the user often makes the mistake of forgetting his / her phone during the commute. Before leaving, the AI ​​notifies the user not to forget to take the phone. Furthermore, in one embodiment of the present invention, the AI ​​includes an emotion engine. The emotion engine recognizes emotions from the user's tone of voice and facial expressions. Based on the recognized emotions, the AI ​​better supports the user in processing tasks. For example, if the AI ​​recognizes that the user is feeling impatient or anxious, the AI ​​reduces the user's anxiety by speeding up the reminder notification time.

[1780] The process flow of each embodiment will be described below.

[1781] "Example 1"

[1782] Step 1: The AI ​​collects data such as the user's operation history, location information, communication history, etc. Step 2: The AI ​​analyzes the collected data and learns the user's behavioral patterns.

[1783] Step 3: Predict the user's next action based on learned behavioral patterns.

[1784] Step 4: Based on the prediction results, the AI ​​provides the user with suggestions for next steps and information on how to prevent mistakes.

[1785] Step 5: The emotion engine recognizes emotions from the user's tone of voice and facial expressions.

[1786] Step 6: Based on the recognized emotions, the AI ​​will better assist the user in completing tasks. For example, if the AI ​​recognizes that the user is feeling stressed, it will help the user reduce stress by playing relaxation music or providing a message encouraging deep breathing. "Example 2"

[1787] Step 1: The AI ​​learns the user's behavioral pattern: that they leave for work at 8am every morning.

[1788] Step 2: Just before 8am, the AI ​​suggests the user start getting ready for work.

[1789] Step 3: The AI ​​learns that users often make the mistake of forgetting their phone during their commute. Before leaving, the AI ​​will notify the user to remember to take their phone with them.

[1790] Step 4: The ...

Claims

1. A collection means for collecting data including a user's operation history, location information, and communication history; A means for transmitting the collected data to a server; analysis means for analyzing the data received by the server and learning the behavioral patterns of the user; A prediction means for predicting a next action based on the learned action pattern using a generative AI model; a support means for supporting the user in processing a task based on a result of the prediction; A system including:

2. The system according to claim 1 , wherein the analysis means has a function of learning the behavioral patterns of the user using a machine learning algorithm.

3. 2. The system according to claim 1, wherein the support means has a function of notifying a user of a predicted error.

Images