system

Abstract

We provide the system. [Solution] Means for acquiring movement information and visit information, Means of obtaining a user's payment history, A means of obtaining the user's search terms, Means for obtaining user posts, A means of integrating the acquired information and performing analysis based on user interests, A means of generating messages based on each user's interests, A means of presenting the generated message to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Many current information providing systems and services often provide the same information to users. Therefore, it is impossible to provide information that suits the interests, concerns, and emotional states of individual users, and it is difficult to sufficiently improve user satisfaction. In addition, a large amount of data needs to be integrated and analyzed for personalization of information, and if the processing method is not appropriate, the privacy of users may be violated. It is necessary to solve such problems.

Means for Solving the Problems

[0005] This invention provides a system for comprehensively acquiring and analyzing user movement information, visit information, payment history, search terms, and posting information on social networking services. Based on the acquired data, this system analyzes the user's interests and emotional state and generates personalized messages for each user. These messages are presented to the user via their device, potentially improving their quality of life. Specifically, movement and visit information are acquired using a location information system, and payment history is acquired via payment service APIs. Search terms are collected from browser APIs, and posting information is collected via social networking service APIs. By sending this information to a server and comprehensively analyzing it, a system is constructed that generates and presents appropriate messages to the user.

[0006] "Movement information" refers to information about the geographical location of a user when they physically move.

[0007] "Visit information" refers to information about the date, time, and location of a user's visit to a specific place or facility.

[0008] "Payment history" refers to a record of financial transactions a user has made in the past, including the date and time of the transaction, the place of purchase, and the items purchased.

[0009] "Search terms" refer to a record of keywords and phrases that users enter into internet search engines.

[0010] "Posted information" refers to the content of posts and comments made by users on social networking services.

[0011] A "location information system" is a system that uses technologies such as GPS to determine the geographical location of a user.

[0012] An "API" is an interface that allows different software programs to communicate with each other and utilize each other's functions.

[0013] A "server" is a computer that collects, analyzes, and stores data over a network, and provides services to other devices.

[0014] A "terminal" is a device that a user can directly operate and that acquires and displays information.

[0015] "Personalized" refers to a state that is customized to the individual user's interests, preferences, and circumstances.

[0016] A "message" is a piece of text or phrase that presents information to the user, generated based on the analyzed data.

[0017] A "user" is an individual who uses this system and receives information from it. [Brief explanation of the drawing]

[0018] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] Shows an emotion map to which a plurality of emotions are mapped. [Figure 10] Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be described.

[0021] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] This invention aims to enrich users' lives by providing personalized messages based on their daily behavior and interests. This system comprehensively analyzes users' movement information, payment history, search terms, and social networking service postings to generate individually optimized messages.

[0040] Collection of user information

[0041] Acquisition of travel and visit information

[0042] 1. The device periodically records the user's current location and visited locations using a GPS sensor.

[0043] 2. The device sends recorded information about visited locations and travel routes to the server at appropriate intervals, such as every hour.

[0044] Retrieving payment history

[0045] 1. After the user gives consent to retrieve payment history, the device calls a payment service API such as PayPay.

[0046] 2. The device retrieves the payment history and sends it to the server.

[0047] Retrieving search terms

[0048] 1. The device periodically collects search history via the browser API with the user's consent.

[0049] 2. The device sends the collected search terms to the server.

[0050] Retrieving writing information

[0051] 1. After the user agrees to use the SNS API, the device retrieves the user's posting information from major SNS platforms.

[0052] 2. The device sends the acquired SNS posting information to the server.

[0053] Data analysis and estimation by interest

[0054] 3. The server integrates all received data into the database.

[0055] 4. The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[0056] 5. The server updates the user profile based on the analysis results.

[0057] Generating personalized messages

[0058] 6. Use a generative model in which the server generates multiple message candidates based on the user profile.

[0059] 7. The server scores the relevance of each message candidate and selects the most appropriate message.

[0060] 8. The server prepares to deliver the selected messages to the user in a daily calendar format.

[0061] Presenting a message

[0062] 9. The device retrieves the message from the server at the scheduled time.

[0063] 10. The device notifies the user of the message. Possible notification methods include push notifications, home screen widgets, and dedicated app screens.

[0064] Specific example

[0065] For example, suppose a user visits a cafe on the weekend and purchases coffee using PayPay. Also, suppose that user has posted on social media that they "want to relax" and recently searched for "recommended movies." In this case,

[0066] 1. The device records the user's location information, including visits to cafes, and obtains coffee purchase information from the payment history.

[0067] 2. The device also collects SNS posts and search terms, and sends all the information to the server.

[0068] 3. The server integrates and analyzes this information to estimate that the user needs to relax and that they like coffee.

[0069] 4. The server generates a message saying, "Why not relax today, enjoy a cup of coffee, and watch a recommended movie?"

[0070] 5. The device displays this message to the user via push notification.

[0071] In this way, we can provide messages that are tailored to the user's lifestyle and interests. Through this entire system, we can increase user satisfaction and improve their quality of life.

[0072] The following describes the processing flow.

[0073] Step 1:

[0074] The device activates its GPS sensor and obtains the user's current location.

[0075] Step 2:

[0076] The device records location information it acquires and saves the places and times the user has visited.

[0077] Step 3:

[0078] The device sends recorded movement information to the server at regular intervals (e.g., every hour).

[0079] Step 4:

[0080] The user consents to the collection of their payment history.

[0081] Step 5:

[0082] The terminal calls the payment service's API to retrieve the user's payment history.

[0083] Step 6:

[0084] The device sends the acquired payment history to the server.

[0085] Step 7:

[0086] The device periodically collects the user's search history using the browser's API.

[0087] Step 8:

[0088] The device sends the collected search history to the server.

[0089] Step 9:

[0090] The user agrees to the use of the SNS API.

[0091] Step 10:

[0092] The device uses the SNS API to retrieve user posting information.

[0093] Step 11:

[0094] The terminal sends the written information it has acquired to the server.

[0095] Step 12:

[0096] The server integrates received movement information, payment history, search history, and writing information into a database.

[0097] Step 13:

[0098] The server applies natural language processing (NLP) and machine learning models to analyze the user's interests, concerns, and emotional state.

[0099] Step 14:

[0100] The server updates the user profile based on the analysis results.

[0101] Step 15:

[0102] The server generates multiple message candidates using a generative model based on the user profile.

[0103] Step 16:

[0104] The server scores the relevance of each message candidate and selects the most appropriate message.

[0105] Step 17:

[0106] The server prepares selected messages for the user in a daily calendar format.

[0107] Step 18:

[0108] The device retrieves the latest message from the server at a set time (e.g., 8 AM).

[0109] Step 19:

[0110] The device displays the message to the user via push notifications, home screen widgets, or a dedicated app screen.

[0111] (Example 1)

[0112] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0113] In modern society, providing personalized messages based on individual interests and behaviors is crucial for enriching users' lives. However, no system exists that effectively integrates diverse user information (such as travel data, payment history, search terms, and social networking service posts) to generate appropriate messages. Therefore, it is necessary to address the problems of information overload that users experience daily and the resulting lack of appropriate information.

[0114] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0115] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means for integrating the acquired information and performing analysis according to the user's interests, means for updating the user profile, means for applying natural language processing and machine learning algorithms, means for generating message candidates using a generative AI model, means for scoring the generated message candidates and selecting the optimal message, and notification means for presenting the selected message to the user. This enables the effective integration and analysis of diverse user information and the provision of individually optimized personalized messages.

[0116] "Movement information" refers to data about a user's location and places they have visited, and is primarily obtained using location information systems such as GPS sensors.

[0117] "Visit information" refers to detailed data about a user's visit to a specific location, including the date and time of the visit and the name of the location.

[0118] "Payment history" refers to data about financial transactions and purchase behaviors performed by a user, and includes information such as the date and time of payment, amount, and place of purchase, obtained through payment service APIs.

[0119] "Search terms" are keywords or phrases entered by users during internet searches, and are collected via browser APIs and other means.

[0120] "Posts" refer to content or comments that users submit to social networking services or other online platforms.

[0121] "Integration" is the process of centrally combining different types of data and converting them into a format that is easy to store in a database or similar system.

[0122] "Interest-based analysis" is a method that uses natural language processing and machine learning algorithms to analyze users' interests, concerns, and emotional states from collected data.

[0123] A "user profile" is a database description created based on a user's behavior, interests, and concerns, and includes individual characteristics and tendencies.

[0124] "Natural language processing" is a computer science technique for understanding and analyzing human language, and it includes tasks such as sentiment analysis of text and keyword extraction.

[0125] A "machine learning algorithm" is a computer algorithm that learns patterns and knowledge from data, and is used for clustering, classification, prediction, and other purposes.

[0126] A "generative AI model" is an artificial intelligence model that generates sentences and texts based on large amounts of data, and is a method for creating new messages and recommendations.

[0127] "Scoring" is the process of evaluating generated message candidates and quantifying their relevance and relevance.

[0128] The "optimal message" is the message that best matches the user's profile and analysis results, and provides value to them.

[0129] "Notifications" are methods for conveying information to users based on specific times or events, and include push notifications and widgets.

[0130] This invention is a system that provides personalized messages based on daily behaviors and interests, with the aim of enriching users' lives. This system collects user information, analyzes data and estimates interests, generates personalized messages, and presents those messages. Its detailed configuration is described below.

[0131] Collection of user information

[0132] 1. Obtaining travel and visit information:

[0133] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. This utilizes the GPS function of the smartphone.

[0134] The device transmits recorded information to the server via a communication module. Mobile communication networks or Wi-Fi are used as the means of communication.

[0135] 2. Obtaining payment history:

[0136] When a user consents to the retrieval of their payment history, the device calls the payment service API to retrieve the payment history. The retrieved information includes the payment date and time, amount, and store name.

[0137] The device temporarily saves the acquired payment history to local storage and then sends it to the server. For example, it may be updated periodically.

[0138] 3. Obtaining search terms:

[0139] The device periodically collects the user's search terms via the browser API. This process uses APIs such as Google Chrome's history API to extract search keywords every 24 hours.

[0140] The terminal sends the extracted search words to the server, and the transmitted data includes the search date and time and the search query.

[0141] 4. Obtaining writing information:

[0142] After a user consents to the use of the SNS API, the device retrieves posting information from social networking services. Specifically, this includes the content of the post, the date and time of posting, and the number of likes and comments.

[0143] The device sends the acquired SNS posting information to the server.

[0144] Data analysis and estimation by interest

[0145] The server integrates all received data into the database. Databases such as MySQL® or PostgreSQL are used.

[0146] The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state. Here, we use spaCy, a Python NLP library, to perform sentiment analysis on text, and scikit-learn to cluster the user's interests.

[0147] The server updates the user profile based on the analysis results. The user profile includes estimated user interests, frequently visited locations, and typical consumption patterns.

[0148] Generating personalized messages

[0149] The server uses a generative AI model to generate multiple message suggestions based on the updated user profile. Specifically, it uses the OpenAI® API's GPT-3® to input a prompt message such as, "The user bought coffee at a cafe, posted on social media that they want to relax, and are looking for recently recommended movies. Please generate relevant personalized messages."

[0150] The server scores the relevance of each candidate message and selects the optimal message. Methods such as TF-IDF and Doc2Vec are used for relevance scoring.

[0151] The server converts the selected messages into a daily calendar format and prepares to deliver them to the user.

[0152] Presenting a message

[0153] The device retrieves messages from the server at a set time. For example, you can set it to retrieve new messages every morning at 7:00 AM.

[0154] The device notifies the user of a message. Notification methods include smartphone push notifications, widgets placed on the home screen, or dedicated app screens. For example, a push notification might display a message such as, "Why not relax with a cup of coffee today and enjoy a recommended movie?"

[0155] This configuration enables the effective integration and analysis of diverse user information, allowing for the delivery of individually optimized, personalized messages. Through this entire system, user satisfaction can be enhanced, and their quality of life can be improved.

[0156] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0157] Step 1:

[0158] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. The input is location information from the GPS sensor, and the output is the recorded location data. For example, it might use the smartphone's GPS function to acquire location information every 5 minutes.

[0159] Step 2:

[0160] The device transmits recorded movement information to a server via a communication module. The input is the recorded location data, and the output is the data transmitted to the server. For example, it might perform an operation that transmits location information via Wi-Fi every hour.

[0161] Step 3:

[0162] After the user consents to the retrieval of their payment history, the device calls the payment service API to retrieve the payment history. The input is a request to the payment service API, and the output is payment history data. Specifically, it retrieves information such as the payment date and time, amount, and store name.

[0163] Step 4:

[0164] The device first saves the acquired payment history to local storage and then sends it to the server. The input is the payment history data, and the output is the data sent to the server. For example, it can perform an action to send data at a set time.

[0165] Step 5:

[0166] The device periodically collects user search terms via the browser API. The input is browser history data, and the output is a list of search terms. For example, the Google® Chrome history API is used to extract search keywords every 24 hours.

[0167] Step 6:

[0168] The terminal extracts search terms and sends them to the server. The input is a list of search terms, and the output is the data sent to the server. For example, it can perform an operation to periodically upload collected search terms to the server.

[0169] Step 7:

[0170] After the user consents to the use of the SNS API, the device retrieves posting information from the social networking service. The input is an SNS API request, and the output is the posting information. Specifically, it retrieves the post content, posting date and time, number of likes and comments, etc.

[0171] Step 8:

[0172] The device sends the acquired SNS posting information to the server. The input is the posting information, and the output is the data sent to the server. For example, the device might collect data every hour and send it to the server.

[0173] Step 9:

[0174] The server integrates all received data into a database. The input consists of various data sent to the server, and the output is the integrated database entry. MySQL or PostgreSQL can be used as the database.

[0175] Step 10:

[0176] The server applies natural language processing and machine learning algorithms to analyze the user's interests, concerns, and emotional state. The input is an integrated database entry, and the output is the analysis result. Specifically, it uses Python's spaCy to perform sentiment analysis on text and scikit-learn to cluster user interests.

[0177] Step 11:

[0178] The server updates the user profile based on the analysis results. The input is the analysis results, and the output is the updated user profile. The user profile includes estimated user interests, frequently visited locations, and typical consumption patterns.

[0179] Step 12:

[0180] The server uses a generative AI model to generate multiple message candidates based on an updated user profile. The input is the user profile, and the output is the message candidates. It uses the OpenAI API's GPT-3 to input prompts and generate messages.

[0181] Example prompt: "A user has bought coffee at a cafe, posted on social media that they want to relax, and is looking for recently recommended movies. Generate a relevant personalized message."

[0182] Step 13:

[0183] The server scores the relevance of each message candidate and selects the optimal message. The input is the message candidates, and the output is the selected optimal message. Methods such as TF-IDF and Doc2Vec are used for relevance scoring.

[0184] Step 14:

[0185] The server converts the selected messages into a daily calendar format and prepares them for delivery to the user. The input is the selected messages, and the output is the messages ready for delivery.

[0186] Step 15:

[0187] The device retrieves messages from the server at a set time. The input is the message ready for delivery on the server, and the output is the message retrieved by the device. For example, you can set it to retrieve a new message every morning at 7:00 AM.

[0188] Step 16:

[0189] The device notifies the user of a message. The input is a message acquired within the device, and the output is the notified message. Possible notification methods include smartphone push notifications, widgets placed on the home screen, or a dedicated app screen. As a specific example, a push notification might display a message such as, "Why not relax with a cup of coffee today and enjoy a recommended movie?"

[0190] (Application Example 1)

[0191] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0192] Traditional personalized messaging systems have limited ability to accurately analyze user behavior and interests and provide content that matches the user's lifestyle. Furthermore, there is a need for more sophisticated methods to analyze user interests and preferences and recommend the most suitable content based on that analysis.

[0193] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0194] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means including natural language processing and machine learning algorithms for integrating the acquired information and performing analysis according to the user's interests, and means for generating prompt sentences using a generative AI model that recommends content according to the user's interests and providing optimal content. This makes it possible to precisely analyze the user's behavior information and interests and provide optimal content as a personalized message.

[0195] "Movement information" refers to data about the user's current location and places visited, obtained using GPS or other location information systems.

[0196] "Visit information" refers to the history and detailed information of a user's visits to specific locations, and is data recorded by location information systems.

[0197] "Payment history" refers to a record of a user's purchasing activities, which is obtained through the API of an electronic payment service.

[0198] "Search terms" are keywords or phrases that users search for on internet search engines, and are collected via browser APIs.

[0199] "Posts" refer to information such as posts and comments made by users on social networking services, and are obtained via the APIs of each SNS.

[0200] "Integration" refers to the process of unifying information obtained from multiple different data sources and treating it as a single entity for analysis.

[0201] "Interest-based analysis" is the process of analyzing users' interests, preferences, and emotional states based on acquired data, and creating user profiles.

[0202] "Natural language processing" is a technology that enables computers to understand, analyze, and generate human language, and is used to understand text data such as user posts and search terms.

[0203] A "machine learning algorithm" is a computational method used to learn from large amounts of data and perform predictions and classifications, and is used to analyze user interests and preferences.

[0204] A "generative AI model" is a model that uses artificial intelligence technology to generate appropriate output (e.g., message or content recommendation) from input data.

[0205] A "prompt" is the input text given to a generative AI model and is used to control the model's output.

[0206] "Content" is a general term for information and entertainment provided to users, including movies, video clips, music, and ebooks.

[0207] This invention is a system that provides personalized messages based on users' daily behaviors and interests in order to enrich their lives. This system comprehensively analyzes users' movement information, payment history, search terms, and social networking service postings to generate and deliver individually optimized messages and content to the user.

[0208] Hardware and software usage:

[0209] Device: Smartphones have a built-in GPS sensor that records location information.

[0210] Software: Utilizes various APIs (payment service APIs, browser APIs, SNS APIs) for data collection.

[0211] Servers: High-performance computers and specialized software (such as TENSORFLOW® and OpenAI GPT-3) are used to process and analyze large amounts of data.

[0212] Data collection and analysis:

[0213] 1. Obtaining travel and visit information:

[0214] The device periodically records the user's current location and visited places using a GPS sensor and transmits this information to the server.

[0215] 2. Obtaining payment history:

[0216] After obtaining the user's consent, the device calls the payment service API to retrieve the user's payment history and sends it to the server.

[0217] 3. Obtaining search terms:

[0218] The system periodically collects users' search history via the browser API and sends it to the server.

[0219] 4. Obtaining writing information:

[0220] After obtaining permission to use the SNS API, the device retrieves user posting information from major SNS platforms and sends it to the server.

[0221] 5. Data Analysis:

[0222] The server integrates all received data into a database and applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[0223] Content generation and delivery:

[0224] 1. Update your user profile:

[0225] The server updates the user profile based on the analysis results.

[0226] 2. Generating personalized messages:

[0227] The server generates multiple message candidates using a generative AI model based on the user profile.

[0228] The relevance of each message candidate is scored, and the most suitable message is selected.

[0229] 3. Presenting the message:

[0230] The device retrieves a message from the server at a set time and notifies the user. Possible notification methods include push notifications, home screen widgets, and dedicated app screens.

[0231] Examples of specific cases and prompt statements:

[0232] For example, suppose a user visits a cafe on the weekend and buys a coffee. Also, consider a scenario where this user posts on a social networking service that they "want to relax" and recently searched for "recommended movies." In this case,

[0233] The server integrates and analyzes this information to estimate that "the user needs to relax" and "likes coffee."

[0234] Based on this, enter the following prompt into the generative AI model:

[0235] "User interest: Relaxation, Coffee, Movies. Generate a message."

[0236] The generated message will be a push notification to the user suggesting, "Why not relax with a cup of coffee today and enjoy a recommended movie?"

[0237] In this way, we can provide messages that are tailored to the user's lifestyle and interests, thereby increasing user satisfaction and improving their quality of life.

[0238] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0239] Step 1:

[0240] The device periodically records the user's current location and visited locations using a GPS sensor. Specifically, the device acquires location information at regular intervals and stores it as location data (latitude, longitude, and timestamp). The input is GPS sensor data, and the output is location data.

[0241] Step 2:

[0242] The device transmits recorded information about visited locations and travel routes to the server at appropriate intervals, such as every hour. The input is the location data acquired in step 1, and the output is the data transmitted to the server.

[0243] Step 3:

[0244] After the user gives consent to retrieve payment history, the device calls the payment service API to retrieve the payment history. Specifically, the device sends a request to the API endpoint and receives payment history data (purchase date and time, place of purchase, purchase amount) as a response. The input is the user's consent and the API request, and the output is the payment history data.

[0245] Step 4:

[0246] The terminal sends the acquired payment history data to the server. The input is the payment history data acquired in step 3, and the output is the data sent to the server.

[0247] Step 5:

[0248] After obtaining the user's consent to collect their search history, the device periodically collects the search history via the browser API. Specifically, the device sends a request to the browser API and stores the received search terms and search date and time. The input is the user's consent and the API request, and the output is the search history data.

[0249] Step 6:

[0250] The terminal sends the collected search history to the server. The input is the search history data obtained in step 5, and the output is the data sent to the server.

[0251] Step 7:

[0252] After the user gives their consent to use the SNS API, the device retrieves the user's posting information from major SNS platforms. Specifically, the device sends a request to the SNS API and receives the posting information (post content, posting date and time) as a response. The input is the user's consent and the API request, and the output is the posting information data.

[0253] Step 8:

[0254] The terminal sends the acquired write information to the server. The input is the write information data acquired in step 7, and the output is the data sent to the server.

[0255] Step 9:

[0256] The server integrates all received data (location information, payment history, search history, and writing information) into a database. Input is various data sent from the terminal, and output is the integrated database.

[0257] Step 10:

[0258] The server applies natural language processing (NLP) and machine learning algorithms to an integrated database to analyze users' interests, concerns, and emotional states. Specifically, it uses NLP to analyze text data and machine learning models to extract user behavior patterns. The input is data from the integrated database, and the output is the analysis results.

[0259] Step 11:

[0260] The server updates the user profile based on the analysis results. Specifically, it adds or modifies the user's interest categories and real-time sentiment indicators to the profile. The input is the analysis results, and the output is the updated user profile.

[0261] Step 12:

[0262] The server generates multiple message candidates using a generative AI model based on the user profile. The input is the updated user profile, and the output is the message candidates. As a concrete example, the AI ​​model is given the following prompt: "User interest: Relaxation, Coffee, Movies. Generate a message."

[0263] Step 13:

[0264] The server scores the relevance of each message candidate and selects the optimal message. The input is message candidates generated by the AI ​​model, and the output is the selected optimal message.

[0265] Step 14:

[0266] The device retrieves the most appropriate message from the server at a set time and notifies the user. Specific notification methods include push notifications, home screen widgets, and dedicated app screens. The input is the optimal message sent from the server, and the output is the notification to the user.

[0267] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0268] This invention is a system that provides personalized messages based on users' daily behaviors and interests in order to enrich their lives. This system collects and analyzes user movement information, payment history, search terms, and social networking service postings, and further recognizes the user's emotions using an emotion engine to generate individually optimized messages.

[0269] Collection of user information

[0270] Acquisition of travel and visit information

[0271] 1. The device periodically records the user's current location and visited locations using a GPS sensor.

[0272] 2. The device sends recorded information about visited locations and travel routes to the server at appropriate intervals, such as every hour.

[0273] Retrieving payment history

[0274] 1. After the user gives consent to retrieve payment history, the device calls the payment service API.

[0275] 2. The device retrieves the payment history and sends it to the server.

[0276] Retrieving search terms

[0277] 1. The device periodically collects search history via the browser API with the user's consent.

[0278] 2. The device sends the collected search terms to the server.

[0279] Retrieving writing information

[0280] 1. After the user agrees to use the SNS API, the device retrieves the user's posting information from major SNS platforms.

[0281] 2. The terminal sends the obtained SNS writing information to the server.

[0282] Data Analysis and Estimation for Each Interest

[0283] 1. The server integrates the received various data into the database.

[0284] 2. The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional states.

[0285] 3. The server updates the user profile based on the analysis results.

[0286] Utilization of the Emotion Engine

[0287] 1. The server uses the emotion engine to analyze the emotion from the user's writings and search words.

[0288] 2. The server uses the emotion engine to update the user's emotional state in real time.

[0289] 3. The server performs more accurate message generation considering the emotional state.

[0290] Generation of Personalized Messages

[0291] 1. The server uses a generation model to generate a plurality of message candidates based on the user profile and emotional state.

[0292] 2. The server scores the relevance of each message candidate and selects the optimal message.

[0293] 3. The server prepares to provide the selected message to the user in a flip-book format.

[0294] Presentation of Messages

[0295] 1. The device retrieves the latest message from the server at the scheduled time.

[0296] 2. The device displays messages to the user via push notifications, home screen widgets, and dedicated app screens.

[0297] Specific example

[0298] For example, suppose a user visits a cafe on the weekend and purchases coffee using a payment service. Furthermore, they post on social media saying they "want to relax" and recently searched for "recommended movies." In this case,

[0299] 1. The device records the user's location information, including visits to cafes, and obtains coffee purchase information from the payment history.

[0300] 2. The device also collects SNS posts and search terms, and sends all the information to the server.

[0301] 3. The server integrates and analyzes various pieces of information to estimate that the user needs to relax and that they like coffee.

[0302] 4. The server uses an emotion engine to more deeply analyze the user's desire to relax.

[0303] 5. The server generates a personalized message such as, "Why not relax today with a cup of coffee and enjoy a recommended movie?"

[0304] 6. The device will display this message to the user via push notification.

[0305] In this way, by providing messages that are tailored to the user's life circumstances and emotions, it is possible to increase user satisfaction and improve their quality of life. Through this system, users can more easily receive more personalized support.

[0306] The processing flow will be described below.

[0307] Step 1:

[0308] The terminal activates the GPS sensor and obtains the user's current location.

[0309] Step 2:

[0310] The terminal records the obtained location information and saves the places and times visited by the user.

[0311] Step 3:

[0312] The terminal transmits the movement information recorded at regular intervals (e.g., every hour) to the server.

[0313] Step 4:

[0314] The user agrees to obtain the payment history.

[0315] Step 5:

[0316] The terminal calls the API of the payment service to obtain the user's payment history.

[0317] Step 6:

[0318] The terminal transmits the obtained payment history to the server.

[0319] Step 7:

[0320] The terminal uses the API of the browser to periodically collect the user's search history.

[0321] Step 8:

[0322] The terminal transmits the collected search history to the server.

[0323] Step 9:

[0324] The user agrees to the use of the SNS API.

[0325] Step 10:

[0326] The device uses the SNS API to retrieve user posting information.

[0327] Step 11:

[0328] The terminal sends the written information it has acquired to the server.

[0329] Step 12:

[0330] The server integrates received movement information, payment history, search history, and writing information into a database.

[0331] Step 13:

[0332] The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[0333] Step 14:

[0334] The server updates the user profile based on the results of the information analysis.

[0335] Step 15:

[0336] The server uses an emotion engine to analyze the sentiment of user posts and search terms.

[0337] Step 16:

[0338] The server uses an emotion engine to update the user's emotional state in real time.

[0339] Step 17:

[0340] The server uses a generative model that generates multiple message candidates based on emotional state and user profile.

[0341] Step 18:

[0342] The server scores the relevance of each message candidate and selects the most appropriate message.

[0343] Step 19:

[0344] The server prepares to deliver selected messages to the user in a daily calendar format.

[0345] Step 20:

[0346] The device retrieves the latest message from the server at a set time (e.g., 8 AM).

[0347] Step 21:

[0348] The device displays messages to the user via push notifications, home screen widgets, and dedicated app screens.

[0349] (Example 2)

[0350] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0351] In recent years, with the advancement of information technology, personalized systems that provide optimal information to individual users have attracted attention. However, conventional systems have struggled to adequately analyze user behavior and emotions and deliver the most appropriate message at the right time. Furthermore, they lacked mechanisms for integrating data from multiple sources and reflecting users' emotional states in real time. Therefore, effectively delivering personalized messages to users has been a challenge.

[0352] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0353] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means for integrating the acquired information and analyzing the user's interests and emotional state using natural language processing and machine learning algorithms, means including a generative AI model for generating messages based on the user's interests and emotional state, means for scoring the generated messages and selecting the optimal message, and means for presenting the selected messages to the user in a daily calendar format. This makes it possible to enrich the user's life and provide optimal personalized messages based on their emotions and behavior.

[0354] "Movement information" refers to information about where a user has moved. Specifically, it refers to data on latitude, longitude, and travel route obtained by GPS sensors.

[0355] "Visit information" refers to information about a user's visit to a specific location. This includes the name, location, and date and time of the visit.

[0356] "Payment history" refers to a record of purchases and payments made by a user. It includes information such as the date and time of purchase, store name, and purchased items, which are obtained through payment service APIs.

[0357] "Search terms" refer to the search queries that users enter into internet search engines. This data is used to analyze users' interests and concerns.

[0358] "Posting" refers to posts and comments made by users on social networking services or other platforms.

[0359] "Natural language processing" is the technology that enables computers to understand, interpret, and generate human language. Specifically, it includes text analysis, sentiment analysis, and keyword extraction.

[0360] A "machine learning algorithm" is a technique of artificial intelligence that automatically learns regularities and patterns from data. This makes it possible to estimate a user's interests and emotions.

[0361] An "emotion engine" refers to specialized software or algorithms used to analyze emotions from text data. It plays a role in inferring emotional states from user posts and search terms.

[0362] A "generative AI model" is an artificial intelligence model that utilizes deep learning-based text generation technology to generate messages based on the user's profile and emotional state.

[0363] "Scoring" is the process of assigning scores to generated message candidates based on factors such as relevance and appropriateness.

[0364] An "optimal message" refers to a personalized message that is best suited to the user's current situation and emotional state.

[0365] The "daily calendar format" is a format that presents messages sequentially day by day, providing users with an experience of receiving new information on a regular basis.

[0366] A "location information system" is a system that uses GPS and other location-determining technologies to determine the user's current location.

[0367] A "social networking service" is a platform on the internet for users to create, share, and comment on content.

[0368] This invention is a system that provides personalized messages based on users' daily behaviors and interests in order to enrich their lives. This system collects and analyzes user movement information, payment history, search terms, and social networking service postings, and further recognizes the user's emotions using an emotion engine to generate individually optimized messages.

[0369] Collection of user information

[0370] Acquisition of travel and visit information

[0371] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. For example, it collects latitude and longitude information every 10 minutes.

[0372] The device sends recorded information about visited locations and travel routes to the server in a batch process every hour.

[0373] Retrieving payment history

[0374] The user gives consent to the collection of payment history on their device. For example, they might check a checkbox in the app's settings screen that says, "I agree to the collection of payment history."

[0375] After the device has given consent, it calls a payment service API (e.g., PayPal or Stripe) to retrieve the user's latest payment history.

[0376] The device sends the acquired payment history data to the server.

[0377] Retrieving search terms

[0378] The device, with the user's consent, uses a browser API (for example, a Chrome extension) to collect the user's search history at regular intervals (for example, daily).

[0379] The device sends the collected search terms to the server.

[0380] Retrieving writing information

[0381] The user agrees to the use of APIs from major social networking services (e.g., Facebook and Twitter).

[0382] After the device gives its consent, it calls the SNS API to retrieve user posting information. Specifically, it collects recent posts and comments.

[0383] The terminal sends the acquired write information to the server.

[0384] Data analysis and estimation by interest

[0385] The server stores the various data it receives in a centralized database (for example, MySQL or MongoDB).

[0386] The server applies a natural language processing (NLP) engine (e.g., spaCy or NLTK) to the integrated data to analyze user search terms and social media posts.

[0387] Next, the server uses statistical and machine learning models (e.g., scikit-learn or TensorFlow) to estimate the user's interests and preferences. For example, it predicts what actions the user will take in the future based on keywords such as "relax" or "movies."

[0388] Based on the analysis results, the server updates the user profile to include the user's interests, preferences, and emotional state.

[0389] Utilizing the Emotion Engine

[0390] The server uses an emotion engine (for example, IBM Watson® sentiment analysis) to analyze user sentiment from their posts and search terms. For example, it might infer the sentiment "I want to relax" from a post that says "I want to relax."

[0391] The server reflects the analyzed emotion data in the user profile in real time.

[0392] The server takes emotional states into consideration and generates personalized messages based on detailed analysis.

[0393] Generating personalized messages

[0394] The server uses a generative AI model (e.g., OpenAI GPT-3) to generate multiple message suggestions based on the user profile and emotional state. For example, it might generate a message such as, "Why not relax with a cup of coffee and enjoy a recommended movie today?"

[0395] Next, the server assigns relevance scores to the multiple message candidates that have been generated (for example, scores based on the user's current behavior and sentiment) and selects the most suitable message.

[0396] Presenting a message

[0397] The server stores the selected messages in a daily log format and prepares for the next presentation.

[0398] The device retrieves the latest personalized message from the server at a set time (for example, 8 AM).

[0399] The device displays messages it has received to the user via push notifications, home screen widgets, and a dedicated app screen. For example, it might display a message like, "Why not relax with a cup of coffee and enjoy a recommended movie today?" in the smartphone's notification bar.

[0400] Specific example

[0401] For example, a user visited a cafe on the weekend and purchased coffee using a payment service. Furthermore, they posted on social media that they wanted to "relax" and recently searched for "recommended movies." In this case,

[0402] 1. The device records the user's location information, including visits to cafes, and obtains coffee purchase information from the payment history.

[0403] 2. The device also collects SNS posts and search terms, and sends all the information to the server.

[0404] 3. The server integrates and analyzes various pieces of information to estimate that the user needs to relax and that they like coffee.

[0405] 4. The server uses an emotion engine to more deeply analyze the user's desire to relax.

[0406] 5. The server generates a personalized message such as, "Why not relax today with a cup of coffee and enjoy a recommended movie?"

[0407] 6. The device will display this message to the user via push notification.

[0408] In this way, by providing messages that are tailored to the user's life circumstances and emotions, it is possible to increase user satisfaction and improve their quality of life. Through this system, users can more easily receive more personalized support.

[0409] Example of a prompt

[0410] For example, a prompt statement to be input to a generative AI model can be written as follows:

[0411] "A user visited a cafe on the weekend and purchased coffee using a payment service. Furthermore, they posted on social media that they wanted to 'relax,' and recently searched for 'recommended movies.' Please generate a personalized message for this user."

[0412] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0413] Step 1:

[0414] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. For example, it acquires latitude and longitude information every 10 minutes and stores it in local memory.

[0415] Input: Program start command, GPS sensor data

[0416] Output: User's current location and visited locations information (latitude and longitude data)

[0417] Step 2:

[0418] The device sends recorded information about visited locations and travel routes to the server in a batch process every hour. Specifically, it sends the data to a message queue via an API.

[0419] Input: Recorded latitude and longitude data

[0420] Output: Data on visited locations and travel routes sent to the server.

[0421] Step 3:

[0422] The user gives consent to the collection of payment history on their device. For example, they might check a checkbox in the app's settings screen that says, "I agree to the collection of payment history."

[0423] Input: User consent action

[0424] Output: Consent information flags

[0425] Step 4:

[0426] After the device has given consent, it calls a payment service API to retrieve the user's latest payment history. For example, it might use the PayPal or Stripe API to retrieve transaction data.

[0427] Input: Consent information flag

[0428] Output: Payment history data

[0429] Step 5:

[0430] The device sends the acquired payment history data to the server. Specifically, it sends the data to the server via an HTTP request.

[0431] Input: Payment history data

[0432] Output: Payment history data sent to the server

[0433] Step 6:

[0434] With the user's consent, the device uses the browser API to collect the user's search history at regular intervals (for example, daily). Specifically, it uses a Chrome extension to collect search queries.

[0435] Input: User consent information, browser search data

[0436] Output: Collected search terms

[0437] Step 7:

[0438] The device sends the collected search terms to the server. For example, it might send them in JSON format via an HTTP request.

[0439] Input: Collected search terms

[0440] Output: Search words sent to the server

[0441] Step 8:

[0442] The user agrees to the use of APIs from major social networking services (SNS). Specifically, they check a checkbox in the app's settings screen that says something like, "I agree to the use of SNS APIs."

[0443] Input: User consent action

[0444] Output: Consent information flags

[0445] Step 9:

[0446] After the device gives its consent, it calls the SNS API to retrieve user posting information. For example, it uses the Facebook or Twitter API to retrieve recent posts and comments.

[0447] Input: Consent information flag

[0448] Output: SNS posting information

[0449] Step 10:

[0450] The device sends the acquired SNS posting information to the server. Specifically, it sends the data to the server via an HTTP request.

[0451] Input: Social media posting information

[0452] Output: SNS posting information sent to the server

[0453] Step 11:

[0454] The server stores various data it receives in a centralized database. For example, MySQL or MongoDB can be used to store the data.

[0455] Input: Visit location data, payment history data, search terms, social media postings

[0456] Output: Various data stored in the database

[0457] Step 12:

[0458] The server applies a natural language processing (NLP) engine to the integrated data to analyze user information. For example, it might use spaCy or NLTK to analyze text data and extract keywords and sentiments.

[0459] Input: Integrated data stored in the database

[0460] Output: Analyzed user interests and emotional states

[0461] Step 13:

[0462] The server uses statistical and machine learning models to estimate user interests and preferences. For example, it can use scikit-learn or TensorFlow to train data and generate an interest prediction model.

[0463] Input: Analyzed user interests and emotional states

[0464] Output: Estimated user interests and emotional states

[0465] Step 14:

[0466] The server uses a generative AI model to generate multiple message candidates based on the user profile and emotional state. For example, it can use OpenAI GPT-3 to generate personalized messages for the user.

[0467] Input: Estimated user interests and emotional state

[0468] Output: List of message candidates

[0469] Step 15:

[0470] The server assigns a relevance score to the generated message candidates and selects the most suitable message. Specifically, it uses a scoring algorithm to evaluate the relevance of each message.

[0471] Input: List of message suggestions

[0472] Output: Optimal message

[0473] Step 16:

[0474] The server stores the selected messages in a daily log format and prepares for the next presentation.

[0475] Input: Optimal message

[0476] Output: Messages stored in a daily calendar format

[0477] Step 17:

[0478] The device retrieves the latest personalized message from the server at a set time (for example, 8 AM). Specifically, it periodically sends requests to the server via an API.

[0479] Input: Time setting

[0480] Output: Latest message retrieved

[0481] Step 18:

[0482] The device displays messages it has received to the user via push notifications, home screen widgets, and dedicated app screens. For example, it displays messages in the smartphone's notification bar.

[0483] Input: Latest message retrieved

[0484] Output: Message displayed to the user

[0485] (Application Example 2)

[0486] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0487] In modern factories, improving operator efficiency is a critical challenge. However, because individualized support is required, taking into account each operator's work history and current emotional state, general assistance and advice have limited effectiveness. Furthermore, operators themselves may experience stress or a lack of job satisfaction, which can lead to decreased productivity. Therefore, a system is needed that can grasp each operator's individual situation and emotions in real time and provide personalized messages based on that information.

[0488] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0489] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means for integrating the acquired information and performing analysis according to the user's interests, means for analyzing the user's emotions using an emotion engine, means for generating personalized messages based on the analysis results and emotional state, and means for presenting the generated personalized messages to the user. This makes it possible for robots deployed in a factory to generate and present personalized messages based on the operator's past work history, the situation during work, and information acquired from social networking services.

[0490] "Movement information" refers to the movement history and location data of users and operators, including where they have moved to.

[0491] "Visit information" refers to records of when users or operators visited a specific location and data on the time they spent at that location.

[0492] "Payment history" refers to records of transactions made by users or operators, as well as data on what goods or services were paid for.

[0493] "Search terms" refer to keywords or phrases that users or operators enter into internet search engines.

[0494] "Posts" refer to text or comments that users or operators post on social networking services or other platforms.

[0495] An "emotion engine" refers to an algorithm or system that analyzes emotions and emotional states from user or operator text data.

[0496] "Personalized messages" refer to messages and notifications that are customized based on the individual data of the user or operator.

[0497] A "robot" refers to a machine or device used in factories or other work environments to perform automated tasks.

[0498] An "operator" refers to a human worker who is responsible for operating machinery or systems in a factory or other work environment.

[0499] This invention provides a system that delivers personalized messages based on individual circumstances and emotional states to improve the operational efficiency of operators in a factory. This system collects and analyzes user movement information, payment history, search terms, and writing information, and uses an emotion engine to recognize emotional states, thereby generating and presenting individually optimized messages.

[0500] First, the system program uses GPS devices to acquire operator movement and visit information. For example, it utilizes a location information system such as the Garmin GLO 2. Next, to obtain payment history, the PaymentService API is used to collect payment information for goods and services made by operators within the factory. In addition, the Browser History API is used to obtain keywords searched by operators on the internet. Furthermore, APIs of major social networking services (SNS) are used to collect operator posting information. All of this data is sent to the server.

[0501] The server first integrates the collected data and uses natural language processing (NLP) and machine learning algorithms to analyze the operator's interests, concerns, and emotional state. Next, it uses an emotion engine (e.g., Google Cloud Natural Language API) to analyze the operator's emotional state in detail. Based on this analysis and the emotional state, a generative AI model (e.g., GPT-3 model) is used to generate personalized messages. These messages are created considering the operator's past work history, current work situation, and social media posts.

[0502] The generated personalized messages are presented to operators through robots placed in the factory. Presentation methods include the robot's display, voice output, or a dedicated application. For example, if an operator posts on social media that their work efficiency is low and is searching for "tips for improving productivity," the AI ​​model might generate a message such as "Take more breaks today and relax," and the robot would then present that message.

[0503] (Specific example)

[0504] Operator A posted on social media that "work efficiency is declining." They also searched for "tips for improving productivity" in their browser.

[0505] Movement information: The GPS device tracked Operator A's movements within the factory.

[0506] Payment history: Purchased an energy drink from a vending machine inside the factory.

[0507] Search terms: "Tips for improving productivity"

[0508] Social media post: "Work efficiency is declining."

[0509] (Example of a prompt message)

[0510] "Operator A has posted on social media about a decline in work efficiency. Their search history also shows they are looking for information on improving productivity. Based on this information, we will provide refreshing advice and personalized messages."

[0511] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0512] Step 1:

[0513] The terminal periodically records the operator's movement and visit information using a GPS device. Specifically, it acquires location data from a GPS device (e.g., Garmin GLO 2) and stores this data in its internal memory. In this step, the input is the location information from the GPS device, and the output is the recorded movement information.

[0514] Step 2:

[0515] The terminal periodically retrieves payment history, so it calls the PaymentService API to collect payment data. It retrieves transaction information from vending machines and shops. The inputs in this step are the user ID and payment event, and the output is payment history data.

[0516] Step 3:

[0517] The device periodically collects the user's search terms using the browser history API. This is a list of keywords and phrases the user has recently searched for. The input for this step is browser history data, and the output is a list of search terms.

[0518] Step 4:

[0519] The device uses the SNS API to retrieve user posting information. It periodically collects SNS posts and sends them to the server. The input in this step is the posted data obtained via the SNS API, and the output is the posting information.

[0520] Step 5:

[0521] The terminal sends all collected information (movement information, payment history, search terms, and posting information) to the server. The server integrates this information and stores it in a database. The input in this step is the user data sent from the terminal, and the output is the integrated dataset.

[0522] Step 6:

[0523] The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state. This process identifies the user's current state and trends based on the collected data. The input in this step is an integrated dataset, and the output is the analyzed user profile.

[0524] Step 7:

[0525] The server uses an emotion engine (e.g., Google Cloud Natural Language API) to analyze the user's emotional state. It analyzes collected writing information and search terms to identify the user's emotions. The input in this step is the user's text data, and the output is emotional state data.

[0526] Step 8:

[0527] The server uses a generated AI model (e.g., a GPT-3 model) to generate personalized messages based on the analysis results and emotional state. It generates the optimal message tailored to the user's profile. The input in this step is the analyzed user profile and emotional state data, and the output is the personalized message.

[0528] Step 9:

[0529] The terminal retrieves a personalized message generated from the server and presents it to the operator via the display or voice output of a robot placed in the factory. In this step, the input is the personalized message, and the output is a notification to the operator.

[0530] Step 10:

[0531] The user acts based on the personalized message presented to them, aiming to improve efficiency and reduce stress. In this step, the input is the personalized message, and the output is the user's action.

[0532] By following these steps, it is possible to understand the individual circumstances and emotions of users in real time and provide optimal messages based on that understanding, thereby improving operational efficiency in the factory.

[0533] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0534] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0535] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0536] [Second Embodiment]

[0537] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0538] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0539] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0540] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0541] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0542] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0543] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0544] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0545] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0546] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0547] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0548] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0549] This invention aims to enrich users' lives by providing personalized messages based on their daily behavior and interests. This system comprehensively analyzes users' movement information, payment history, search terms, and social networking service postings to generate individually optimized messages.

[0550] Collection of user information

[0551] Acquisition of travel and visit information

[0552] 1. The device periodically records the user's current location and visited locations using a GPS sensor.

[0553] 2. The device sends recorded information about visited locations and travel routes to the server at appropriate intervals, such as every hour.

[0554] Retrieving payment history

[0555] 1. After the user gives consent to retrieve payment history, the device calls a payment service API such as PayPay.

[0556] 2. The device retrieves the payment history and sends it to the server.

[0557] Retrieving search terms

[0558] 1. The device periodically collects search history via the browser API with the user's consent.

[0559] 2. The device sends the collected search terms to the server.

[0560] Retrieving writing information

[0561] 1. After the user agrees to use the SNS API, the device retrieves the user's posting information from major SNS platforms.

[0562] 2. The device sends the acquired SNS posting information to the server.

[0563] Data analysis and estimation by interest

[0564] 3. The server integrates all received data into the database.

[0565] 4. The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[0566] 5. The server updates the user profile based on the analysis results.

[0567] Generating personalized messages

[0568] 6. Use a generative model in which the server generates multiple message candidates based on the user profile.

[0569] 7. The server scores the relevance of each message candidate and selects the most appropriate message.

[0570] 8. The server prepares to deliver the selected messages to the user in a daily calendar format.

[0571] Presenting a message

[0572] 9. The device retrieves the message from the server at the scheduled time.

[0573] 10. The device notifies the user of the message. Possible notification methods include push notifications, home screen widgets, and dedicated app screens.

[0574] Specific example

[0575] For example, suppose a user visits a cafe on the weekend and purchases coffee using PayPay. Also, suppose that user has posted on social media that they "want to relax" and recently searched for "recommended movies." In this case,

[0576] 1. The device records the user's location information, including visits to cafes, and obtains coffee purchase information from the payment history.

[0577] 2. The device also collects SNS posts and search terms, and sends all the information to the server.

[0578] 3. The server integrates and analyzes this information to estimate that the user needs to relax and that they like coffee.

[0579] 4. The server generates a message saying, "Why not relax today, enjoy a cup of coffee, and watch a recommended movie?"

[0580] 5. The device displays this message to the user via push notification.

[0581] In this way, we can provide messages that are tailored to the user's lifestyle and interests. Through this entire system, we can increase user satisfaction and improve their quality of life.

[0582] The following describes the processing flow.

[0583] Step 1:

[0584] The device activates its GPS sensor and obtains the user's current location.

[0585] Step 2:

[0586] The device records location information it acquires and saves the places and times the user has visited.

[0587] Step 3:

[0588] The device sends recorded movement information to the server at regular intervals (e.g., every hour).

[0589] Step 4:

[0590] The user consents to the collection of their payment history.

[0591] Step 5:

[0592] The terminal calls the payment service's API to retrieve the user's payment history.

[0593] Step 6:

[0594] The device sends the acquired payment history to the server.

[0595] Step 7:

[0596] The device periodically collects the user's search history using the browser's API.

[0597] Step 8:

[0598] The device sends the collected search history to the server.

[0599] Step 9:

[0600] The user agrees to the use of the SNS API.

[0601] Step 10:

[0602] The device uses the SNS API to retrieve user posting information.

[0603] Step 11:

[0604] The terminal sends the written information it has acquired to the server.

[0605] Step 12:

[0606] The server integrates received movement information, payment history, search history, and writing information into a database.

[0607] Step 13:

[0608] The server applies natural language processing (NLP) and machine learning models to analyze the user's interests, concerns, and emotional state.

[0609] Step 14:

[0610] The server updates the user profile based on the analysis results.

[0611] Step 15:

[0612] The server generates multiple message candidates using a generative model based on the user profile.

[0613] Step 16:

[0614] The server scores the relevance of each message candidate and selects the most appropriate message.

[0615] Step 17:

[0616] The server prepares selected messages for the user in a daily calendar format.

[0617] Step 18:

[0618] The device retrieves the latest message from the server at a set time (e.g., 8 AM).

[0619] Step 19:

[0620] The device displays the message to the user via push notifications, home screen widgets, or a dedicated app screen.

[0621] (Example 1)

[0622] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0623] In modern society, providing personalized messages based on individual interests and behaviors is crucial for enriching users' lives. However, no system exists that effectively integrates diverse user information (such as travel data, payment history, search terms, and social networking service posts) to generate appropriate messages. Therefore, it is necessary to address the problems of information overload that users experience daily and the resulting lack of appropriate information.

[0624] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0625] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means for integrating the acquired information and performing analysis according to the user's interests, means for updating the user profile, means for applying natural language processing and machine learning algorithms, means for generating message candidates using a generative AI model, means for scoring the generated message candidates and selecting the optimal message, and notification means for presenting the selected message to the user. This enables the effective integration and analysis of diverse user information and the provision of individually optimized personalized messages.

[0626] "Movement information" refers to data about a user's location and places they have visited, and is primarily obtained using location information systems such as GPS sensors.

[0627] "Visit information" refers to detailed data about a user's visit to a specific location, including the date and time of the visit and the name of the location.

[0628] "Payment history" refers to data about financial transactions and purchase behaviors performed by a user, and includes information such as the date and time of payment, amount, and place of purchase, obtained through payment service APIs.

[0629] "Search terms" are keywords or phrases entered by users during internet searches, and are collected via browser APIs and other means.

[0630] "Posts" refer to content or comments that users submit to social networking services or other online platforms.

[0631] "Integration" is the process of centrally combining different types of data and converting them into a format that is easy to store in a database or similar system.

[0632] "Interest-based analysis" is a method that uses natural language processing and machine learning algorithms to analyze users' interests, concerns, and emotional states from collected data.

[0633] A "user profile" is a database description created based on a user's behavior, interests, and concerns, and includes individual characteristics and tendencies.

[0634] "Natural language processing" is a computer science technique for understanding and analyzing human language, and it includes tasks such as sentiment analysis of text and keyword extraction.

[0635] A "machine learning algorithm" is a computer algorithm that learns patterns and knowledge from data, and is used for clustering, classification, prediction, and other purposes.

[0636] A "generative AI model" is an artificial intelligence model that generates sentences and texts based on large amounts of data, and is a method for creating new messages and recommendations.

[0637] "Scoring" is the process of evaluating generated message candidates and quantifying their relevance and relevance.

[0638] The "optimal message" is the message that best matches the user's profile and analysis results, and provides value to them.

[0639] "Notifications" are methods for conveying information to users based on specific times or events, and include push notifications and widgets.

[0640] This invention is a system that provides personalized messages based on daily behaviors and interests, with the aim of enriching users' lives. This system collects user information, analyzes data and estimates interests, generates personalized messages, and presents those messages. Its detailed configuration is described below.

[0641] Collection of user information

[0642] 1. Obtaining travel and visit information:

[0643] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. This utilizes the GPS function of the smartphone.

[0644] The device transmits recorded information to the server via a communication module. Mobile communication networks or Wi-Fi are used as the means of communication.

[0645] 2. Obtaining payment history:

[0646] When a user consents to the retrieval of their payment history, the device calls the payment service API to retrieve the payment history. The retrieved information includes the payment date and time, amount, and store name.

[0647] The device temporarily saves the acquired payment history to local storage and then sends it to the server. For example, it may be updated periodically.

[0648] 3. Obtaining search terms:

[0649] The device periodically collects the user's search terms via the browser API. This process uses APIs such as Google Chrome's history API to extract search keywords every 24 hours.

[0650] The terminal sends the extracted search words to the server, and the transmitted data includes the search date and time and the search query.

[0651] 4. Obtaining writing information:

[0652] After a user consents to the use of the SNS API, the device retrieves posting information from social networking services. Specifically, this includes the content of the post, the date and time of posting, and the number of likes and comments.

[0653] The device sends the acquired SNS posting information to the server.

[0654] Data analysis and estimation by interest

[0655] The server integrates all received data into a database. Databases such as MySQL or PostgreSQL are used.

[0656] The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state. Here, we use spaCy, a Python NLP library, to perform sentiment analysis on text, and scikit-learn to cluster the user's interests.

[0657] The server updates the user profile based on the analysis results. The user profile includes estimated user interests, frequently visited locations, and typical consumption patterns.

[0658] Generating personalized messages

[0659] The server uses a generative AI model to generate multiple message suggestions based on an updated user profile. Specifically, it uses the OpenAI API's GPT-3 to input a prompt message such as, "The user bought coffee at a cafe, posted on social media that they want to relax, and are looking for recently recommended movies. Please generate relevant personalized messages."

[0660] The server scores the relevance of each candidate message and selects the optimal message. Methods such as TF-IDF and Doc2Vec are used for relevance scoring.

[0661] The server converts the selected messages into a daily calendar format and prepares to deliver them to the user.

[0662] Presenting a message

[0663] The device retrieves messages from the server at a set time. For example, you can set it to retrieve new messages every morning at 7:00 AM.

[0664] The device notifies the user of a message. Notification methods include smartphone push notifications, widgets placed on the home screen, or dedicated app screens. For example, a push notification might display a message such as, "Why not relax with a cup of coffee today and enjoy a recommended movie?"

[0665] This configuration enables the effective integration and analysis of diverse user information, allowing for the delivery of individually optimized, personalized messages. Through this entire system, user satisfaction can be enhanced, and their quality of life can be improved.

[0666] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0667] Step 1:

[0668] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. The input is location information from the GPS sensor, and the output is the recorded location data. For example, it might use the smartphone's GPS function to acquire location information every 5 minutes.

[0669] Step 2:

[0670] The device transmits recorded movement information to a server via a communication module. The input is the recorded location data, and the output is the data transmitted to the server. For example, it might perform an operation that transmits location information via Wi-Fi every hour.

[0671] Step 3:

[0672] After the user consents to the retrieval of their payment history, the device calls the payment service API to retrieve the payment history. The input is a request to the payment service API, and the output is payment history data. Specifically, it retrieves information such as the payment date and time, amount, and store name.

[0673] Step 4:

[0674] The device first saves the acquired payment history to local storage and then sends it to the server. The input is the payment history data, and the output is the data sent to the server. For example, it can perform an action to send data at a set time.

[0675] Step 5:

[0676] The device periodically collects the user's search terms via the browser API. The input is browser history data, and the output is a list of search terms. For example, the Google Chrome history API is used to extract search keywords every 24 hours.

[0677] Step 6:

[0678] The terminal extracts search terms and sends them to the server. The input is a list of search terms, and the output is the data sent to the server. For example, it can perform an operation to periodically upload collected search terms to the server.

[0679] Step 7:

[0680] After the user consents to the use of the SNS API, the device retrieves posting information from the social networking service. The input is an SNS API request, and the output is the posting information. Specifically, it retrieves the post content, posting date and time, number of likes and comments, etc.

[0681] Step 8:

[0682] The device sends the acquired SNS posting information to the server. The input is the posting information, and the output is the data sent to the server. For example, the device might collect data every hour and send it to the server.

[0683] Step 9:

[0684] The server integrates all received data into a database. The input consists of various data sent to the server, and the output is the integrated database entry. MySQL or PostgreSQL can be used as the database.

[0685] Step 10:

[0686] The server applies natural language processing and machine learning algorithms to analyze the user's interests, concerns, and emotional state. The input is an integrated database entry, and the output is the analysis result. Specifically, it uses Python's spaCy to perform sentiment analysis on text and scikit-learn to cluster user interests.

[0687] Step 11:

[0688] The server updates the user profile based on the analysis results. The input is the analysis results, and the output is the updated user profile. The user profile includes estimated user interests, frequently visited locations, and typical consumption patterns.

[0689] Step 12:

[0690] The server uses a generative AI model to generate multiple message candidates based on an updated user profile. The input is the user profile, and the output is the message candidates. It uses the OpenAI API's GPT-3 to input prompts and generate messages.

[0691] Example prompt: "A user has bought coffee at a cafe, posted on social media that they want to relax, and is looking for recently recommended movies. Generate a relevant personalized message."

[0692] Step 13:

[0693] The server scores the relevance of each message candidate and selects the optimal message. The input is the message candidates, and the output is the selected optimal message. Methods such as TF-IDF and Doc2Vec are used for relevance scoring.

[0694] Step 14:

[0695] The server converts the selected messages into a daily calendar format and prepares them for delivery to the user. The input is the selected messages, and the output is the messages ready for delivery.

[0696] Step 15:

[0697] The device retrieves messages from the server at a set time. The input is the message ready for delivery on the server, and the output is the message retrieved by the device. For example, you can set it to retrieve a new message every morning at 7:00 AM.

[0698] Step 16:

[0699] The device notifies the user of a message. The input is a message acquired within the device, and the output is the notified message. Possible notification methods include smartphone push notifications, widgets placed on the home screen, or a dedicated app screen. As a specific example, a push notification might display a message such as, "Why not relax with a cup of coffee today and enjoy a recommended movie?"

[0700] (Application Example 1)

[0701] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0702] Traditional personalized messaging systems have limited ability to accurately analyze user behavior and interests and provide content that matches the user's lifestyle. Furthermore, there is a need for more sophisticated methods to analyze user interests and preferences and recommend the most suitable content based on that analysis.

[0703] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0704] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means including natural language processing and machine learning algorithms for integrating the acquired information and performing analysis according to the user's interests, and means for generating prompt sentences using a generative AI model that recommends content according to the user's interests and providing optimal content. This makes it possible to precisely analyze the user's behavior information and interests and provide optimal content as a personalized message.

[0705] "Movement information" refers to data about the user's current location and places visited, obtained using GPS or other location information systems.

[0706] "Visit information" refers to the history and detailed information of a user's visits to specific locations, and is data recorded by location information systems.

[0707] "Payment history" refers to a record of a user's purchasing activities, which is obtained through the API of an electronic payment service.

[0708] "Search terms" are keywords or phrases that users search for on internet search engines, and are collected via browser APIs.

[0709] "Posts" refer to information such as posts and comments made by users on social networking services, and are obtained via the APIs of each SNS.

[0710] "Integration" refers to the process of unifying information obtained from multiple different data sources and treating it as a single entity for analysis.

[0711] "Interest-based analysis" is the process of analyzing users' interests, preferences, and emotional states based on acquired data, and creating user profiles.

[0712] "Natural language processing" is a technology that enables computers to understand, analyze, and generate human language, and is used to understand text data such as user posts and search terms.

[0713] A "machine learning algorithm" is a computational method used to learn from large amounts of data and perform predictions and classifications, and is used to analyze user interests and preferences.

[0714] A "generative AI model" is a model that uses artificial intelligence technology to generate appropriate output (e.g., message or content recommendation) from input data.

[0715] A "prompt" is the input text given to a generative AI model and is used to control the model's output.

[0716] "Content" is a general term for information and entertainment provided to users, including movies, video clips, music, and ebooks.

[0717] This invention is a system that provides personalized messages based on users' daily behaviors and interests in order to enrich their lives. This system comprehensively analyzes users' movement information, payment history, search terms, and social networking service postings to generate and deliver individually optimized messages and content to the user.

[0718] Hardware and software usage:

[0719] Device: Smartphones have a built-in GPS sensor that records location information.

[0720] Software: Utilizes various APIs (payment service APIs, browser APIs, SNS APIs) for data collection.

[0721] Server: High-performance computers and specialized software (such as TensorFlow and OpenAI GPT-3) are used to process and analyze large amounts of data.

[0722] Data collection and analysis:

[0723] 1. Obtaining travel and visit information:

[0724] The device periodically records the user's current location and visited places using a GPS sensor and transmits this information to the server.

[0725] 2. Obtaining payment history:

[0726] After obtaining the user's consent, the device calls the payment service API to retrieve the user's payment history and sends it to the server.

[0727] 3. Obtaining search terms:

[0728] The system periodically collects users' search history via the browser API and sends it to the server.

[0729] 4. Obtaining writing information:

[0730] After obtaining permission to use the SNS API, the device retrieves user posting information from major SNS platforms and sends it to the server.

[0731] 5. Data Analysis:

[0732] The server integrates all received data into a database and applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[0733] Content generation and delivery:

[0734] 1. Update your user profile:

[0735] The server updates the user profile based on the analysis results.

[0736] 2. Generating personalized messages:

[0737] The server generates multiple message candidates using a generative AI model based on the user profile.

[0738] The relevance of each message candidate is scored, and the most suitable message is selected.

[0739] 3. Presenting the message:

[0740] The device retrieves a message from the server at a set time and notifies the user. Possible notification methods include push notifications, home screen widgets, and dedicated app screens.

[0741] Examples of specific cases and prompt statements:

[0742] For example, suppose a user visits a cafe on the weekend and buys a coffee. Also, consider a scenario where this user posts on a social networking service that they "want to relax" and recently searched for "recommended movies." In this case,

[0743] The server integrates and analyzes this information to estimate that "the user needs to relax" and "likes coffee."

[0744] Based on this, enter the following prompt into the generative AI model:

[0745] "User interest: Relaxation, Coffee, Movies. Generate a message."

[0746] The generated message will be a push notification to the user suggesting, "Why not relax with a cup of coffee today and enjoy a recommended movie?"

[0747] In this way, we can provide messages that are tailored to the user's lifestyle and interests, thereby increasing user satisfaction and improving their quality of life.

[0748] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0749] Step 1:

[0750] The device periodically records the user's current location and visited locations using a GPS sensor. Specifically, the device acquires location information at regular intervals and stores it as location data (latitude, longitude, and timestamp). The input is GPS sensor data, and the output is location data.

[0751] Step 2:

[0752] The device transmits recorded information about visited locations and travel routes to the server at appropriate intervals, such as every hour. The input is the location data acquired in step 1, and the output is the data transmitted to the server.

[0753] Step 3:

[0754] After the user gives consent to retrieve payment history, the device calls the payment service API to retrieve the payment history. Specifically, the device sends a request to the API endpoint and receives payment history data (purchase date and time, place of purchase, purchase amount) as a response. The input is the user's consent and the API request, and the output is the payment history data.

[0755] Step 4:

[0756] The terminal sends the acquired payment history data to the server. The input is the payment history data acquired in step 3, and the output is the data sent to the server.

[0757] Step 5:

[0758] After obtaining the user's consent to collect their search history, the device periodically collects the search history via the browser API. Specifically, the device sends a request to the browser API and stores the received search terms and search date and time. The input is the user's consent and the API request, and the output is the search history data.

[0759] Step 6:

[0760] The terminal sends the collected search history to the server. The input is the search history data obtained in step 5, and the output is the data sent to the server.

[0761] Step 7:

[0762] After the user gives their consent to use the SNS API, the device retrieves the user's posting information from major SNS platforms. Specifically, the device sends a request to the SNS API and receives the posting information (post content, posting date and time) as a response. The input is the user's consent and the API request, and the output is the posting information data.

[0763] Step 8:

[0764] The terminal sends the acquired write information to the server. The input is the write information data acquired in step 7, and the output is the data sent to the server.

[0765] Step 9:

[0766] The server integrates all received data (location information, payment history, search history, and writing information) into a database. Input is various data sent from the terminal, and output is the integrated database.

[0767] Step 10:

[0768] The server applies natural language processing (NLP) and machine learning algorithms to an integrated database to analyze users' interests, concerns, and emotional states. Specifically, it uses NLP to analyze text data and machine learning models to extract user behavior patterns. The input is data from the integrated database, and the output is the analysis results.

[0769] Step 11:

[0770] The server updates the user profile based on the analysis results. Specifically, it adds or modifies the user's interest categories and real-time sentiment indicators to the profile. The input is the analysis results, and the output is the updated user profile.

[0771] Step 12:

[0772] The server generates multiple message candidates using a generative AI model based on the user profile. The input is the updated user profile, and the output is the message candidates. As a concrete example, the AI ​​model is given the following prompt: "User interest: Relaxation, Coffee, Movies. Generate a message."

[0773] Step 13:

[0774] The server scores the relevance of each message candidate and selects the optimal message. The input is message candidates generated by the AI ​​model, and the output is the selected optimal message.

[0775] Step 14:

[0776] The device retrieves the most appropriate message from the server at a set time and notifies the user. Specific notification methods include push notifications, home screen widgets, and dedicated app screens. The input is the optimal message sent from the server, and the output is the notification to the user.

[0777] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0778] This invention is a system that provides personalized messages based on users' daily behaviors and interests in order to enrich their lives. This system collects and analyzes user movement information, payment history, search terms, and social networking service postings, and further recognizes the user's emotions using an emotion engine to generate individually optimized messages.

[0779] Collection of user information

[0780] Acquisition of travel and visit information

[0781] 1. The device periodically records the user's current location and visited locations using a GPS sensor.

[0782] 2. The device sends recorded information about visited locations and travel routes to the server at appropriate intervals, such as every hour.

[0783] Retrieving payment history

[0784] 1. After the user gives consent to retrieve payment history, the device calls the payment service API.

[0785] 2. The device retrieves the payment history and sends it to the server.

[0786] Retrieving search terms

[0787] 1. The device periodically collects search history via the browser API with the user's consent.

[0788] 2. The device sends the collected search terms to the server.

[0789] Retrieving writing information

[0790] 1. After the user agrees to use the SNS API, the device retrieves the user's posting information from major SNS platforms.

[0791] 2. The device sends the acquired SNS posting information to the server.

[0792] Data analysis and estimation by interest

[0793] 1. The server integrates the various data it receives into the database.

[0794] 2. The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[0795] 3. The server updates the user profile based on the analysis results.

[0796] Utilizing the Emotion Engine

[0797] 1. The server uses an emotion engine to analyze the sentiment of user posts and search terms.

[0798] 2. The server uses an emotion engine to update the user's emotional state in real time.

[0799] 3. The server takes emotional states into account to generate even more accurate messages.

[0800] Generating personalized messages

[0801] 1. The server uses a generative model that generates multiple message candidates based on the user profile and emotional state.

[0802] 2. The server scores the relevance of each message candidate and selects the most appropriate message.

[0803] 3. The server prepares to deliver the selected messages to the user in a daily calendar format.

[0804] Presenting a message

[0805] 1. The device retrieves the latest message from the server at the scheduled time.

[0806] 2. The device displays messages to the user via push notifications, home screen widgets, and dedicated app screens.

[0807] Specific example

[0808] For example, suppose a user visits a cafe on the weekend and purchases coffee using a payment service. Furthermore, they post on social media saying they "want to relax" and recently searched for "recommended movies." In this case,

[0809] 1. The device records the user's location information, including visits to cafes, and obtains coffee purchase information from the payment history.

[0810] 2. The device also collects SNS posts and search terms, and sends all the information to the server.

[0811] 3. The server integrates and analyzes various pieces of information to estimate that the user needs to relax and that they like coffee.

[0812] 4. The server uses an emotion engine to more deeply analyze the user's desire to relax.

[0813] 5. The server generates a personalized message such as, "Why not relax today with a cup of coffee and enjoy a recommended movie?"

[0814] 6. The device will display this message to the user via push notification.

[0815] In this way, by providing messages that are tailored to the user's life circumstances and emotions, it is possible to increase user satisfaction and improve their quality of life. Through this system, users can more easily receive more personalized support.

[0816] The following describes the processing flow.

[0817] Step 1:

[0818] The device activates its GPS sensor and obtains the user's current location.

[0819] Step 2:

[0820] The device records location information it acquires and saves the places and times the user has visited.

[0821] Step 3:

[0822] The device sends recorded movement information to the server at regular intervals (e.g., every hour).

[0823] Step 4:

[0824] The user consents to the collection of their payment history.

[0825] Step 5:

[0826] The terminal calls the payment service's API to retrieve the user's payment history.

[0827] Step 6:

[0828] The device sends the acquired payment history to the server.

[0829] Step 7:

[0830] The device periodically collects the user's search history using the browser's API.

[0831] Step 8:

[0832] The device sends the collected search history to the server.

[0833] Step 9:

[0834] The user agrees to the use of the SNS API.

[0835] Step 10:

[0836] The device uses the SNS API to retrieve user posting information.

[0837] Step 11:

[0838] The terminal sends the written information it has acquired to the server.

[0839] Step 12:

[0840] The server integrates received movement information, payment history, search history, and writing information into a database.

[0841] Step 13:

[0842] The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[0843] Step 14:

[0844] The server updates the user profile based on the results of the information analysis.

[0845] Step 15:

[0846] The server uses an emotion engine to analyze the sentiment of user posts and search terms.

[0847] Step 16:

[0848] The server uses an emotion engine to update the user's emotional state in real time.

[0849] Step 17:

[0850] The server uses a generative model that generates multiple message candidates based on emotional state and user profile.

[0851] Step 18:

[0852] The server scores the relevance of each message candidate and selects the most appropriate message.

[0853] Step 19:

[0854] The server prepares to deliver selected messages to the user in a daily calendar format.

[0855] Step 20:

[0856] The device retrieves the latest message from the server at a set time (e.g., 8 AM).

[0857] Step 21:

[0858] The device displays messages to the user via push notifications, home screen widgets, and dedicated app screens.

[0859] (Example 2)

[0860] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0861] In recent years, with the advancement of information technology, personalized systems that provide optimal information to individual users have attracted attention. However, conventional systems have struggled to adequately analyze user behavior and emotions and deliver the most appropriate message at the right time. Furthermore, they lacked mechanisms for integrating data from multiple sources and reflecting users' emotional states in real time. Therefore, effectively delivering personalized messages to users has been a challenge.

[0862] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0863] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means for integrating the acquired information and analyzing the user's interests and emotional state using natural language processing and machine learning algorithms, means including a generative AI model for generating messages based on the user's interests and emotional state, means for scoring the generated messages and selecting the optimal message, and means for presenting the selected messages to the user in a daily calendar format. This makes it possible to enrich the user's life and provide optimal personalized messages based on their emotions and behavior.

[0864] "Movement information" refers to information about where a user has moved. Specifically, it refers to data on latitude, longitude, and travel route obtained by GPS sensors.

[0865] "Visit information" refers to information about a user's visit to a specific location. This includes the name, location, and date and time of the visit.

[0866] "Payment history" refers to a record of purchases and payments made by a user. It includes information such as the date and time of purchase, store name, and purchased items, which are obtained through payment service APIs.

[0867] "Search terms" refer to the search queries that users enter into internet search engines. This data is used to analyze users' interests and concerns.

[0868] "Posting" refers to posts and comments made by users on social networking services or other platforms.

[0869] "Natural language processing" is the technology that enables computers to understand, interpret, and generate human language. Specifically, it includes text analysis, sentiment analysis, and keyword extraction.

[0870] A "machine learning algorithm" is a technique of artificial intelligence that automatically learns regularities and patterns from data. This makes it possible to estimate a user's interests and emotions.

[0871] An "emotion engine" refers to specialized software or algorithms used to analyze emotions from text data. It plays a role in inferring emotional states from user posts and search terms.

[0872] A "generative AI model" is an artificial intelligence model that utilizes deep learning-based text generation technology to generate messages based on the user's profile and emotional state.

[0873] "Scoring" is the process of assigning scores to generated message candidates based on factors such as relevance and appropriateness.

[0874] An "optimal message" refers to a personalized message that is best suited to the user's current situation and emotional state.

[0875] The "daily calendar format" is a format that presents messages sequentially day by day, providing users with an experience of receiving new information on a regular basis.

[0876] A "location information system" is a system that uses GPS and other location-determining technologies to determine the user's current location.

[0877] A "social networking service" is a platform on the internet for users to create, share, and comment on content.

[0878] This invention is a system that provides personalized messages based on users' daily behaviors and interests in order to enrich their lives. This system collects and analyzes user movement information, payment history, search terms, and social networking service postings, and further recognizes the user's emotions using an emotion engine to generate individually optimized messages.

[0879] Collection of user information

[0880] Acquisition of travel and visit information

[0881] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. For example, it collects latitude and longitude information every 10 minutes.

[0882] The device sends recorded information about visited locations and travel routes to the server in a batch process every hour.

[0883] Retrieving payment history

[0884] The user gives consent to the collection of payment history on their device. For example, they might check a checkbox in the app's settings screen that says, "I agree to the collection of payment history."

[0885] After the device has given consent, it calls a payment service API (e.g., PayPal or Stripe) to retrieve the user's latest payment history.

[0886] The device sends the acquired payment history data to the server.

[0887] Retrieving search terms

[0888] The device, with the user's consent, uses a browser API (for example, a Chrome extension) to collect the user's search history at regular intervals (for example, daily).

[0889] The device sends the collected search terms to the server.

[0890] Retrieving writing information

[0891] The user agrees to the use of APIs from major social networking services (e.g., Facebook and Twitter).

[0892] After the device gives its consent, it calls the SNS API to retrieve user posting information. Specifically, it collects recent posts and comments.

[0893] The terminal sends the acquired write information to the server.

[0894] Data analysis and estimation by interest

[0895] The server stores the various data it receives in a centralized database (for example, MySQL or MongoDB).

[0896] The server applies a natural language processing (NLP) engine (e.g., spaCy or NLTK) to the integrated data to analyze user search terms and social media posts.

[0897] Next, the server uses statistical and machine learning models (e.g., scikit-learn or TensorFlow) to estimate the user's interests and preferences. For example, it predicts what actions the user will take in the future based on keywords such as "relax" or "movies."

[0898] Based on the analysis results, the server updates the user profile to include the user's interests, preferences, and emotional state.

[0899] Utilizing the Emotion Engine

[0900] The server uses an emotion engine (for example, IBM Watson's Sentiment Analysis) to analyze user sentiment from their posts and search terms. For example, it might infer the sentiment "I want to relax" from a post that says "I want to relax."

[0901] The server reflects the analyzed emotion data in the user profile in real time.

[0902] The server takes emotional states into consideration and generates personalized messages based on detailed analysis.

[0903] Generating personalized messages

[0904] The server uses a generative AI model (e.g., OpenAI GPT-3) to generate multiple message suggestions based on the user profile and emotional state. For example, it might generate a message such as, "Why not relax with a cup of coffee and enjoy a recommended movie today?"

[0905] Next, the server assigns relevance scores to the multiple message candidates that have been generated (for example, scores based on the user's current behavior and sentiment) and selects the most suitable message.

[0906] Presenting a message

[0907] The server stores the selected messages in a daily log format and prepares for the next presentation.

[0908] The device retrieves the latest personalized message from the server at a set time (for example, 8 AM).

[0909] The device displays messages it has received to the user via push notifications, home screen widgets, and a dedicated app screen. For example, it might display a message like, "Why not relax with a cup of coffee and enjoy a recommended movie today?" in the smartphone's notification bar.

[0910] Specific example

[0911] For example, a user visited a cafe on the weekend and purchased coffee using a payment service. Furthermore, they posted on social media that they wanted to "relax" and recently searched for "recommended movies." In this case,

[0912] 1. The device records the user's location information, including visits to cafes, and obtains coffee purchase information from the payment history.

[0913] 2. The device also collects SNS posts and search terms, and sends all the information to the server.

[0914] 3. The server integrates and analyzes various pieces of information to estimate that the user needs to relax and that they like coffee.

[0915] 4. The server uses an emotion engine to more deeply analyze the user's desire to relax.

[0916] 5. The server generates a personalized message such as, "Why not relax today with a cup of coffee and enjoy a recommended movie?"

[0917] 6. The device will display this message to the user via push notification.

[0918] In this way, by providing messages that are tailored to the user's life circumstances and emotions, it is possible to increase user satisfaction and improve their quality of life. Through this system, users can more easily receive more personalized support.

[0919] Example of a prompt

[0920] For example, a prompt statement to be input to a generative AI model can be written as follows:

[0921] "A user visited a cafe on the weekend and purchased coffee using a payment service. Furthermore, they posted on social media that they wanted to 'relax,' and recently searched for 'recommended movies.' Please generate a personalized message for this user."

[0922] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0923] Step 1:

[0924] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. For example, it acquires latitude and longitude information every 10 minutes and stores it in local memory.

[0925] Input: Program start command, GPS sensor data

[0926] Output: User's current location and visited locations information (latitude and longitude data)

[0927] Step 2:

[0928] The device sends recorded information about visited locations and travel routes to the server in a batch process every hour. Specifically, it sends the data to a message queue via an API.

[0929] Input: Recorded latitude and longitude data

[0930] Output: Data on visited locations and travel routes sent to the server.

[0931] Step 3:

[0932] The user gives consent to the collection of payment history on their device. For example, they might check a checkbox in the app's settings screen that says, "I agree to the collection of payment history."

[0933] Input: User consent action

[0934] Output: Consent information flags

[0935] Step 4:

[0936] After the device has given consent, it calls a payment service API to retrieve the user's latest payment history. For example, it might use the PayPal or Stripe API to retrieve transaction data.

[0937] Input: Consent information flag

[0938] Output: Payment history data

[0939] Step 5:

[0940] The device sends the acquired payment history data to the server. Specifically, it sends the data to the server via an HTTP request.

[0941] Input: Payment history data

[0942] Output: Payment history data sent to the server

[0943] Step 6:

[0944] With the user's consent, the device uses the browser API to collect the user's search history at regular intervals (for example, daily). Specifically, it uses a Chrome extension to collect search queries.

[0945] Input: User consent information, browser search data

[0946] Output: Collected search terms

[0947] Step 7:

[0948] The device sends the collected search terms to the server. For example, it might send them in JSON format via an HTTP request.

[0949] Input: Collected search terms

[0950] Output: Search words sent to the server

[0951] Step 8:

[0952] The user agrees to the use of APIs from major social networking services (SNS). Specifically, they check a checkbox in the app's settings screen that says something like, "I agree to the use of SNS APIs."

[0953] Input: User consent action

[0954] Output: Consent information flags

[0955] Step 9:

[0956] After the device gives its consent, it calls the SNS API to retrieve user posting information. For example, it uses the Facebook or Twitter API to retrieve recent posts and comments.

[0957] Input: Consent information flag

[0958] Output: SNS posting information

[0959] Step 10:

[0960] The device sends the acquired SNS posting information to the server. Specifically, it sends the data to the server via an HTTP request.

[0961] Input: Social media posting information

[0962] Output: SNS posting information sent to the server

[0963] Step 11:

[0964] The server stores various data it receives in a centralized database. For example, MySQL or MongoDB can be used to store the data.

[0965] Input: Visit location data, payment history data, search terms, social media postings

[0966] Output: Various data stored in the database

[0967] Step 12:

[0968] The server applies a natural language processing (NLP) engine to the integrated data to analyze user information. For example, it might use spaCy or NLTK to analyze text data and extract keywords and sentiments.

[0969] Input: Integrated data stored in the database

[0970] Output: Analyzed user interests and emotional states

[0971] Step 13:

[0972] The server uses statistical and machine learning models to estimate user interests and preferences. For example, it can use scikit-learn or TensorFlow to train data and generate an interest prediction model.

[0973] Input: Analyzed user interests and emotional states

[0974] Output: Estimated user interests and emotional states

[0975] Step 14:

[0976] The server uses a generative AI model to generate multiple message candidates based on the user profile and emotional state. For example, it can use OpenAI GPT-3 to generate personalized messages for the user.

[0977] Input: Estimated user interests and emotional state

[0978] Output: List of message candidates

[0979] Step 15:

[0980] The server assigns a relevance score to the generated message candidates and selects the most suitable message. Specifically, it uses a scoring algorithm to evaluate the relevance of each message.

[0981] Input: List of message suggestions

[0982] Output: Optimal message

[0983] Step 16:

[0984] The server stores the selected messages in a daily log format and prepares for the next presentation.

[0985] Input: Optimal message

[0986] Output: Messages stored in a daily calendar format

[0987] Step 17:

[0988] The device retrieves the latest personalized message from the server at a set time (for example, 8 AM). Specifically, it periodically sends requests to the server via an API.

[0989] Input: Time setting

[0990] Output: Latest message retrieved

[0991] Step 18:

[0992] The device displays messages it has received to the user via push notifications, home screen widgets, and dedicated app screens. For example, it displays messages in the smartphone's notification bar.

[0993] Input: Latest message retrieved

[0994] Output: Message displayed to the user

[0995] (Application Example 2)

[0996] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0997] In modern factories, improving operator efficiency is a critical challenge. However, because individualized support is required, taking into account each operator's work history and current emotional state, general assistance and advice have limited effectiveness. Furthermore, operators themselves may experience stress or a lack of job satisfaction, which can lead to decreased productivity. Therefore, a system is needed that can grasp each operator's individual situation and emotions in real time and provide personalized messages based on that information.

[0998] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0999] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means for integrating the acquired information and performing analysis according to the user's interests, means for analyzing the user's emotions using an emotion engine, means for generating personalized messages based on the analysis results and emotional state, and means for presenting the generated personalized messages to the user. This makes it possible for robots deployed in a factory to generate and present personalized messages based on the operator's past work history, the situation during work, and information acquired from social networking services.

[1000] "Movement information" refers to the movement history and location data of users and operators, including where they have moved to.

[1001] "Visit information" refers to records of when users or operators visited a specific location and data on the time they spent at that location.

[1002] "Payment history" refers to records of transactions made by users or operators, as well as data on what goods or services were paid for.

[1003] "Search terms" refer to keywords or phrases that users or operators enter into internet search engines.

[1004] "Posts" refer to text or comments that users or operators post on social networking services or other platforms.

[1005] An "emotion engine" refers to an algorithm or system that analyzes emotions and emotional states from user or operator text data.

[1006] "Personalized messages" refer to messages and notifications that are customized based on the individual data of the user or operator.

[1007] A "robot" refers to a machine or device used in factories or other work environments to perform automated tasks.

[1008] An "operator" refers to a human worker who is responsible for operating machinery or systems in a factory or other work environment.

[1009] This invention provides a system that delivers personalized messages based on individual circumstances and emotional states to improve the operational efficiency of operators in a factory. This system collects and analyzes user movement information, payment history, search terms, and writing information, and uses an emotion engine to recognize emotional states, thereby generating and presenting individually optimized messages.

[1010] First, the system program uses GPS devices to acquire operator movement and visit information. For example, it utilizes a location information system such as the Garmin GLO 2. Next, to obtain payment history, the PaymentService API is used to collect payment information for goods and services made by operators within the factory. In addition, the Browser History API is used to obtain keywords searched by operators on the internet. Furthermore, APIs of major social networking services (SNS) are used to collect operator posting information. All of this data is sent to the server.

[1011] The server first integrates the collected data and uses natural language processing (NLP) and machine learning algorithms to analyze the operator's interests, concerns, and emotional state. Next, it uses an emotion engine (e.g., Google Cloud Natural Language API) to analyze the operator's emotional state in detail. Based on this analysis and the emotional state, a generative AI model (e.g., GPT-3 model) is used to generate personalized messages. These messages are created considering the operator's past work history, current work situation, and social media posts.

[1012] The generated personalized messages are presented to operators through robots placed in the factory. Presentation methods include the robot's display, voice output, or a dedicated application. For example, if an operator posts on social media that their work efficiency is low and is searching for "tips for improving productivity," the AI ​​model might generate a message such as "Take more breaks today and relax," and the robot would then present that message.

[1013] (Specific example)

[1014] Operator A posted on social media that "work efficiency is declining." They also searched for "tips for improving productivity" in their browser.

[1015] Movement information: The GPS device tracked Operator A's movements within the factory.

[1016] Payment history: Purchased an energy drink from a vending machine inside the factory.

[1017] Search terms: "Tips for improving productivity"

[1018] Social media post: "Work efficiency is declining."

[1019] (Example of a prompt message)

[1020] "Operator A has posted on social media about a decline in work efficiency. Their search history also shows they are looking for information on improving productivity. Based on this information, we will provide refreshing advice and personalized messages."

[1021] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1022] Step 1:

[1023] The terminal periodically records the operator's movement and visit information using a GPS device. Specifically, it acquires location data from a GPS device (e.g., Garmin GLO 2) and stores this data in its internal memory. In this step, the input is the location information from the GPS device, and the output is the recorded movement information.

[1024] Step 2:

[1025] The terminal periodically retrieves payment history, so it calls the PaymentService API to collect payment data. It retrieves transaction information from vending machines and shops. The inputs in this step are the user ID and payment event, and the output is payment history data.

[1026] Step 3:

[1027] The device periodically collects the user's search terms using the browser history API. This is a list of keywords and phrases the user has recently searched for. The input for this step is browser history data, and the output is a list of search terms.

[1028] Step 4:

[1029] The device uses the SNS API to retrieve user posting information. It periodically collects SNS posts and sends them to the server. The input in this step is the posted data obtained via the SNS API, and the output is the posting information.

[1030] Step 5:

[1031] The terminal sends all collected information (movement information, payment history, search terms, and posting information) to the server. The server integrates this information and stores it in a database. The input in this step is the user data sent from the terminal, and the output is the integrated dataset.

[1032] Step 6:

[1033] The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state. This process identifies the user's current state and trends based on the collected data. The input in this step is an integrated dataset, and the output is the analyzed user profile.

[1034] Step 7:

[1035] The server uses an emotion engine (e.g., Google Cloud Natural Language API) to analyze the user's emotional state. It analyzes collected writing information and search terms to identify the user's emotions. The input in this step is the user's text data, and the output is emotional state data.

[1036] Step 8:

[1037] The server uses a generated AI model (e.g., a GPT-3 model) to generate personalized messages based on the analysis results and emotional state. It generates the optimal message tailored to the user's profile. The input in this step is the analyzed user profile and emotional state data, and the output is the personalized message.

[1038] Step 9:

[1039] The terminal retrieves a personalized message generated from the server and presents it to the operator via the display or voice output of a robot placed in the factory. In this step, the input is the personalized message, and the output is a notification to the operator.

[1040] Step 10:

[1041] The user acts based on the personalized message presented to them, aiming to improve efficiency and reduce stress. In this step, the input is the personalized message, and the output is the user's action.

[1042] By following these steps, it is possible to understand the individual circumstances and emotions of users in real time and provide optimal messages based on that understanding, thereby improving operational efficiency in the factory.

[1043] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1044] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1045] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1046] [Third Embodiment]

[1047] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1048] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1049] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1050] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1051] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1052] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1053] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1054] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1055] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1056] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1057] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1058] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1059] This invention aims to enrich users' lives by providing personalized messages based on their daily behavior and interests. This system comprehensively analyzes users' movement information, payment history, search terms, and social networking service postings to generate individually optimized messages.

[1060] Collection of user information

[1061] Acquisition of travel and visit information

[1062] 1. The device periodically records the user's current location and visited locations using a GPS sensor.

[1063] 2. The device sends recorded information about visited locations and travel routes to the server at appropriate intervals, such as every hour.

[1064] Retrieving payment history

[1065] 1. After the user gives consent to retrieve payment history, the device calls a payment service API such as PayPay.

[1066] 2. The device retrieves the payment history and sends it to the server.

[1067] Retrieving search terms

[1068] 1. The device periodically collects search history via the browser API with the user's consent.

[1069] 2. The device sends the collected search terms to the server.

[1070] Retrieving writing information

[1071] 1. After the user agrees to use the SNS API, the device retrieves the user's posting information from major SNS platforms.

[1072] 2. The device sends the acquired SNS posting information to the server.

[1073] Data analysis and estimation by interest

[1074] 3. The server integrates all received data into the database.

[1075] 4. The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[1076] 5. The server updates the user profile based on the analysis results.

[1077] Generating personalized messages

[1078] 6. Use a generative model in which the server generates multiple message candidates based on the user profile.

[1079] 7. The server scores the relevance of each message candidate and selects the most appropriate message.

[1080] 8. The server prepares to deliver the selected messages to the user in a daily calendar format.

[1081] Presenting a message

[1082] 9. The device retrieves the message from the server at the scheduled time.

[1083] 10. The device notifies the user of the message. Possible notification methods include push notifications, home screen widgets, and dedicated app screens.

[1084] Specific example

[1085] For example, suppose a user visits a cafe on the weekend and purchases coffee using PayPay. Also, suppose that user has posted on social media that they "want to relax" and recently searched for "recommended movies." In this case,

[1086] 1. The device records the user's location information, including visits to cafes, and obtains coffee purchase information from the payment history.

[1087] 2. The device also collects SNS posts and search terms, and sends all the information to the server.

[1088] 3. The server integrates and analyzes this information to estimate that the user needs to relax and that they like coffee.

[1089] 4. The server generates a message saying, "Why not relax today, enjoy a cup of coffee, and watch a recommended movie?"

[1090] 5. The device displays this message to the user via push notification.

[1091] In this way, we can provide messages that are tailored to the user's lifestyle and interests. Through this entire system, we can increase user satisfaction and improve their quality of life.

[1092] The following describes the processing flow.

[1093] Step 1:

[1094] The device activates its GPS sensor and obtains the user's current location.

[1095] Step 2:

[1096] The device records location information it acquires and saves the places and times the user has visited.

[1097] Step 3:

[1098] The device sends recorded movement information to the server at regular intervals (e.g., every hour).

[1099] Step 4:

[1100] The user consents to the collection of their payment history.

[1101] Step 5:

[1102] The terminal calls the payment service's API to retrieve the user's payment history.

[1103] Step 6:

[1104] The device sends the acquired payment history to the server.

[1105] Step 7:

[1106] The device periodically collects the user's search history using the browser's API.

[1107] Step 8:

[1108] The device sends the collected search history to the server.

[1109] Step 9:

[1110] The user agrees to the use of the SNS API.

[1111] Step 10:

[1112] The device uses the SNS API to retrieve user posting information.

[1113] Step 11:

[1114] The terminal sends the written information it has acquired to the server.

[1115] Step 12:

[1116] The server integrates received movement information, payment history, search history, and writing information into a database.

[1117] Step 13:

[1118] The server applies natural language processing (NLP) and machine learning models to analyze the user's interests, concerns, and emotional state.

[1119] Step 14:

[1120] The server updates the user profile based on the analysis results.

[1121] Step 15:

[1122] The server generates multiple message candidates using a generative model based on the user profile.

[1123] Step 16:

[1124] The server scores the relevance of each message candidate and selects the most appropriate message.

[1125] Step 17:

[1126] The server prepares selected messages for the user in a daily calendar format.

[1127] Step 18:

[1128] The device retrieves the latest message from the server at a set time (e.g., 8 AM).

[1129] Step 19:

[1130] The device displays the message to the user via push notifications, home screen widgets, or a dedicated app screen.

[1131] (Example 1)

[1132] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1133] In modern society, providing personalized messages based on individual interests and behaviors is crucial for enriching users' lives. However, no system exists that effectively integrates diverse user information (such as travel data, payment history, search terms, and social networking service posts) to generate appropriate messages. Therefore, it is necessary to address the problems of information overload that users experience daily and the resulting lack of appropriate information.

[1134] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1135] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means for integrating the acquired information and performing analysis according to the user's interests, means for updating the user profile, means for applying natural language processing and machine learning algorithms, means for generating message candidates using a generative AI model, means for scoring the generated message candidates and selecting the optimal message, and notification means for presenting the selected message to the user. This enables the effective integration and analysis of diverse user information and the provision of individually optimized personalized messages.

[1136] "Movement information" refers to data about a user's location and places they have visited, and is primarily obtained using location information systems such as GPS sensors.

[1137] "Visit information" refers to detailed data about a user's visit to a specific location, including the date and time of the visit and the name of the location.

[1138] "Payment history" refers to data about financial transactions and purchase behaviors performed by a user, and includes information such as the date and time of payment, amount, and place of purchase, obtained through payment service APIs.

[1139] "Search terms" are keywords or phrases entered by users during internet searches, and are collected via browser APIs and other means.

[1140] "Posts" refer to content or comments that users submit to social networking services or other online platforms.

[1141] "Integration" is the process of centrally combining different types of data and converting them into a format that is easy to store in a database or similar system.

[1142] "Interest-based analysis" is a method that uses natural language processing and machine learning algorithms to analyze users' interests, concerns, and emotional states from collected data.

[1143] A "user profile" is a database description created based on a user's behavior, interests, and concerns, and includes individual characteristics and tendencies.

[1144] "Natural language processing" is a computer science technique for understanding and analyzing human language, and it includes tasks such as sentiment analysis of text and keyword extraction.

[1145] A "machine learning algorithm" is a computer algorithm that learns patterns and knowledge from data, and is used for clustering, classification, prediction, and other purposes.

[1146] A "generative AI model" is an artificial intelligence model that generates sentences and texts based on large amounts of data, and is a method for creating new messages and recommendations.

[1147] "Scoring" is the process of evaluating generated message candidates and quantifying their relevance and relevance.

[1148] The "optimal message" is the message that best matches the user's profile and analysis results, and provides value to them.

[1149] "Notifications" are methods for conveying information to users based on specific times or events, and include push notifications and widgets.

[1150] This invention is a system that provides personalized messages based on daily behaviors and interests, with the aim of enriching users' lives. This system collects user information, analyzes data and estimates interests, generates personalized messages, and presents those messages. Its detailed configuration is described below.

[1151] Collection of user information

[1152] 1. Obtaining travel and visit information:

[1153] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. This utilizes the GPS function of the smartphone.

[1154] The device transmits recorded information to the server via a communication module. Mobile communication networks or Wi-Fi are used as the means of communication.

[1155] 2. Obtaining payment history:

[1156] When a user consents to the retrieval of their payment history, the device calls the payment service API to retrieve the payment history. The retrieved information includes the payment date and time, amount, and store name.

[1157] The device temporarily saves the acquired payment history to local storage and then sends it to the server. For example, it may be updated periodically.

[1158] 3. Obtaining search terms:

[1159] The device periodically collects the user's search terms via the browser API. This process uses APIs such as Google Chrome's history API to extract search keywords every 24 hours.

[1160] The terminal sends the extracted search words to the server, and the transmitted data includes the search date and time and the search query.

[1161] 4. Obtaining writing information:

[1162] After a user consents to the use of the SNS API, the device retrieves posting information from social networking services. Specifically, this includes the content of the post, the date and time of posting, and the number of likes and comments.

[1163] The device sends the acquired SNS posting information to the server.

[1164] Data analysis and estimation by interest

[1165] The server integrates all received data into a database. Databases such as MySQL or PostgreSQL are used.

[1166] The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state. Here, we use spaCy, a Python NLP library, to perform sentiment analysis on text, and scikit-learn to cluster the user's interests.

[1167] The server updates the user profile based on the analysis results. The user profile includes estimated user interests, frequently visited locations, and typical consumption patterns.

[1168] Generating personalized messages

[1169] The server uses a generative AI model to generate multiple message suggestions based on an updated user profile. Specifically, it uses the OpenAI API's GPT-3 to input a prompt message such as, "The user bought coffee at a cafe, posted on social media that they want to relax, and are looking for recently recommended movies. Please generate relevant personalized messages."

[1170] The server scores the relevance of each candidate message and selects the optimal message. Methods such as TF-IDF and Doc2Vec are used for relevance scoring.

[1171] The server converts the selected messages into a daily calendar format and prepares to deliver them to the user.

[1172] Presenting a message

[1173] The device retrieves messages from the server at a set time. For example, you can set it to retrieve new messages every morning at 7:00 AM.

[1174] The device notifies the user of a message. Notification methods include smartphone push notifications, widgets placed on the home screen, or dedicated app screens. For example, a push notification might display a message such as, "Why not relax with a cup of coffee today and enjoy a recommended movie?"

[1175] This configuration enables the effective integration and analysis of diverse user information, allowing for the delivery of individually optimized, personalized messages. Through this entire system, user satisfaction can be enhanced, and their quality of life can be improved.

[1176] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1177] Step 1:

[1178] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. The input is location information from the GPS sensor, and the output is the recorded location data. For example, it might use the smartphone's GPS function to acquire location information every 5 minutes.

[1179] Step 2:

[1180] The device transmits recorded movement information to a server via a communication module. The input is the recorded location data, and the output is the data transmitted to the server. For example, it might perform an operation that transmits location information via Wi-Fi every hour.

[1181] Step 3:

[1182] After the user consents to the retrieval of their payment history, the device calls the payment service API to retrieve the payment history. The input is a request to the payment service API, and the output is payment history data. Specifically, it retrieves information such as the payment date and time, amount, and store name.

[1183] Step 4:

[1184] The device first saves the acquired payment history to local storage and then sends it to the server. The input is the payment history data, and the output is the data sent to the server. For example, it can perform an action to send data at a set time.

[1185] Step 5:

[1186] The device periodically collects the user's search terms via the browser API. The input is browser history data, and the output is a list of search terms. For example, the Google Chrome history API is used to extract search keywords every 24 hours.

[1187] Step 6:

[1188] The terminal extracts search terms and sends them to the server. The input is a list of search terms, and the output is the data sent to the server. For example, it can perform an operation to periodically upload collected search terms to the server.

[1189] Step 7:

[1190] After the user consents to the use of the SNS API, the device retrieves posting information from the social networking service. The input is an SNS API request, and the output is the posting information. Specifically, it retrieves the post content, posting date and time, number of likes and comments, etc.

[1191] Step 8:

[1192] The device sends the acquired SNS posting information to the server. The input is the posting information, and the output is the data sent to the server. For example, the device might collect data every hour and send it to the server.

[1193] Step 9:

[1194] The server integrates all received data into a database. The input consists of various data sent to the server, and the output is the integrated database entry. MySQL or PostgreSQL can be used as the database.

[1195] Step 10:

[1196] The server applies natural language processing and machine learning algorithms to analyze the user's interests, concerns, and emotional state. The input is an integrated database entry, and the output is the analysis result. Specifically, it uses Python's spaCy to perform sentiment analysis on text and scikit-learn to cluster user interests.

[1197] Step 11:

[1198] The server updates the user profile based on the analysis results. The input is the analysis results, and the output is the updated user profile. The user profile includes estimated user interests, frequently visited locations, and typical consumption patterns.

[1199] Step 12:

[1200] The server uses a generative AI model to generate multiple message candidates based on an updated user profile. The input is the user profile, and the output is the message candidates. It uses the OpenAI API's GPT-3 to input prompts and generate messages.

[1201] Example prompt: "A user has bought coffee at a cafe, posted on social media that they want to relax, and is looking for recently recommended movies. Generate a relevant personalized message."

[1202] Step 13:

[1203] The server scores the relevance of each message candidate and selects the optimal message. The input is the message candidates, and the output is the selected optimal message. Methods such as TF-IDF and Doc2Vec are used for relevance scoring.

[1204] Step 14:

[1205] The server converts the selected messages into a daily calendar format and prepares them for delivery to the user. The input is the selected messages, and the output is the messages ready for delivery.

[1206] Step 15:

[1207] The device retrieves messages from the server at a set time. The input is the message ready for delivery on the server, and the output is the message retrieved by the device. For example, you can set it to retrieve a new message every morning at 7:00 AM.

[1208] Step 16:

[1209] The device notifies the user of a message. The input is a message acquired within the device, and the output is the notified message. Possible notification methods include smartphone push notifications, widgets placed on the home screen, or a dedicated app screen. As a specific example, a push notification might display a message such as, "Why not relax with a cup of coffee today and enjoy a recommended movie?"

[1210] (Application Example 1)

[1211] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1212] Traditional personalized messaging systems have limited ability to accurately analyze user behavior and interests and provide content that matches the user's lifestyle. Furthermore, there is a need for more sophisticated methods to analyze user interests and preferences and recommend the most suitable content based on that analysis.

[1213] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1214] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means including natural language processing and machine learning algorithms for integrating the acquired information and performing analysis according to the user's interests, and means for generating prompt sentences using a generative AI model that recommends content according to the user's interests and providing optimal content. This makes it possible to precisely analyze the user's behavior information and interests and provide optimal content as a personalized message.

[1215] "Movement information" refers to data about the user's current location and places visited, obtained using GPS or other location information systems.

[1216] "Visit information" refers to the history and detailed information of a user's visits to specific locations, and is data recorded by location information systems.

[1217] "Payment history" refers to a record of a user's purchasing activities, which is obtained through the API of an electronic payment service.

[1218] "Search terms" are keywords or phrases that users search for on internet search engines, and are collected via browser APIs.

[1219] "Posts" refer to information such as posts and comments made by users on social networking services, and are obtained via the APIs of each SNS.

[1220] "Integration" refers to the process of unifying information obtained from multiple different data sources and treating it as a single entity for analysis.

[1221] "Interest-based analysis" is the process of analyzing users' interests, preferences, and emotional states based on acquired data, and creating user profiles.

[1222] "Natural language processing" is a technology that enables computers to understand, analyze, and generate human language, and is used to understand text data such as user posts and search terms.

[1223] A "machine learning algorithm" is a computational method used to learn from large amounts of data and perform predictions and classifications, and is used to analyze user interests and preferences.

[1224] A "generative AI model" is a model that uses artificial intelligence technology to generate appropriate output (e.g., message or content recommendation) from input data.

[1225] A "prompt" is the input text given to a generative AI model and is used to control the model's output.

[1226] "Content" is a general term for information and entertainment provided to users, including movies, video clips, music, and ebooks.

[1227] This invention is a system that provides personalized messages based on users' daily behaviors and interests in order to enrich their lives. This system comprehensively analyzes users' movement information, payment history, search terms, and social networking service postings to generate and deliver individually optimized messages and content to the user.

[1228] Hardware and software usage:

[1229] Device: Smartphones have a built-in GPS sensor that records location information.

[1230] Software: Utilizes various APIs (payment service APIs, browser APIs, SNS APIs) for data collection.

[1231] Server: High-performance computers and specialized software (such as TensorFlow and OpenAI GPT-3) are used to process and analyze large amounts of data.

[1232] Data collection and analysis:

[1233] 1. Obtaining travel and visit information:

[1234] The device periodically records the user's current location and visited places using a GPS sensor and transmits this information to the server.

[1235] 2. Obtaining payment history:

[1236] After obtaining the user's consent, the device calls the payment service API to retrieve the user's payment history and sends it to the server.

[1237] 3. Obtaining search terms:

[1238] The system periodically collects users' search history via the browser API and sends it to the server.

[1239] 4. Obtaining writing information:

[1240] After obtaining permission to use the SNS API, the device retrieves user posting information from major SNS platforms and sends it to the server.

[1241] 5. Data Analysis:

[1242] The server integrates all received data into a database and applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[1243] Content generation and delivery:

[1244] 1. Update your user profile:

[1245] The server updates the user profile based on the analysis results.

[1246] 2. Generating personalized messages:

[1247] The server generates multiple message candidates using a generative AI model based on the user profile.

[1248] The relevance of each message candidate is scored, and the most suitable message is selected.

[1249] 3. Presenting the message:

[1250] The device retrieves a message from the server at a set time and notifies the user. Possible notification methods include push notifications, home screen widgets, and dedicated app screens.

[1251] Examples of specific cases and prompt statements:

[1252] For example, suppose a user visits a cafe on the weekend and buys a coffee. Also, consider a scenario where this user posts on a social networking service that they "want to relax" and recently searched for "recommended movies." In this case,

[1253] The server integrates and analyzes this information to estimate that "the user needs to relax" and "likes coffee."

[1254] Based on this, enter the following prompt into the generative AI model:

[1255] "User interest: Relaxation, Coffee, Movies. Generate a message."

[1256] The generated message will be a push notification to the user suggesting, "Why not relax with a cup of coffee today and enjoy a recommended movie?"

[1257] In this way, we can provide messages that are tailored to the user's lifestyle and interests, thereby increasing user satisfaction and improving their quality of life.

[1258] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1259] Step 1:

[1260] The device periodically records the user's current location and visited locations using a GPS sensor. Specifically, the device acquires location information at regular intervals and stores it as location data (latitude, longitude, and timestamp). The input is GPS sensor data, and the output is location data.

[1261] Step 2:

[1262] The device transmits recorded information about visited locations and travel routes to the server at appropriate intervals, such as every hour. The input is the location data acquired in step 1, and the output is the data transmitted to the server.

[1263] Step 3:

[1264] After the user gives consent to retrieve payment history, the device calls the payment service API to retrieve the payment history. Specifically, the device sends a request to the API endpoint and receives payment history data (purchase date and time, place of purchase, purchase amount) as a response. The input is the user's consent and the API request, and the output is the payment history data.

[1265] Step 4:

[1266] The terminal sends the acquired payment history data to the server. The input is the payment history data acquired in step 3, and the output is the data sent to the server.

[1267] Step 5:

[1268] After obtaining the user's consent to collect their search history, the device periodically collects the search history via the browser API. Specifically, the device sends a request to the browser API and stores the received search terms and search date and time. The input is the user's consent and the API request, and the output is the search history data.

[1269] Step 6:

[1270] The terminal sends the collected search history to the server. The input is the search history data obtained in step 5, and the output is the data sent to the server.

[1271] Step 7:

[1272] After the user gives their consent to use the SNS API, the device retrieves the user's posting information from major SNS platforms. Specifically, the device sends a request to the SNS API and receives the posting information (post content, posting date and time) as a response. The input is the user's consent and the API request, and the output is the posting information data.

[1273] Step 8:

[1274] The terminal sends the acquired write information to the server. The input is the write information data acquired in step 7, and the output is the data sent to the server.

[1275] Step 9:

[1276] The server integrates all received data (location information, payment history, search history, and writing information) into a database. Input is various data sent from the terminal, and output is the integrated database.

[1277] Step 10:

[1278] The server applies natural language processing (NLP) and machine learning algorithms to an integrated database to analyze users' interests, concerns, and emotional states. Specifically, it uses NLP to analyze text data and machine learning models to extract user behavior patterns. The input is data from the integrated database, and the output is the analysis results.

[1279] Step 11:

[1280] The server updates the user profile based on the analysis results. Specifically, it adds or modifies the user's interest categories and real-time sentiment indicators to the profile. The input is the analysis results, and the output is the updated user profile.

[1281] Step 12:

[1282] The server generates multiple message candidates using a generative AI model based on the user profile. The input is the updated user profile, and the output is the message candidates. As a concrete example, the AI ​​model is given the following prompt: "User interest: Relaxation, Coffee, Movies. Generate a message."

[1283] Step 13:

[1284] The server scores the relevance of each message candidate and selects the optimal message. The input is message candidates generated by the AI ​​model, and the output is the selected optimal message.

[1285] Step 14:

[1286] The device retrieves the most appropriate message from the server at a set time and notifies the user. Specific notification methods include push notifications, home screen widgets, and dedicated app screens. The input is the optimal message sent from the server, and the output is the notification to the user.

[1287] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1288] This invention is a system that provides personalized messages based on users' daily behaviors and interests in order to enrich their lives. This system collects and analyzes user movement information, payment history, search terms, and social networking service postings, and further recognizes the user's emotions using an emotion engine to generate individually optimized messages.

[1289] Collection of user information

[1290] Acquisition of travel and visit information

[1291] 1. The device periodically records the user's current location and visited locations using a GPS sensor.

[1292] 2. The device sends recorded information about visited locations and travel routes to the server at appropriate intervals, such as every hour.

[1293] Retrieving payment history

[1294] 1. After the user gives consent to retrieve payment history, the device calls the payment service API.

[1295] 2. The device retrieves the payment history and sends it to the server.

[1296] Retrieving search terms

[1297] 1. The device periodically collects search history via the browser API with the user's consent.

[1298] 2. The device sends the collected search terms to the server.

[1299] Retrieving writing information

[1300] 1. After the user agrees to use the SNS API, the device retrieves the user's posting information from major SNS platforms.

[1301] 2. The device sends the acquired SNS posting information to the server.

[1302] Data analysis and estimation by interest

[1303] 1. The server integrates the various data it receives into the database.

[1304] 2. The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[1305] 3. The server updates the user profile based on the analysis results.

[1306] Utilizing the Emotion Engine

[1307] 1. The server uses an emotion engine to analyze the sentiment of user posts and search terms.

[1308] 2. The server uses an emotion engine to update the user's emotional state in real time.

[1309] 3. The server takes emotional states into account to generate even more accurate messages.

[1310] Generating personalized messages

[1311] 1. The server uses a generative model that generates multiple message candidates based on the user profile and emotional state.

[1312] 2. The server scores the relevance of each message candidate and selects the most appropriate message.

[1313] 3. The server prepares to deliver the selected messages to the user in a daily calendar format.

[1314] Presenting a message

[1315] 1. The device retrieves the latest message from the server at the scheduled time.

[1316] 2. The device displays messages to the user via push notifications, home screen widgets, and dedicated app screens.

[1317] Specific example

[1318] For example, suppose a user visits a cafe on the weekend and purchases coffee using a payment service. Furthermore, they post on social media saying they "want to relax" and recently searched for "recommended movies." In this case,

[1319] 1. The device records the user's location information, including visits to cafes, and obtains coffee purchase information from the payment history.

[1320] 2. The device also collects SNS posts and search terms, and sends all the information to the server.

[1321] 3. The server integrates and analyzes various pieces of information to estimate that the user needs to relax and that they like coffee.

[1322] 4. The server uses an emotion engine to more deeply analyze the user's desire to relax.

[1323] 5. The server generates a personalized message such as, "Why not relax today with a cup of coffee and enjoy a recommended movie?"

[1324] 6. The device will display this message to the user via push notification.

[1325] In this way, by providing messages that are tailored to the user's life circumstances and emotions, it is possible to increase user satisfaction and improve their quality of life. Through this system, users can more easily receive more personalized support.

[1326] The following describes the processing flow.

[1327] Step 1:

[1328] The device activates its GPS sensor and obtains the user's current location.

[1329] Step 2:

[1330] The device records location information it acquires and saves the places and times the user has visited.

[1331] Step 3:

[1332] The device sends recorded movement information to the server at regular intervals (e.g., every hour).

[1333] Step 4:

[1334] The user consents to the collection of their payment history.

[1335] Step 5:

[1336] The terminal calls the payment service's API to retrieve the user's payment history.

[1337] Step 6:

[1338] The device sends the acquired payment history to the server.

[1339] Step 7:

[1340] The device periodically collects the user's search history using the browser's API.

[1341] Step 8:

[1342] The device sends the collected search history to the server.

[1343] Step 9:

[1344] The user agrees to the use of the SNS API.

[1345] Step 10:

[1346] The device uses the SNS API to retrieve user posting information.

[1347] Step 11:

[1348] The terminal sends the written information it has acquired to the server.

[1349] Step 12:

[1350] The server integrates received movement information, payment history, search history, and writing information into a database.

[1351] Step 13:

[1352] The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[1353] Step 14:

[1354] The server updates the user profile based on the results of the information analysis.

[1355] Step 15:

[1356] The server uses an emotion engine to analyze the sentiment of user posts and search terms.

[1357] Step 16:

[1358] The server uses an emotion engine to update the user's emotional state in real time.

[1359] Step 17:

[1360] The server uses a generative model that generates multiple message candidates based on emotional state and user profile.

[1361] Step 18:

[1362] The server scores the relevance of each message candidate and selects the most appropriate message.

[1363] Step 19:

[1364] The server prepares to deliver selected messages to the user in a daily calendar format.

[1365] Step 20:

[1366] The device retrieves the latest message from the server at a set time (e.g., 8 AM).

[1367] Step 21:

[1368] The device displays messages to the user via push notifications, home screen widgets, and dedicated app screens.

[1369] (Example 2)

[1370] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1371] In recent years, with the advancement of information technology, personalized systems that provide optimal information to individual users have attracted attention. However, conventional systems have struggled to adequately analyze user behavior and emotions and deliver the most appropriate message at the right time. Furthermore, they lacked mechanisms for integrating data from multiple sources and reflecting users' emotional states in real time. Therefore, effectively delivering personalized messages to users has been a challenge.

[1372] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1373] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means for integrating the acquired information and analyzing the user's interests and emotional state using natural language processing and machine learning algorithms, means including a generative AI model for generating messages based on the user's interests and emotional state, means for scoring the generated messages and selecting the optimal message, and means for presenting the selected messages to the user in a daily calendar format. This makes it possible to enrich the user's life and provide optimal personalized messages based on their emotions and behavior.

[1374] "Movement information" refers to information about where a user has moved. Specifically, it refers to data on latitude, longitude, and travel route obtained by GPS sensors.

[1375] "Visit information" refers to information about a user's visit to a specific location. This includes the name, location, and date and time of the visit.

[1376] "Payment history" refers to a record of purchases and payments made by a user. It includes information such as the date and time of purchase, store name, and purchased items, which are obtained through payment service APIs.

[1377] "Search terms" refer to the search queries that users enter into internet search engines. This data is used to analyze users' interests and concerns.

[1378] "Posting" refers to posts and comments made by users on social networking services or other platforms.

[1379] "Natural language processing" is the technology that enables computers to understand, interpret, and generate human language. Specifically, it includes text analysis, sentiment analysis, and keyword extraction.

[1380] A "machine learning algorithm" is a technique of artificial intelligence that automatically learns regularities and patterns from data. This makes it possible to estimate a user's interests and emotions.

[1381] An "emotion engine" refers to specialized software or algorithms used to analyze emotions from text data. It plays a role in inferring emotional states from user posts and search terms.

[1382] A "generative AI model" is an artificial intelligence model that utilizes deep learning-based text generation technology to generate messages based on the user's profile and emotional state.

[1383] "Scoring" is the process of assigning scores to generated message candidates based on factors such as relevance and appropriateness.

[1384] An "optimal message" refers to a personalized message that is best suited to the user's current situation and emotional state.

[1385] The "daily calendar format" is a format that presents messages sequentially day by day, providing users with an experience of receiving new information on a regular basis.

[1386] A "location information system" is a system that uses GPS and other location-determining technologies to determine the user's current location.

[1387] A "social networking service" is a platform on the internet for users to create, share, and comment on content.

[1388] This invention is a system that provides personalized messages based on users' daily behaviors and interests in order to enrich their lives. This system collects and analyzes user movement information, payment history, search terms, and social networking service postings, and further recognizes the user's emotions using an emotion engine to generate individually optimized messages.

[1389] Collection of user information

[1390] Acquisition of travel and visit information

[1391] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. For example, it collects latitude and longitude information every 10 minutes.

[1392] The device sends recorded information about visited locations and travel routes to the server in a batch process every hour.

[1393] Retrieving payment history

[1394] The user gives consent to the collection of payment history on their device. For example, they might check a checkbox in the app's settings screen that says, "I agree to the collection of payment history."

[1395] After the device has given consent, it calls a payment service API (e.g., PayPal or Stripe) to retrieve the user's latest payment history.

[1396] The device sends the acquired payment history data to the server.

[1397] Retrieving search terms

[1398] The device, with the user's consent, uses a browser API (for example, a Chrome extension) to collect the user's search history at regular intervals (for example, daily).

[1399] The device sends the collected search terms to the server.

[1400] Retrieving writing information

[1401] The user agrees to the use of APIs from major social networking services (e.g., Facebook and Twitter).

[1402] After the device gives its consent, it calls the SNS API to retrieve user posting information. Specifically, it collects recent posts and comments.

[1403] The terminal sends the acquired write information to the server.

[1404] Data analysis and estimation by interest

[1405] The server stores the various data it receives in a centralized database (for example, MySQL or MongoDB).

[1406] The server applies a natural language processing (NLP) engine (e.g., spaCy or NLTK) to the integrated data to analyze user search terms and social media posts.

[1407] Next, the server uses statistical and machine learning models (e.g., scikit-learn or TensorFlow) to estimate the user's interests and preferences. For example, it predicts what actions the user will take in the future based on keywords such as "relax" or "movies."

[1408] Based on the analysis results, the server updates the user profile to include the user's interests, preferences, and emotional state.

[1409] Utilizing the Emotion Engine

[1410] The server uses an emotion engine (for example, IBM Watson's Sentiment Analysis) to analyze user sentiment from their posts and search terms. For example, it might infer the sentiment "I want to relax" from a post that says "I want to relax."

[1411] The server reflects the analyzed emotion data in the user profile in real time.

[1412] The server takes emotional states into consideration and generates personalized messages based on detailed analysis.

[1413] Generating personalized messages

[1414] The server uses a generative AI model (e.g., OpenAI GPT-3) to generate multiple message suggestions based on the user profile and emotional state. For example, it might generate a message such as, "Why not relax with a cup of coffee and enjoy a recommended movie today?"

[1415] Next, the server assigns relevance scores to the multiple message candidates that have been generated (for example, scores based on the user's current behavior and sentiment) and selects the most suitable message.

[1416] Presenting a message

[1417] The server stores the selected messages in a daily log format and prepares for the next presentation.

[1418] The device retrieves the latest personalized message from the server at a set time (for example, 8 AM).

[1419] The device displays messages it has received to the user via push notifications, home screen widgets, and a dedicated app screen. For example, it might display a message like, "Why not relax with a cup of coffee and enjoy a recommended movie today?" in the smartphone's notification bar.

[1420] Specific example

[1421] For example, a user visited a cafe on the weekend and purchased coffee using a payment service. Furthermore, they posted on social media that they wanted to "relax" and recently searched for "recommended movies." In this case,

[1422] 1. The device records the user's location information, including visits to cafes, and obtains coffee purchase information from the payment history.

[1423] 2. The device also collects SNS posts and search terms, and sends all the information to the server.

[1424] 3. The server integrates and analyzes various pieces of information to estimate that the user needs to relax and that they like coffee.

[1425] 4. The server uses an emotion engine to more deeply analyze the user's desire to relax.

[1426] 5. The server generates a personalized message such as, "Why not relax today with a cup of coffee and enjoy a recommended movie?"

[1427] 6. The device will display this message to the user via push notification.

[1428] In this way, by providing messages that are tailored to the user's life circumstances and emotions, it is possible to increase user satisfaction and improve their quality of life. Through this system, users can more easily receive more personalized support.

[1429] Example of a prompt

[1430] For example, a prompt statement to be input to a generative AI model can be written as follows:

[1431] "A user visited a cafe on the weekend and purchased coffee using a payment service. Furthermore, they posted on social media that they wanted to 'relax,' and recently searched for 'recommended movies.' Please generate a personalized message for this user."

[1432] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1433] Step 1:

[1434] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. For example, it acquires latitude and longitude information every 10 minutes and stores it in local memory.

[1435] Input: Program start command, GPS sensor data

[1436] Output: User's current location and visited locations information (latitude and longitude data)

[1437] Step 2:

[1438] The device sends recorded information about visited locations and travel routes to the server in a batch process every hour. Specifically, it sends the data to a message queue via an API.

[1439] Input: Recorded latitude and longitude data

[1440] Output: Data on visited locations and travel routes sent to the server.

[1441] Step 3:

[1442] The user gives consent to the collection of payment history on their device. For example, they might check a checkbox in the app's settings screen that says, "I agree to the collection of payment history."

[1443] Input: User consent action

[1444] Output: Consent information flags

[1445] Step 4:

[1446] After the device has given consent, it calls a payment service API to retrieve the user's latest payment history. For example, it might use the PayPal or Stripe API to retrieve transaction data.

[1447] Input: Consent information flag

[1448] Output: Payment history data

[1449] Step 5:

[1450] The device sends the acquired payment history data to the server. Specifically, it sends the data to the server via an HTTP request.

[1451] Input: Payment history data

[1452] Output: Payment history data sent to the server

[1453] Step 6:

[1454] With the user's consent, the device uses the browser API to collect the user's search history at regular intervals (for example, daily). Specifically, it uses a Chrome extension to collect search queries.

[1455] Input: User consent information, browser search data

[1456] Output: Collected search terms

[1457] Step 7:

[1458] The device sends the collected search terms to the server. For example, it might send them in JSON format via an HTTP request.

[1459] Input: Collected search terms

[1460] Output: Search words sent to the server

[1461] Step 8:

[1462] The user agrees to the use of APIs from major social networking services (SNS). Specifically, they check a checkbox in the app's settings screen that says something like, "I agree to the use of SNS APIs."

[1463] Input: User consent action

[1464] Output: Consent information flags

[1465] Step 9:

[1466] After the device gives its consent, it calls the SNS API to retrieve user posting information. For example, it uses the Facebook or Twitter API to retrieve recent posts and comments.

[1467] Input: Consent information flag

[1468] Output: SNS posting information

[1469] Step 10:

[1470] The device sends the acquired SNS posting information to the server. Specifically, it sends the data to the server via an HTTP request.

[1471] Input: Social media posting information

[1472] Output: SNS posting information sent to the server

[1473] Step 11:

[1474] The server stores various data it receives in a centralized database. For example, MySQL or MongoDB can be used to store the data.

[1475] Input: Visit location data, payment history data, search terms, social media postings

[1476] Output: Various data stored in the database

[1477] Step 12:

[1478] The server applies a natural language processing (NLP) engine to the integrated data to analyze user information. For example, it might use spaCy or NLTK to analyze text data and extract keywords and sentiments.

[1479] Input: Integrated data stored in the database

[1480] Output: Analyzed user interests and emotional states

[1481] Step 13:

[1482] The server uses statistical and machine learning models to estimate user interests and preferences. For example, it can use scikit-learn or TensorFlow to train data and generate an interest prediction model.

[1483] Input: Analyzed user interests and emotional states

[1484] Output: Estimated user interests and emotional states

[1485] Step 14:

[1486] The server uses a generative AI model to generate multiple message candidates based on the user profile and emotional state. For example, it can use OpenAI GPT-3 to generate personalized messages for the user.

[1487] Input: Estimated user interests and emotional state

[1488] Output: List of message candidates

[1489] Step 15:

[1490] The server assigns a relevance score to the generated message candidates and selects the most suitable message. Specifically, it uses a scoring algorithm to evaluate the relevance of each message.

[1491] Input: List of message suggestions

[1492] Output: Optimal message

[1493] Step 16:

[1494] The server stores the selected messages in a daily log format and prepares for the next presentation.

[1495] Input: Optimal message

[1496] Output: Messages stored in a daily calendar format

[1497] Step 17:

[1498] The device retrieves the latest personalized message from the server at a set time (for example, 8 AM). Specifically, it periodically sends requests to the server via an API.

[1499] Input: Time setting

[1500] Output: Latest message retrieved

[1501] Step 18:

[1502] The device displays messages it has received to the user via push notifications, home screen widgets, and dedicated app screens. For example, it displays messages in the smartphone's notification bar.

[1503] Input: Latest message retrieved

[1504] Output: Message displayed to the user

[1505] (Application Example 2)

[1506] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1507] In modern factories, improving operator efficiency is a critical challenge. However, because individualized support is required, taking into account each operator's work history and current emotional state, general assistance and advice have limited effectiveness. Furthermore, operators themselves may experience stress or a lack of job satisfaction, which can lead to decreased productivity. Therefore, a system is needed that can grasp each operator's individual situation and emotions in real time and provide personalized messages based on that information.

[1508] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1509] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means for integrating the acquired information and performing analysis according to the user's interests, means for analyzing the user's emotions using an emotion engine, means for generating personalized messages based on the analysis results and emotional state, and means for presenting the generated personalized messages to the user. This makes it possible for robots deployed in a factory to generate and present personalized messages based on the operator's past work history, the situation during work, and information acquired from social networking services.

[1510] "Movement information" refers to the movement history and location data of users and operators, including where they have moved to.

[1511] "Visit information" refers to records of when users or operators visited a specific location and data on the time they spent at that location.

[1512] "Payment history" refers to records of transactions made by users or operators, as well as data on what goods or services were paid for.

[1513] "Search terms" refer to keywords or phrases that users or operators enter into internet search engines.

[1514] "Posts" refer to text or comments that users or operators post on social networking services or other platforms.

[1515] An "emotion engine" refers to an algorithm or system that analyzes emotions and emotional states from user or operator text data.

[1516] "Personalized messages" refer to messages and notifications that are customized based on the individual data of the user or operator.

[1517] A "robot" refers to a machine or device used in factories or other work environments to perform automated tasks.

[1518] An "operator" refers to a human worker who is responsible for operating machinery or systems in a factory or other work environment.

[1519] This invention provides a system that delivers personalized messages based on individual circumstances and emotional states to improve the operational efficiency of operators in a factory. This system collects and analyzes user movement information, payment history, search terms, and writing information, and uses an emotion engine to recognize emotional states, thereby generating and presenting individually optimized messages.

[1520] First, the system program uses GPS devices to acquire operator movement and visit information. For example, it utilizes a location information system such as the Garmin GLO 2. Next, to obtain payment history, the PaymentService API is used to collect payment information for goods and services made by operators within the factory. In addition, the Browser History API is used to obtain keywords searched by operators on the internet. Furthermore, APIs of major social networking services (SNS) are used to collect operator posting information. All of this data is sent to the server.

[1521] The server first integrates the collected data and uses natural language processing (NLP) and machine learning algorithms to analyze the operator's interests, concerns, and emotional state. Next, it uses an emotion engine (e.g., Google Cloud Natural Language API) to analyze the operator's emotional state in detail. Based on this analysis and the emotional state, a generative AI model (e.g., GPT-3 model) is used to generate personalized messages. These messages are created considering the operator's past work history, current work situation, and social media posts.

[1522] The generated personalized messages are presented to operators through robots placed in the factory. Presentation methods include the robot's display, voice output, or a dedicated application. For example, if an operator posts on social media that their work efficiency is low and is searching for "tips for improving productivity," the AI ​​model might generate a message such as "Take more breaks today and relax," and the robot would then present that message.

[1523] (Specific example)

[1524] Operator A posted on social media that "work efficiency is declining." They also searched for "tips for improving productivity" in their browser.

[1525] Movement information: The GPS device tracked Operator A's movements within the factory.

[1526] Payment history: Purchased an energy drink from a vending machine inside the factory.

[1527] Search terms: "Tips for improving productivity"

[1528] Social media post: "Work efficiency is declining."

[1529] (Example of a prompt message)

[1530] "Operator A has posted on social media about a decline in work efficiency. Their search history also shows they are looking for information on improving productivity. Based on this information, we will provide refreshing advice and personalized messages."

[1531] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1532] Step 1:

[1533] The terminal periodically records the operator's movement and visit information using a GPS device. Specifically, it acquires location data from a GPS device (e.g., Garmin GLO 2) and stores this data in its internal memory. In this step, the input is the location information from the GPS device, and the output is the recorded movement information.

[1534] Step 2:

[1535] The terminal periodically retrieves payment history, so it calls the PaymentService API to collect payment data. It retrieves transaction information from vending machines and shops. The inputs in this step are the user ID and payment event, and the output is payment history data.

[1536] Step 3:

[1537] The device periodically collects the user's search terms using the browser history API. This is a list of keywords and phrases the user has recently searched for. The input for this step is browser history data, and the output is a list of search terms.

[1538] Step 4:

[1539] The device uses the SNS API to retrieve user posting information. It periodically collects SNS posts and sends them to the server. The input in this step is the posted data obtained via the SNS API, and the output is the posting information.

[1540] Step 5:

[1541] The terminal sends all collected information (movement information, payment history, search terms, and posting information) to the server. The server integrates this information and stores it in a database. The input in this step is the user data sent from the terminal, and the output is the integrated dataset.

[1542] Step 6:

[1543] The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state. This process identifies the user's current state and trends based on the collected data. The input in this step is an integrated dataset, and the output is the analyzed user profile.

[1544] Step 7:

[1545] The server uses an emotion engine (e.g., Google Cloud Natural Language API) to analyze the user's emotional state. It analyzes collected writing information and search terms to identify the user's emotions. The input in this step is the user's text data, and the output is emotional state data.

[1546] Step 8:

[1547] The server uses a generated AI model (e.g., a GPT-3 model) to generate personalized messages based on the analysis results and emotional state. It generates the optimal message tailored to the user's profile. The input in this step is the analyzed user profile and emotional state data, and the output is the personalized message.

[1548] Step 9:

[1549] The terminal retrieves a personalized message generated from the server and presents it to the operator via the display or voice output of a robot placed in the factory. In this step, the input is the personalized message, and the output is a notification to the operator.

[1550] Step 10:

[1551] The user acts based on the personalized message presented to them, aiming to improve efficiency and reduce stress. In this step, the input is the personalized message, and the output is the user's action.

[1552] By following these steps, it is possible to understand the individual circumstances and emotions of users in real time and provide optimal messages based on that understanding, thereby improving operational efficiency in the factory.

[1553] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1554] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1555] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1556] [Fourth Embodiment]

[1557] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1558] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1559] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1560] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1561] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1562] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1563] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1564] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1565] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1566] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1567] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1568] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1569] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1570] This invention aims to enrich users' lives by providing personalized messages based on their daily behavior and interests. This system comprehensively analyzes users' movement information, payment history, search terms, and social networking service postings to generate individually optimized messages.

[1571] Collection of user information

[1572] Acquisition of travel and visit information

[1573] 1. The device periodically records the user's current location and visited locations using a GPS sensor.

[1574] 2. The device sends recorded information about visited locations and travel routes to the server at appropriate intervals, such as every hour.

[1575] Retrieving payment history

[1576] 1. After the user gives consent to retrieve payment history, the device calls a payment service API such as PayPay.

[1577] 2. The device retrieves the payment history and sends it to the server.

[1578] Retrieving search terms

[1579] 1. The device periodically collects search history via the browser API with the user's consent.

[1580] 2. The device sends the collected search terms to the server.

[1581] Retrieving writing information

[1582] 1. After the user agrees to use the SNS API, the device retrieves the user's posting information from major SNS platforms.

[1583] 2. The device sends the acquired SNS posting information to the server.

[1584] Data analysis and estimation by interest

[1585] 3. The server integrates all received data into the database.

[1586] 4. The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[1587] 5. The server updates the user profile based on the analysis results.

[1588] Generating personalized messages

[1589] 6. Use a generative model in which the server generates multiple message candidates based on the user profile.

[1590] 7. The server scores the relevance of each message candidate and selects the most appropriate message.

[1591] 8. The server prepares to deliver the selected messages to the user in a daily calendar format.

[1592] Presenting a message

[1593] 9. The device retrieves the message from the server at the scheduled time.

[1594] 10. The device notifies the user of the message. Possible notification methods include push notifications, home screen widgets, and dedicated app screens.

[1595] Specific example

[1596] For example, suppose a user visits a cafe on the weekend and purchases coffee using PayPay. Also, suppose that user has posted on social media that they "want to relax" and recently searched for "recommended movies." In this case,

[1597] 1. The device records the user's location information, including visits to cafes, and obtains coffee purchase information from the payment history.

[1598] 2. The device also collects SNS posts and search terms, and sends all the information to the server.

[1599] 3. The server integrates and analyzes this information to estimate that the user needs to relax and that they like coffee.

[1600] 4. The server generates a message saying, "Why not relax today, enjoy a cup of coffee, and watch a recommended movie?"

[1601] 5. The device displays this message to the user via push notification.

[1602] In this way, we can provide messages that are tailored to the user's lifestyle and interests. Through this entire system, we can increase user satisfaction and improve their quality of life.

[1603] The following describes the processing flow.

[1604] Step 1:

[1605] The device activates its GPS sensor and obtains the user's current location.

[1606] Step 2:

[1607] The device records location information it acquires and saves the places and times the user has visited.

[1608] Step 3:

[1609] The device sends recorded movement information to the server at regular intervals (e.g., every hour).

[1610] Step 4:

[1611] The user consents to the collection of their payment history.

[1612] Step 5:

[1613] The terminal calls the payment service's API to retrieve the user's payment history.

[1614] Step 6:

[1615] The device sends the acquired payment history to the server.

[1616] Step 7:

[1617] The device periodically collects the user's search history using the browser's API.

[1618] Step 8:

[1619] The device sends the collected search history to the server.

[1620] Step 9:

[1621] The user agrees to the use of the SNS API.

[1622] Step 10:

[1623] The device uses the SNS API to retrieve user posting information.

[1624] Step 11:

[1625] The terminal sends the written information it has acquired to the server.

[1626] Step 12:

[1627] The server integrates received movement information, payment history, search history, and writing information into a database.

[1628] Step 13:

[1629] The server applies natural language processing (NLP) and machine learning models to analyze the user's interests, concerns, and emotional state.

[1630] Step 14:

[1631] The server updates the user profile based on the analysis results.

[1632] Step 15:

[1633] The server generates multiple message candidates using a generative model based on the user profile.

[1634] Step 16:

[1635] The server scores the relevance of each message candidate and selects the most appropriate message.

[1636] Step 17:

[1637] The server prepares selected messages for the user in a daily calendar format.

[1638] Step 18:

[1639] The device retrieves the latest message from the server at a set time (e.g., 8 AM).

[1640] Step 19:

[1641] The device displays the message to the user via push notifications, home screen widgets, or a dedicated app screen.

[1642] (Example 1)

[1643] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1644] In modern society, providing personalized messages based on individual interests and behaviors is crucial for enriching users' lives. However, no system exists that effectively integrates diverse user information (such as travel data, payment history, search terms, and social networking service posts) to generate appropriate messages. Therefore, it is necessary to address the problems of information overload that users experience daily and the resulting lack of appropriate information.

[1645] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1646] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means for integrating the acquired information and performing analysis according to the user's interests, means for updating the user profile, means for applying natural language processing and machine learning algorithms, means for generating message candidates using a generative AI model, means for scoring the generated message candidates and selecting the optimal message, and notification means for presenting the selected message to the user. This enables the effective integration and analysis of diverse user information and the provision of individually optimized personalized messages.

[1647] "Movement information" refers to data about a user's location and places they have visited, and is primarily obtained using location information systems such as GPS sensors.

[1648] "Visit information" refers to detailed data about a user's visit to a specific location, including the date and time of the visit and the name of the location.

[1649] "Payment history" refers to data about financial transactions and purchase behaviors performed by a user, and includes information such as the date and time of payment, amount, and place of purchase, obtained through payment service APIs.

[1650] "Search terms" are keywords or phrases entered by users during internet searches, and are collected via browser APIs and other means.

[1651] "Posts" refer to content or comments that users submit to social networking services or other online platforms.

[1652] "Integration" is the process of centrally combining different types of data and converting them into a format that is easy to store in a database or similar system.

[1653] "Interest-based analysis" is a method that uses natural language processing and machine learning algorithms to analyze users' interests, concerns, and emotional states from collected data.

[1654] A "user profile" is a database description created based on a user's behavior, interests, and concerns, and includes individual characteristics and tendencies.

[1655] "Natural language processing" is a computer science technique for understanding and analyzing human language, and it includes tasks such as sentiment analysis of text and keyword extraction.

[1656] A "machine learning algorithm" is a computer algorithm that learns patterns and knowledge from data, and is used for clustering, classification, prediction, and other purposes.

[1657] A "generative AI model" is an artificial intelligence model that generates sentences and texts based on large amounts of data, and is a method for creating new messages and recommendations.

[1658] "Scoring" is the process of evaluating generated message candidates and quantifying their relevance and relevance.

[1659] The "optimal message" is the message that best matches the user's profile and analysis results, and provides value to them.

[1660] "Notifications" are methods for conveying information to users based on specific times or events, and include push notifications and widgets.

[1661] This invention is a system that provides personalized messages based on daily behaviors and interests, with the aim of enriching users' lives. This system collects user information, analyzes data and estimates interests, generates personalized messages, and presents those messages. Its detailed configuration is described below.

[1662] Collection of user information

[1663] 1. Obtaining travel and visit information:

[1664] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. This utilizes the GPS function of the smartphone.

[1665] The device transmits recorded information to the server via a communication module. Mobile communication networks or Wi-Fi are used as the means of communication.

[1666] 2. Obtaining payment history:

[1667] When a user consents to the retrieval of their payment history, the device calls the payment service API to retrieve the payment history. The retrieved information includes the payment date and time, amount, and store name.

[1668] The device temporarily saves the acquired payment history to local storage and then sends it to the server. For example, it may be updated periodically.

[1669] 3. Obtaining search terms:

[1670] The device periodically collects the user's search terms via the browser API. This process uses APIs such as Google Chrome's history API to extract search keywords every 24 hours.

[1671] The terminal sends the extracted search words to the server, and the transmitted data includes the search date and time and the search query.

[1672] 4. Obtaining writing information:

[1673] After a user consents to the use of the SNS API, the device retrieves posting information from social networking services. Specifically, this includes the content of the post, the date and time of posting, and the number of likes and comments.

[1674] The device sends the acquired SNS posting information to the server.

[1675] Data analysis and estimation by interest

[1676] The server integrates all received data into a database. Databases such as MySQL or PostgreSQL are used.

[1677] The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state. Here, we use spaCy, a Python NLP library, to perform sentiment analysis on text, and scikit-learn to cluster the user's interests.

[1678] The server updates the user profile based on the analysis results. The user profile includes estimated user interests, frequently visited locations, and typical consumption patterns.

[1679] Generating personalized messages

[1680] The server uses a generative AI model to generate multiple message suggestions based on an updated user profile. Specifically, it uses the OpenAI API's GPT-3 to input a prompt message such as, "The user bought coffee at a cafe, posted on social media that they want to relax, and are looking for recently recommended movies. Please generate relevant personalized messages."

[1681] The server scores the relevance of each candidate message and selects the optimal message. Methods such as TF-IDF and Doc2Vec are used for relevance scoring.

[1682] The server converts the selected messages into a daily calendar format and prepares to deliver them to the user.

[1683] Presenting a message

[1684] The device retrieves messages from the server at a set time. For example, you can set it to retrieve new messages every morning at 7:00 AM.

[1685] The device notifies the user of a message. Notification methods include smartphone push notifications, widgets placed on the home screen, or dedicated app screens. For example, a push notification might display a message such as, "Why not relax with a cup of coffee today and enjoy a recommended movie?"

[1686] This configuration enables the effective integration and analysis of diverse user information, allowing for the delivery of individually optimized, personalized messages. Through this entire system, user satisfaction can be enhanced, and their quality of life can be improved.

[1687] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1688] Step 1:

[1689] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. The input is location information from the GPS sensor, and the output is the recorded location data. For example, it might use the smartphone's GPS function to acquire location information every 5 minutes.

[1690] Step 2:

[1691] The device transmits recorded movement information to a server via a communication module. The input is the recorded location data, and the output is the data transmitted to the server. For example, it might perform an operation that transmits location information via Wi-Fi every hour.

[1692] Step 3:

[1693] After the user consents to the retrieval of their payment history, the device calls the payment service API to retrieve the payment history. The input is a request to the payment service API, and the output is payment history data. Specifically, it retrieves information such as the payment date and time, amount, and store name.

[1694] Step 4:

[1695] The device first saves the acquired payment history to local storage and then sends it to the server. The input is the payment history data, and the output is the data sent to the server. For example, it can perform an action to send data at a set time.

[1696] Step 5:

[1697] The device periodically collects the user's search terms via the browser API. The input is browser history data, and the output is a list of search terms. For example, the Google Chrome history API is used to extract search keywords every 24 hours.

[1698] Step 6:

[1699] The terminal extracts search terms and sends them to the server. The input is a list of search terms, and the output is the data sent to the server. For example, it can perform an operation to periodically upload collected search terms to the server.

[1700] Step 7:

[1701] After the user consents to the use of the SNS API, the device retrieves posting information from the social networking service. The input is an SNS API request, and the output is the posting information. Specifically, it retrieves the post content, posting date and time, number of likes and comments, etc.

[1702] Step 8:

[1703] The device sends the acquired SNS posting information to the server. The input is the posting information, and the output is the data sent to the server. For example, the device might collect data every hour and send it to the server.

[1704] Step 9:

[1705] The server integrates all received data into a database. The input consists of various data sent to the server, and the output is the integrated database entry. MySQL or PostgreSQL can be used as the database.

[1706] Step 10:

[1707] The server applies natural language processing and machine learning algorithms to analyze the user's interests, concerns, and emotional state. The input is an integrated database entry, and the output is the analysis result. Specifically, it uses Python's spaCy to perform sentiment analysis on text and scikit-learn to cluster user interests.

[1708] Step 11:

[1709] The server updates the user profile based on the analysis results. The input is the analysis results, and the output is the updated user profile. The user profile includes estimated user interests, frequently visited locations, and typical consumption patterns.

[1710] Step 12:

[1711] The server uses a generative AI model to generate multiple message candidates based on an updated user profile. The input is the user profile, and the output is the message candidates. It uses the OpenAI API's GPT-3 to input prompts and generate messages.

[1712] Example prompt: "A user has bought coffee at a cafe, posted on social media that they want to relax, and is looking for recently recommended movies. Generate a relevant personalized message."

[1713] Step 13:

[1714] The server scores the relevance of each message candidate and selects the optimal message. The input is the message candidates, and the output is the selected optimal message. Methods such as TF-IDF and Doc2Vec are used for relevance scoring.

[1715] Step 14:

[1716] The server converts the selected messages into a daily calendar format and prepares them for delivery to the user. The input is the selected messages, and the output is the messages ready for delivery.

[1717] Step 15:

[1718] The device retrieves messages from the server at a set time. The input is the message ready for delivery on the server, and the output is the message retrieved by the device. For example, you can set it to retrieve a new message every morning at 7:00 AM.

[1719] Step 16:

[1720] The device notifies the user of a message. The input is a message acquired within the device, and the output is the notified message. Possible notification methods include smartphone push notifications, widgets placed on the home screen, or a dedicated app screen. As a specific example, a push notification might display a message such as, "Why not relax with a cup of coffee today and enjoy a recommended movie?"

[1721] (Application Example 1)

[1722] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1723] Traditional personalized messaging systems have limited ability to accurately analyze user behavior and interests and provide content that matches the user's lifestyle. Furthermore, there is a need for more sophisticated methods to analyze user interests and preferences and recommend the most suitable content based on that analysis.

[1724] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1725] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means including natural language processing and machine learning algorithms for integrating the acquired information and performing analysis according to the user's interests, and means for generating prompt sentences using a generative AI model that recommends content according to the user's interests and providing optimal content. This makes it possible to precisely analyze the user's behavior information and interests and provide optimal content as a personalized message.

[1726] "Movement information" refers to data about the user's current location and places visited, obtained using GPS or other location information systems.

[1727] "Visit information" refers to the history and detailed information of a user's visits to specific locations, and is data recorded by location information systems.

[1728] "Payment history" refers to a record of a user's purchasing activities, which is obtained through the API of an electronic payment service.

[1729] "Search terms" are keywords or phrases that users search for on internet search engines, and are collected via browser APIs.

[1730] "Posts" refer to information such as posts and comments made by users on social networking services, and are obtained via the APIs of each SNS.

[1731] "Integration" refers to the process of unifying information obtained from multiple different data sources and treating it as a single entity for analysis.

[1732] "Interest-based analysis" is the process of analyzing users' interests, preferences, and emotional states based on acquired data, and creating user profiles.

[1733] "Natural language processing" is a technology that enables computers to understand, analyze, and generate human language, and is used to understand text data such as user posts and search terms.

[1734] A "machine learning algorithm" is a computational method used to learn from large amounts of data and perform predictions and classifications, and is used to analyze user interests and preferences.

[1735] A "generative AI model" is a model that uses artificial intelligence technology to generate appropriate output (e.g., message or content recommendation) from input data.

[1736] A "prompt" is the input text given to a generative AI model and is used to control the model's output.

[1737] "Content" is a general term for information and entertainment provided to users, including movies, video clips, music, and ebooks.

[1738] This invention is a system that provides personalized messages based on users' daily behaviors and interests in order to enrich their lives. This system comprehensively analyzes users' movement information, payment history, search terms, and social networking service postings to generate and deliver individually optimized messages and content to the user.

[1739] Hardware and software usage:

[1740] Device: Smartphones have a built-in GPS sensor that records location information.

[1741] Software: Utilizes various APIs (payment service APIs, browser APIs, SNS APIs) for data collection.

[1742] Server: High-performance computers and specialized software (such as TensorFlow and OpenAI GPT-3) are used to process and analyze large amounts of data.

[1743] Data collection and analysis:

[1744] 1. Obtaining travel and visit information:

[1745] The device periodically records the user's current location and visited places using a GPS sensor and transmits this information to the server.

[1746] 2. Obtaining payment history:

[1747] After obtaining the user's consent, the device calls the payment service API to retrieve the user's payment history and sends it to the server.

[1748] 3. Obtaining search terms:

[1749] The system periodically collects users' search history via the browser API and sends it to the server.

[1750] 4. Obtaining writing information:

[1751] After obtaining permission to use the SNS API, the device retrieves user posting information from major SNS platforms and sends it to the server.

[1752] 5. Data Analysis:

[1753] The server integrates all received data into a database and applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[1754] Content generation and delivery:

[1755] 1. Update your user profile:

[1756] The server updates the user profile based on the analysis results.

[1757] 2. Generating personalized messages:

[1758] The server generates multiple message candidates using a generative AI model based on the user profile.

[1759] The relevance of each message candidate is scored, and the most suitable message is selected.

[1760] 3. Presenting the message:

[1761] The device retrieves a message from the server at a set time and notifies the user. Possible notification methods include push notifications, home screen widgets, and dedicated app screens.

[1762] Examples of specific cases and prompt statements:

[1763] For example, suppose a user visits a cafe on the weekend and buys a coffee. Also, consider a scenario where this user posts on a social networking service that they "want to relax" and recently searched for "recommended movies." In this case,

[1764] The server integrates and analyzes this information to estimate that "the user needs to relax" and "likes coffee."

[1765] Based on this, enter the following prompt into the generative AI model:

[1766] "User interest: Relaxation, Coffee, Movies. Generate a message."

[1767] The generated message will be a push notification to the user suggesting, "Why not relax with a cup of coffee today and enjoy a recommended movie?"

[1768] In this way, we can provide messages that are tailored to the user's lifestyle and interests, thereby increasing user satisfaction and improving their quality of life.

[1769] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1770] Step 1:

[1771] The device periodically records the user's current location and visited locations using a GPS sensor. Specifically, the device acquires location information at regular intervals and stores it as location data (latitude, longitude, and timestamp). The input is GPS sensor data, and the output is location data.

[1772] Step 2:

[1773] The device transmits recorded information about visited locations and travel routes to the server at appropriate intervals, such as every hour. The input is the location data acquired in step 1, and the output is the data transmitted to the server.

[1774] Step 3:

[1775] After the user gives consent to retrieve payment history, the device calls the payment service API to retrieve the payment history. Specifically, the device sends a request to the API endpoint and receives payment history data (purchase date and time, place of purchase, purchase amount) as a response. The input is the user's consent and the API request, and the output is the payment history data.

[1776] Step 4:

[1777] The terminal sends the acquired payment history data to the server. The input is the payment history data acquired in step 3, and the output is the data sent to the server.

[1778] Step 5:

[1779] After obtaining the user's consent to collect their search history, the device periodically collects the search history via the browser API. Specifically, the device sends a request to the browser API and stores the received search terms and search date and time. The input is the user's consent and the API request, and the output is the search history data.

[1780] Step 6:

[1781] The terminal sends the collected search history to the server. The input is the search history data obtained in step 5, and the output is the data sent to the server.

[1782] Step 7:

[1783] After the user gives their consent to use the SNS API, the device retrieves the user's posting information from major SNS platforms. Specifically, the device sends a request to the SNS API and receives the posting information (post content, posting date and time) as a response. The input is the user's consent and the API request, and the output is the posting information data.

[1784] Step 8:

[1785] The terminal sends the acquired write information to the server. The input is the write information data acquired in step 7, and the output is the data sent to the server.

[1786] Step 9:

[1787] The server integrates all received data (location information, payment history, search history, and writing information) into a database. Input is various data sent from the terminal, and output is the integrated database.

[1788] Step 10:

[1789] The server applies natural language processing (NLP) and machine learning algorithms to an integrated database to analyze users' interests, concerns, and emotional states. Specifically, it uses NLP to analyze text data and machine learning models to extract user behavior patterns. The input is data from the integrated database, and the output is the analysis results.

[1790] Step 11:

[1791] The server updates the user profile based on the analysis results. Specifically, it adds or modifies the user's interest categories and real-time sentiment indicators to the profile. The input is the analysis results, and the output is the updated user profile.

[1792] Step 12:

[1793] The server generates multiple message candidates using a generative AI model based on the user profile. The input is the updated user profile, and the output is the message candidates. As a concrete example, the AI ​​model is given the following prompt: "User interest: Relaxation, Coffee, Movies. Generate a message."

[1794] Step 13:

[1795] The server scores the relevance of each message candidate and selects the optimal message. The input is message candidates generated by the AI ​​model, and the output is the selected optimal message.

[1796] Step 14:

[1797] The device retrieves the most appropriate message from the server at a set time and notifies the user. Specific notification methods include push notifications, home screen widgets, and dedicated app screens. The input is the optimal message sent from the server, and the output is the notification to the user.

[1798] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1799] This invention is a system that provides personalized messages based on users' daily behaviors and interests in order to enrich their lives. This system collects and analyzes user movement information, payment history, search terms, and social networking service postings, and further recognizes the user's emotions using an emotion engine to generate individually optimized messages.

[1800] Collection of user information

[1801] Acquisition of travel and visit information

[1802] 1. The device periodically records the user's current location and visited locations using a GPS sensor.

[1803] 2. The device sends recorded information about visited locations and travel routes to the server at appropriate intervals, such as every hour.

[1804] Retrieving payment history

[1805] 1. After the user gives consent to retrieve payment history, the device calls the payment service API.

[1806] 2. The device retrieves the payment history and sends it to the server.

[1807] Retrieving search terms

[1808] 1. The device periodically collects search history via the browser API with the user's consent.

[1809] 2. The device sends the collected search terms to the server.

[1810] Retrieving writing information

[1811] 1. After the user agrees to use the SNS API, the device retrieves the user's posting information from major SNS platforms.

[1812] 2. The device sends the acquired SNS posting information to the server.

[1813] Data analysis and estimation by interest

[1814] 1. The server integrates the various data it receives into the database.

[1815] 2. The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[1816] 3. The server updates the user profile based on the analysis results.

[1817] Utilizing the Emotion Engine

[1818] 1. The server uses an emotion engine to analyze the sentiment of user posts and search terms.

[1819] 2. The server uses an emotion engine to update the user's emotional state in real time.

[1820] 3. The server takes emotional states into account to generate even more accurate messages.

[1821] Generating personalized messages

[1822] 1. The server uses a generative model that generates multiple message candidates based on the user profile and emotional state.

[1823] 2. The server scores the relevance of each message candidate and selects the most appropriate message.

[1824] 3. The server prepares to deliver the selected messages to the user in a daily calendar format.

[1825] Presenting a message

[1826] 1. The device retrieves the latest message from the server at the scheduled time.

[1827] 2. The device displays messages to the user via push notifications, home screen widgets, and dedicated app screens.

[1828] Specific example

[1829] For example, suppose a user visits a cafe on the weekend and purchases coffee using a payment service. Furthermore, they post on social media saying they "want to relax" and recently searched for "recommended movies." In this case,

[1830] 1. The device records the user's location information, including visits to cafes, and obtains coffee purchase information from the payment history.

[1831] 2. The device also collects SNS posts and search terms, and sends all the information to the server.

[1832] 3. The server integrates and analyzes various pieces of information to estimate that the user needs to relax and that they like coffee.

[1833] 4. The server uses an emotion engine to more deeply analyze the user's desire to relax.

[1834] 5. The server generates a personalized message such as, "Why not relax today with a cup of coffee and enjoy a recommended movie?"

[1835] 6. The device will display this message to the user via push notification.

[1836] In this way, by providing messages that are tailored to the user's life circumstances and emotions, it is possible to increase user satisfaction and improve their quality of life. Through this system, users can more easily receive more personalized support.

[1837] The following describes the processing flow.

[1838] Step 1:

[1839] The device activates its GPS sensor and obtains the user's current location.

[1840] Step 2:

[1841] The device records location information it acquires and saves the places and times the user has visited.

[1842] Step 3:

[1843] The device sends recorded movement information to the server at regular intervals (e.g., every hour).

[1844] Step 4:

[1845] The user consents to the collection of their payment history.

[1846] Step 5:

[1847] The terminal calls the payment service's API to retrieve the user's payment history.

[1848] Step 6:

[1849] The device sends the acquired payment history to the server.

[1850] Step 7:

[1851] The device periodically collects the user's search history using the browser's API.

[1852] Step 8:

[1853] The device sends the collected search history to the server.

[1854] Step 9:

[1855] The user agrees to the use of the SNS API.

[1856] Step 10:

[1857] The device uses the SNS API to retrieve user posting information.

[1858] Step 11:

[1859] The terminal sends the written information it has acquired to the server.

[1860] Step 12:

[1861] The server integrates received movement information, payment history, search history, and writing information into a database.

[1862] Step 13:

[1863] The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state.

[1864] Step 14:

[1865] The server updates the user profile based on the results of the information analysis.

[1866] Step 15:

[1867] The server uses an emotion engine to analyze the sentiment of user posts and search terms.

[1868] Step 16:

[1869] The server uses an emotion engine to update the user's emotional state in real time.

[1870] Step 17:

[1871] The server uses a generative model that generates multiple message candidates based on emotional state and user profile.

[1872] Step 18:

[1873] The server scores the relevance of each message candidate and selects the most appropriate message.

[1874] Step 19:

[1875] The server prepares to deliver selected messages to the user in a daily calendar format.

[1876] Step 20:

[1877] The device retrieves the latest message from the server at a set time (e.g., 8 AM).

[1878] Step 21:

[1879] The device displays messages to the user via push notifications, home screen widgets, and dedicated app screens.

[1880] (Example 2)

[1881] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1882] In recent years, with the advancement of information technology, personalized systems that provide optimal information to individual users have attracted attention. However, conventional systems have struggled to adequately analyze user behavior and emotions and deliver the most appropriate message at the right time. Furthermore, they lacked mechanisms for integrating data from multiple sources and reflecting users' emotional states in real time. Therefore, effectively delivering personalized messages to users has been a challenge.

[1883] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1884] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means for integrating the acquired information and analyzing the user's interests and emotional state using natural language processing and machine learning algorithms, means including a generative AI model for generating messages based on the user's interests and emotional state, means for scoring the generated messages and selecting the optimal message, and means for presenting the selected messages to the user in a daily calendar format. This makes it possible to enrich the user's life and provide optimal personalized messages based on their emotions and behavior.

[1885] "Movement information" refers to information about where a user has moved. Specifically, it refers to data on latitude, longitude, and travel route obtained by GPS sensors.

[1886] "Visit information" refers to information about a user's visit to a specific location. This includes the name, location, and date and time of the visit.

[1887] "Payment history" refers to a record of purchases and payments made by a user. It includes information such as the date and time of purchase, store name, and purchased items, which are obtained through payment service APIs.

[1888] "Search terms" refer to the search queries that users enter into internet search engines. This data is used to analyze users' interests and concerns.

[1889] "Posting" refers to posts and comments made by users on social networking services or other platforms.

[1890] "Natural language processing" is the technology that enables computers to understand, interpret, and generate human language. Specifically, it includes text analysis, sentiment analysis, and keyword extraction.

[1891] A "machine learning algorithm" is a technique of artificial intelligence that automatically learns regularities and patterns from data. This makes it possible to estimate a user's interests and emotions.

[1892] An "emotion engine" refers to specialized software or algorithms used to analyze emotions from text data. It plays a role in inferring emotional states from user posts and search terms.

[1893] A "generative AI model" is an artificial intelligence model that utilizes deep learning-based text generation technology to generate messages based on the user's profile and emotional state.

[1894] "Scoring" is the process of assigning scores to generated message candidates based on factors such as relevance and appropriateness.

[1895] An "optimal message" refers to a personalized message that is best suited to the user's current situation and emotional state.

[1896] The "daily calendar format" is a format that presents messages sequentially day by day, providing users with an experience of receiving new information on a regular basis.

[1897] A "location information system" is a system that uses GPS and other location-determining technologies to determine the user's current location.

[1898] A "social networking service" is a platform on the internet for users to create, share, and comment on content.

[1899] This invention is a system that provides personalized messages based on users' daily behaviors and interests in order to enrich their lives. This system collects and analyzes user movement information, payment history, search terms, and social networking service postings, and further recognizes the user's emotions using an emotion engine to generate individually optimized messages.

[1900] Collection of user information

[1901] Acquisition of travel and visit information

[1902] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. For example, it collects latitude and longitude information every 10 minutes.

[1903] The device sends recorded information about visited locations and travel routes to the server in a batch process every hour.

[1904] Retrieving payment history

[1905] The user gives consent to the collection of payment history on their device. For example, they might check a checkbox in the app's settings screen that says, "I agree to the collection of payment history."

[1906] After the device has given consent, it calls a payment service API (e.g., PayPal or Stripe) to retrieve the user's latest payment history.

[1907] The device sends the acquired payment history data to the server.

[1908] Retrieving search terms

[1909] The device, with the user's consent, uses a browser API (for example, a Chrome extension) to collect the user's search history at regular intervals (for example, daily).

[1910] The device sends the collected search terms to the server.

[1911] Retrieving writing information

[1912] The user agrees to the use of APIs from major social networking services (e.g., Facebook and Twitter).

[1913] After the device gives its consent, it calls the SNS API to retrieve user posting information. Specifically, it collects recent posts and comments.

[1914] The terminal sends the acquired write information to the server.

[1915] Data analysis and estimation by interest

[1916] The server stores the various data it receives in a centralized database (for example, MySQL or MongoDB).

[1917] The server applies a natural language processing (NLP) engine (e.g., spaCy or NLTK) to the integrated data to analyze user search terms and social media posts.

[1918] Next, the server uses statistical and machine learning models (e.g., scikit-learn or TensorFlow) to estimate the user's interests and preferences. For example, it predicts what actions the user will take in the future based on keywords such as "relax" or "movies."

[1919] Based on the analysis results, the server updates the user profile to include the user's interests, preferences, and emotional state.

[1920] Utilizing the Emotion Engine

[1921] The server uses an emotion engine (for example, IBM Watson's Sentiment Analysis) to analyze user sentiment from their posts and search terms. For example, it might infer the sentiment "I want to relax" from a post that says "I want to relax."

[1922] The server reflects the analyzed emotion data in the user profile in real time.

[1923] The server takes emotional states into consideration and generates personalized messages based on detailed analysis.

[1924] Generating personalized messages

[1925] The server uses a generative AI model (e.g., OpenAI GPT-3) to generate multiple message suggestions based on the user profile and emotional state. For example, it might generate a message such as, "Why not relax with a cup of coffee and enjoy a recommended movie today?"

[1926] Next, the server assigns relevance scores to the multiple message candidates that have been generated (for example, scores based on the user's current behavior and sentiment) and selects the most suitable message.

[1927] Presenting a message

[1928] The server stores the selected messages in a daily log format and prepares for the next presentation.

[1929] The device retrieves the latest personalized message from the server at a set time (for example, 8 AM).

[1930] The device displays messages it has received to the user via push notifications, home screen widgets, and a dedicated app screen. For example, it might display a message like, "Why not relax with a cup of coffee and enjoy a recommended movie today?" in the smartphone's notification bar.

[1931] Specific example

[1932] For example, a user visited a cafe on the weekend and purchased coffee using a payment service. Furthermore, they posted on social media that they wanted to "relax" and recently searched for "recommended movies." In this case,

[1933] 1. The device records the user's location information, including visits to cafes, and obtains coffee purchase information from the payment history.

[1934] 2. The device also collects SNS posts and search terms, and sends all the information to the server.

[1935] 3. The server integrates and analyzes various pieces of information to estimate that the user needs to relax and that they like coffee.

[1936] 4. The server uses an emotion engine to more deeply analyze the user's desire to relax.

[1937] 5. The server generates a personalized message such as, "Why not relax today with a cup of coffee and enjoy a recommended movie?"

[1938] 6. The device will display this message to the user via push notification.

[1939] In this way, by providing messages that are tailored to the user's life circumstances and emotions, it is possible to increase user satisfaction and improve their quality of life. Through this system, users can more easily receive more personalized support.

[1940] Example of a prompt

[1941] For example, a prompt statement to be input to a generative AI model can be written as follows:

[1942] "A user visited a cafe on the weekend and purchased coffee using a payment service. Furthermore, they posted on social media that they wanted to 'relax,' and recently searched for 'recommended movies.' Please generate a personalized message for this user."

[1943] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1944] Step 1:

[1945] The device uses its built-in GPS sensor to periodically record the user's current location and places visited. For example, it acquires latitude and longitude information every 10 minutes and stores it in local memory.

[1946] Input: Program start command, GPS sensor data

[1947] Output: User's current location and visited locations information (latitude and longitude data)

[1948] Step 2:

[1949] The device sends recorded information about visited locations and travel routes to the server in a batch process every hour. Specifically, it sends the data to a message queue via an API.

[1950] Input: Recorded latitude and longitude data

[1951] Output: Data on visited locations and travel routes sent to the server.

[1952] Step 3:

[1953] The user gives consent to the collection of payment history on their device. For example, they might check a checkbox in the app's settings screen that says, "I agree to the collection of payment history."

[1954] Input: User consent action

[1955] Output: Consent information flags

[1956] Step 4:

[1957] After the device has given consent, it calls a payment service API to retrieve the user's latest payment history. For example, it might use the PayPal or Stripe API to retrieve transaction data.

[1958] Input: Consent information flag

[1959] Output: Payment history data

[1960] Step 5:

[1961] The device sends the acquired payment history data to the server. Specifically, it sends the data to the server via an HTTP request.

[1962] Input: Payment history data

[1963] Output: Payment history data sent to the server

[1964] Step 6:

[1965] With the user's consent, the device uses the browser API to collect the user's search history at regular intervals (for example, daily). Specifically, it uses a Chrome extension to collect search queries.

[1966] Input: User consent information, browser search data

[1967] Output: Collected search terms

[1968] Step 7:

[1969] The device sends the collected search terms to the server. For example, it might send them in JSON format via an HTTP request.

[1970] Input: Collected search terms

[1971] Output: Search words sent to the server

[1972] Step 8:

[1973] The user agrees to the use of APIs from major social networking services (SNS). Specifically, they check a checkbox in the app's settings screen that says something like, "I agree to the use of SNS APIs."

[1974] Input: User consent action

[1975] Output: Consent information flags

[1976] Step 9:

[1977] After the device gives its consent, it calls the SNS API to retrieve user posting information. For example, it uses the Facebook or Twitter API to retrieve recent posts and comments.

[1978] Input: Consent information flag

[1979] Output: SNS posting information

[1980] Step 10:

[1981] The device sends the acquired SNS posting information to the server. Specifically, it sends the data to the server via an HTTP request.

[1982] Input: Social media posting information

[1983] Output: SNS posting information sent to the server

[1984] Step 11:

[1985] The server stores various data it receives in a centralized database. For example, MySQL or MongoDB can be used to store the data.

[1986] Input: Visit location data, payment history data, search terms, social media postings

[1987] Output: Various data stored in the database

[1988] Step 12:

[1989] The server applies a natural language processing (NLP) engine to the integrated data to analyze user information. For example, it might use spaCy or NLTK to analyze text data and extract keywords and sentiments.

[1990] Input: Integrated data stored in the database

[1991] Output: Analyzed user interests and emotional states

[1992] Step 13:

[1993] The server uses statistical and machine learning models to estimate user interests and preferences. For example, it can use scikit-learn or TensorFlow to train data and generate an interest prediction model.

[1994] Input: Analyzed user interests and emotional states

[1995] Output: Estimated user interests and emotional states

[1996] Step 14:

[1997] The server uses a generative AI model to generate multiple message candidates based on the user profile and emotional state. For example, it can use OpenAI GPT-3 to generate personalized messages for the user.

[1998] Input: Estimated user interests and emotional state

[1999] Output: List of message candidates

[2000] Step 15:

[2001] The server assigns a relevance score to the generated message candidates and selects the most suitable message. Specifically, it uses a scoring algorithm to evaluate the relevance of each message.

[2002] Input: List of message suggestions

[2003] Output: Optimal message

[2004] Step 16:

[2005] The server stores the selected messages in a daily log format and prepares for the next presentation.

[2006] Input: Optimal message

[2007] Output: Messages stored in a daily calendar format

[2008] Step 17:

[2009] The device retrieves the latest personalized message from the server at a set time (for example, 8 AM). Specifically, it periodically sends requests to the server via an API.

[2010] Input: Time setting

[2011] Output: Latest message retrieved

[2012] Step 18:

[2013] The device displays messages it has received to the user via push notifications, home screen widgets, and dedicated app screens. For example, it displays messages in the smartphone's notification bar.

[2014] Input: Latest message retrieved

[2015] Output: Message displayed to the user

[2016] (Application Example 2)

[2017] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2018] In modern factories, improving operator efficiency is a critical challenge. However, because individualized support is required, taking into account each operator's work history and current emotional state, general assistance and advice have limited effectiveness. Furthermore, operators themselves may experience stress or a lack of job satisfaction, which can lead to decreased productivity. Therefore, a system is needed that can grasp each operator's individual situation and emotions in real time and provide personalized messages based on that information.

[2019] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[2020] In this invention, the server includes means for acquiring movement information and visit information, means for acquiring the user's payment history, means for acquiring the user's search terms, means for acquiring the user's posts, means for integrating the acquired information and performing analysis according to the user's interests, means for analyzing the user's emotions using an emotion engine, means for generating personalized messages based on the analysis results and emotional state, and means for presenting the generated personalized messages to the user. This makes it possible for robots deployed in a factory to generate and present personalized messages based on the operator's past work history, the situation during work, and information acquired from social networking services.

[2021] "Movement information" refers to the movement history and location data of users and operators, including where they have moved to.

[2022] "Visit information" refers to records of when users or operators visited a specific location and data on the time they spent at that location.

[2023] "Payment history" refers to records of transactions made by users or operators, as well as data on what goods or services were paid for.

[2024] "Search terms" refer to keywords or phrases that users or operators enter into internet search engines.

[2025] "Posts" refer to text or comments that users or operators post on social networking services or other platforms.

[2026] An "emotion engine" refers to an algorithm or system that analyzes emotions and emotional states from user or operator text data.

[2027] "Personalized messages" refer to messages and notifications that are customized based on the individual data of the user or operator.

[2028] A "robot" refers to a machine or device used in factories or other work environments to perform automated tasks.

[2029] An "operator" refers to a human worker who is responsible for operating machinery or systems in a factory or other work environment.

[2030] This invention provides a system that delivers personalized messages based on individual circumstances and emotional states to improve the operational efficiency of operators in a factory. This system collects and analyzes user movement information, payment history, search terms, and writing information, and uses an emotion engine to recognize emotional states, thereby generating and presenting individually optimized messages.

[2031] First, the system program uses GPS devices to acquire operator movement and visit information. For example, it utilizes a location information system such as the Garmin GLO 2. Next, to obtain payment history, the PaymentService API is used to collect payment information for goods and services made by operators within the factory. In addition, the Browser History API is used to obtain keywords searched by operators on the internet. Furthermore, APIs of major social networking services (SNS) are used to collect operator posting information. All of this data is sent to the server.

[2032] The server first integrates the collected data and uses natural language processing (NLP) and machine learning algorithms to analyze the operator's interests, concerns, and emotional state. Next, it uses an emotion engine (e.g., Google Cloud Natural Language API) to analyze the operator's emotional state in detail. Based on this analysis and the emotional state, a generative AI model (e.g., GPT-3 model) is used to generate personalized messages. These messages are created considering the operator's past work history, current work situation, and social media posts.

[2033] The generated personalized messages are presented to operators through robots placed in the factory. Presentation methods include the robot's display, voice output, or a dedicated application. For example, if an operator posts on social media that their work efficiency is low and is searching for "tips for improving productivity," the AI ​​model might generate a message such as "Take more breaks today and relax," and the robot would then present that message.

[2034] (Specific example)

[2035] Operator A posted on social media that "work efficiency is declining." They also searched for "tips for improving productivity" in their browser.

[2036] Movement information: The GPS device tracked Operator A's movements within the factory.

[2037] Payment history: Purchased an energy drink from a vending machine inside the factory.

[2038] Search terms: "Tips for improving productivity"

[2039] Social media post: "Work efficiency is declining."

[2040] (Example of a prompt message)

[2041] "Operator A has posted on social media about a decline in work efficiency. Their search history also shows they are looking for information on improving productivity. Based on this information, we will provide refreshing advice and personalized messages."

[2042] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2043] Step 1:

[2044] The terminal periodically records the operator's movement and visit information using a GPS device. Specifically, it acquires location data from a GPS device (e.g., Garmin GLO 2) and stores this data in its internal memory. In this step, the input is the location information from the GPS device, and the output is the recorded movement information.

[2045] Step 2:

[2046] The terminal periodically retrieves payment history, so it calls the PaymentService API to collect payment data. It retrieves transaction information from vending machines and shops. The inputs in this step are the user ID and payment event, and the output is payment history data.

[2047] Step 3:

[2048] The device periodically collects the user's search terms using the browser history API. This is a list of keywords and phrases the user has recently searched for. The input for this step is browser history data, and the output is a list of search terms.

[2049] Step 4:

[2050] The device uses the SNS API to retrieve user posting information. It periodically collects SNS posts and sends them to the server. The input in this step is the posted data obtained via the SNS API, and the output is the posting information.

[2051] Step 5:

[2052] The terminal sends all collected information (movement information, payment history, search terms, and posting information) to the server. The server integrates this information and stores it in a database. The input in this step is the user data sent from the terminal, and the output is the integrated dataset.

[2053] Step 6:

[2054] The server applies natural language processing (NLP) and machine learning algorithms to analyze the user's interests, concerns, and emotional state. This process identifies the user's current state and trends based on the collected data. The input in this step is an integrated dataset, and the output is the analyzed user profile.

[2055] Step 7:

[2056] The server uses an emotion engine (e.g., Google Cloud Natural Language API) to analyze the user's emotional state. It analyzes collected writing information and search terms to identify the user's emotions. The input in this step is the user's text data, and the output is emotional state data.

[2057] Step 8:

[2058] The server uses a generated AI model (e.g., a GPT-3 model) to generate personalized messages based on the analysis results and emotional state. It generates the optimal message tailored to the user's profile. The input in this step is the analyzed user profile and emotional state data, and the output is the personalized message.

[2059] Step 9:

[2060] The terminal retrieves a personalized message generated from the server and presents it to the operator via the display or voice output of a robot placed in the factory. In this step, the input is the personalized message, and the output is a notification to the operator.

[2061] Step 10:

[2062] The user acts based on the personalized message presented to them, aiming to improve efficiency and reduce stress. In this step, the input is the personalized message, and the output is the user's action.

[2063] By following these steps, it is possible to understand the individual circumstances and emotions of users in real time and provide optimal messages based on that understanding, thereby improving operational efficiency in the factory.

[2064] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2065] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2066] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2067] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2068] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[2069] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[2070] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[2071] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[2072] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[2073] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[2074] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[2075] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[2076] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[2077] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2078] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[2079] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[2080] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[2081] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above type...

Claims

1. Means for obtaining movement information and visit information, Means of obtaining a user's payment history, A means of obtaining the user's search terms, Means for obtaining user posts, A means of integrating the acquired information and performing analysis based on user interests, A means of generating messages based on each user's interests, A means of presenting the generated message to the user, A system that includes this.

2. The system according to claim 1, characterized in that the means for acquiring user movement information and visit information uses a location information system.

3. The system according to claim 1, characterized in that the means for obtaining user posts utilizes the API of a social networking service.

Images