system

Abstract

To provide a system.SOLUTION: A system learns user characteristics and performs an information communication service, information search, and information arrangement instead of a user. The system includes: means for providing a virtual account, which is an avatar of the user, on an information communication application; means for cooperating with the generative AI, pre-learning a history of the information communication application, an internet search tool, and a purchase history, and realizing a virtual account "capable of having conversation" and of "push type," which has the same characteristics as the user; means for preprocessing learned data and extracting characteristics of the user; means for training a generative AI model using the extracted characteristics of the user; and means for collecting related information on the basis of an interest and concern of the user and performing push notification to the user through the virtual account.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Currently, actions such as SNS, web search, and information organization need to be performed by individual users themselves, and a system that centrally substitutes these actions is required. However, it is difficult to substitute while reflecting the characteristics of users, and in particular, it is difficult to provide a "chat-capable" and "push-type" avatar account that learns the characteristics of users and reflects them.

Means for Solving the Problems

[0005] This invention provides a system that learns user characteristics and performs tasks such as SNS, web searches, and information organization on behalf of the user. Specifically, it provides an avatar account that acts as a surrogate for the user on a messenger app and pre-trains it with a generative AI using messenger app chat history and internet search tool shopping purchase history. This realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user. [Brief explanation of the drawing]

[0006] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 1 of Example 1. [Figure 12]This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2 of Embodiment 2. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Form Example 2. [Figure 15] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 3 of Example 3. [Figure 16] This is a sequence diagram showing the processing flow of the data processing system in Application Example 3 of Form Example 3. [Figure 17] This is a sequence diagram showing the processing flow of the data processing system in Example 1 of the Form 1 when an emotion engine is combined. [Figure 18] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] This is a sequence diagram showing the processing flow of the data processing system in Example 2 of the Form 2 when an emotion engine is combined. [Figure 20] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] This is a sequence diagram showing the processing flow of the data processing system in Example 3 of the Form 3 when an emotion engine is combined. [Figure 22] This is a sequence diagram showing the processing flow of the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. [Modes for carrying out the invention]

[0007] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)), etc.

[0010] In the following embodiments, a RAM (Random Access Memory) with a reference number is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0011] In the following embodiments, a storage with a reference number is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0012] In the following embodiments, a communication I / F (Interface) with a reference number is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0014] [First Embodiment]

[0015] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0016] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0019] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0022] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0026] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0027] "Example of form 1"

[0028] One embodiment of the present invention provides a system that learns user characteristics and performs tasks such as SNS, web searches, and information organization on behalf of the user. Specifically, an avatar account that acts as a surrogate for the user is provided on a messenger app. This avatar account pre-learns the user's messenger app chat history and internet search tool shopping purchase history, and by linking with a generative AI, it realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[0029] "Example of form 2"

[0030] As a concrete example, user A provides pre-training data such as their conversation history with friend B on a messenger app and their Amazon purchase history. Based on this data, the system works in conjunction with a generative AI to learn user A's characteristics. After learning is complete, the system provides an avatar account on the messenger app that acts as a digital representation of user A. This avatar account has the same characteristics as user A and is a "conversational," "push-type" avatar account that can perform actions such as posting on social media, sending messages, and performing web searches on behalf of user A.

[0031] "Example of form 3"

[0032] Furthermore, in another embodiment of the present invention, a function is provided to generate information and search results that reflect the user's preferences and tastes. Specifically, an avatar account is provided that automatically performs actions such as posting on social media, sending messages, and performing web searches. This avatar account can learn the user's characteristics and generate information and search results that reflect them.

[0033] The following describes the processing flow for each example of the form.

[0034] "Example of form 1"

[0035] Step 1: The user provides the system with their chat history with friends via a messenger app and their Amazon purchase history.

[0036] Step 2: Based on the provided data, the system works in conjunction with generative AI to learn the user's characteristics.

[0037] Step 3: After learning is complete, the system will provide the user with an avatar account on the messenger app, which will serve as their digital counterpart.

[0038] Step 4: This avatar account becomes a "conversational" and "push-type" avatar account with the same characteristics as the user, and can perform actions such as posting on social media, sending messages, and performing web searches on behalf of the user.

[0039] "Example of form 2"

[0040] Step 1: Users provide the system with behavioral data such as social media posts, message sending, and web searches.

[0041] Step 2: The system learns the user's characteristics based on the provided data.

[0042] Step 3: After learning is complete, the system will provide an avatar account that will serve as a digital representation of the user.

[0043] Step 4: This avatar account can generate information and search results that reflect the user's characteristics.

[0044] (Example 1)

[0045] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0046] In today's information society, users need to efficiently collect and organize vast amounts of information. However, doing so manually is time-consuming, laborious, and inefficient. Furthermore, there is a demand for information tailored to user characteristics and preferences, but conventional systems struggle to adequately achieve this. Additionally, there is a lack of means to centrally manage and utilize the history and characteristics of multiple information and communication services and search tools used by users. A system is needed to address these challenges.

[0047] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0048] In this invention, the server includes means for learning user characteristics and performing information communication services, information retrieval, and information organization on behalf of the user; means for providing a virtual account that acts as a surrogate for the user on an information communication application; means for collaborating with a generative AI to pre-train it on the history of information communication applications, internet search tools, and purchase history to realize a "conversational" and "push-type" virtual account with the same characteristics as the user; means for pre-processing collected data and extracting user characteristics; means for training a generative AI model using the extracted characteristics; and means for collecting relevant information based on the user's interests and preferences and sending push notifications through the virtual account. This makes it possible to provide information and generate search results based on the user's characteristics and preferences, and can significantly improve the efficiency of the user's information gathering and organization.

[0049] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and interests.

[0050] "Information and communication services" refers to communication methods such as messaging, social networking services (SNS), and email that are provided via the internet.

[0051] "Information retrieval" refers to the act of finding specific information on the internet.

[0052] "Information organization" refers to the act of classifying, organizing, and making usable information.

[0053] An "information and communication application" refers to software that allows users to send and receive messages and share information.

[0054] A "virtual account" refers to a digital avatar that operates on behalf of a user within an information and communication application.

[0055] "Generative AI" refers to artificial intelligence technology that learns user characteristics and generates responses and actions similar to those of the user.

[0056] "Pre-training" refers to the process of training a generative AI model in advance using collected data.

[0057] "Conversational" means that a virtual account has the ability to engage in natural conversations with the user.

[0058] "Push notifications" refer to a function that automatically notifies users of relevant information based on their interests and preferences.

[0059] "Preprocessing" refers to the process of preparing collected data into a format that is easy to analyze.

[0060] "Feature extraction" refers to the process of extracting characteristics such as user interests and preferences from pre-processed data.

[0061] "Training" refers to the process of using extracted characteristics to train a generative AI model.

[0062] "Push notifications" refer to a function that sends information to users in real time.

[0063] This invention is a system that learns user characteristics and performs information communication services, information retrieval, information organization, etc., on behalf of the user. Specific embodiments of this system are described below.

[0064] 1. System Overview

[0065] The server learns user characteristics and provides a system to perform information communication services, information retrieval, and information organization on behalf of the user. This system provides a virtual account that acts as a surrogate for the user on the information communication application and operates in conjunction with a generative AI.

[0066] 2. Hardware and software to be used

[0067] The server uses the following hardware and software:

[0068] Hardware: High-performance server machines, database servers

[0069] Software: Information and communication applications, generative AI models, data analysis tools, push notification systems

[0070] 3. Data Collection and Pre-training

[0071] The server collects the user's information and communication application history, internet search tool history, and purchase history. This data serves as foundational data for learning user characteristics. The collected data is preprocessed, with unnecessary data being removed and data normalization performed.

[0072] Next, user characteristics are extracted from the pre-processed data. Text data is analyzed using natural language processing techniques to identify user interests and preferences. A generative AI model is then trained using these extracted characteristics. This pre-training allows the generative AI model to understand user characteristics and generate responses similar to those of the user.

[0073] 4. Creating and configuring virtual accounts

[0074] The server uses a pre-trained generative AI model to generate a virtual account that reflects the user's characteristics. This virtual account can converse on behalf of the user within information and communication applications. Furthermore, the virtual account has the functionality to push relevant information based on the user's interests and preferences.

[0075] 5. Specific Examples

[0076] Example 1: Travel planning

[0077] When a user is planning a trip with a friend using an information and communication application, the server provides information on suitable destinations and accommodations based on past travel and search history. For example, it might suggest places the user has visited in the past that their friend might be interested in.

[0078] Example 2: Providing shopping information

[0079] If a user is interested in products from a particular brand, the server collects information about new products from that brand and sends push notifications through a virtual account. For example, it might immediately notify the user when a new product from a brand they have previously purchased is released.

[0080] 6. Example of a prompt statement

[0081] "Create a virtual account that provides information about travel destinations the user is interested in, based on their information and communication application history and internet search history. Also, implement a function to send push notifications with the latest information about those travel destinations."

[0082] In this way, a system is realized that learns the characteristics of users and performs information and communication services, information retrieval, and information organization on behalf of the user.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1: Data Collection

[0085] The server collects the user's information and communication application history, internet search tool history, and purchase history.

[0086] Input: User's information, communication application history, internet search history, purchase history

[0087] Data processing: The collected data is stored in a database for centralized management.

[0088] Output: User activity history data stored in the database

[0089] Specific operation: The server retrieves data from information and communication applications and internet search tools via APIs and stores it in a database.

[0090] Step 2: Data preprocessing

[0091] The server preprocesses the collected data.

[0092] Input: User activity history data stored in the database

[0093] Data processing: Deletion of unnecessary data, data normalization, tokenization of text data.

[0094] Output: Preprocessed clean dataset

[0095] Specific operation: The server uses a data cleansing tool to remove noisy data and prepares the text data in a format that is easy to analyze.

[0096] Step 3: Feature Extraction

[0097] The server extracts user characteristics from the pre-processed data.

[0098] Input: Preprocessed clean dataset

[0099] Data Processing: Analyze text data using natural language processing techniques to identify user interests and preferences.

[0100] Output: Feature vector representing user characteristics

[0101] Specific operation: The server uses a natural language processing library to analyze text data and extract user interests and preferences.

[0102] Step 4: Training the Generative AI Model

[0103] The server uses the extracted features to train a generative AI model.

[0104] Input: Feature vector representing user characteristics

[0105] Data processing: Input feature vectors into a generative AI model and train the model.

[0106] Output: Generative AI model that learned user characteristics

[0107] Specific operation: The server trains a generative AI model using a machine learning framework to learn the user's characteristics.

[0108] Step 5: Create a virtual account

[0109] The server uses a pre-trained generative AI model to generate virtual accounts that reflect the user's characteristics.

[0110] Input: Generative AI model that has learned user characteristics

[0111] Data processing: Create virtual accounts using a generative AI model and register them in an information and communication application.

[0112] Output: Virtual account on information and communication application

[0113] Specific operation: The server generates virtual accounts using a generated AI model and registers them with the information and communication application via an API.

[0114] Step 6: Implementing push notifications

[0115] The server collects relevant information based on the user's interests and sends push notifications through a virtual account.

[0116] Input: Data on user interests and preferences, external information sources

[0117] Data processing: Collect relevant information and format it in a way that is suitable for the user.

[0118] Output: Push notification message

[0119] Specific operation: The server collects relevant information from external information sources and sends push notifications to the user through a virtual account.

[0120] In this way, a system is realized that learns the characteristics of users and performs information and communication services, information retrieval, and information organization on behalf of the user.

[0121] (Application Example 1)

[0122] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0123] Modern consumers find it difficult and time-consuming to find the best product for them from the vast amount of information available. Furthermore, the lack of personalized product recommendations based on user characteristics and preferences makes it difficult for consumers to find suitable products. Additionally, the absence of systems that handle social media, web searches, and information organization on behalf of users means that users must gather information themselves, posing a significant challenge.

[0124] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0125] In this invention, the server includes means for learning user characteristics and performing SNS, web searches, information organization, etc. on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on a messenger app; means for collaborating with a generative AI to pre-train the AI ​​with messenger app chat history and internet search tool shopping purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; means for learning the user's purchase history and search history and automatically suggesting products that match the user's preferences; and means for suggesting products based on the user's history using a generative AI model. As a result, users can easily find products that are best suited to them and can save time and effort in gathering information.

[0126] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and purchase history.

[0127] "SNS" is an abbreviation for Social Networking Service, which is an online platform for users to share information and communicate with other users.

[0128] "Web search" refers to the act of finding information on the internet using a search engine.

[0129] "Information organization" refers to the act of classifying collected information and compiling it into an easily understandable format.

[0130] A "messenger app" is application software that allows users to send and receive text messages, images, audio, and other data.

[0131] An "avatar account" is a virtual account that functions as a substitute for the user, handling communication and information gathering on their behalf.

[0132] "Generative AI" refers to artificial intelligence technology that generates new information and suggestions based on user input and historical data.

[0133] "Pre-learning" refers to the process by which a system learns from a user's past behavior history and data in advance.

[0134] "Push-type" refers to a system that automatically provides information or suggestions before the user requests them.

[0135] "Purchase history" refers to a record of products that a user has purchased in the past.

[0136] "Search history" refers to a record of keywords and phrases that a user has previously searched for on a search engine.

[0137] A "generative AI model" refers to an artificial intelligence algorithm that generates new information and suggestions based on a user's historical data.

[0138] As an embodiment of this invention, a system is provided that learns user characteristics and performs SNS, web searches, information organization, etc., on behalf of the user. Specifically, an avatar account that acts as a surrogate of the user is provided on a messenger app, and in cooperation with a generative AI, the avatar is pre-trained with the user's messenger app chat history, internet search tool, and shopping purchase history. This avatar account realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[0139] The server has the ability to learn from the user's purchase and search history and automatically suggest products that match the user's preferences. It uses a generative AI model to suggest products based on the user's history. This system makes it easy for users to find the products that are best suited to them, saving them the trouble of gathering information themselves.

[0140] Hardware and software to be used

[0141] Hardware: Servers, user terminals (smartphones, tablets, PCs, etc.)

[0142] Software: Messenger apps, generative AI models (e.g., OpenAI®, GPT-3®)

[0143] Data processing and data calculation

[0144] The server collects the user's messenger app chat history, internet search tool data, and shopping purchase history, and uses this data for pre-training. Using a generative AI model, it analyzes the user's historical data and learns the user's characteristics and preferences. This allows it to automatically suggest products that match the user's preferences.

[0145] Specific example

[0146] If a user has previously purchased a "smartphone" or "wireless earphones" and has searched for "the latest smartphone" or "high-quality earphones," the generative AI model will suggest products related to "the latest smartphone" or "high-quality earphones."

[0147] Example of a prompt

[0148] User purchase history: ["Smartphone", "Wireless earphones"]

[0149] User's search history: ["Latest smartphone", "High-quality earphones"]

[0150] Please suggest products that would be suitable for this user.

[0151] In this way, a personal shopping assistant can be realized that automatically suggests products that match the user's preferences.

[0152] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0153] Step 1:

[0154] The server collects users' messenger app chat history, internet search tool data, and shopping purchase history. This data serves as input for learning user characteristics and preferences. Specifically, it retrieves data from each platform via APIs and stores it in a database.

[0155] Input: Messenger app chat history, internet search tools, shopping purchase history

[0156] Output: User history data stored in the database

[0157] Step 2:

[0158] The server pre-trains on the collected user history data. In this pre-training process, a generative AI model is used to analyze the data and learn user characteristics and preferences. Specifically, the history data is input into the AI ​​model to extract user behavior patterns and preferences.

[0159] Input: User history data stored in the database

[0160] Output: A model that reflects the user's characteristics and preferences.

[0161] Step 3:

[0162] The server uses a generative AI model to suggest products based on the user's history. In this process, prompt sentences are generated based on the user's purchase and search history to suggest the most suitable products, and these are input into the AI ​​model. The AI ​​model then generates product suggestions based on these prompt sentences.

[0163] Input: A model that reflects the user's characteristics and preferences, and prompt text.

[0164] Output: Product proposal

[0165] Step 4:

[0166] The server sends the generated product suggestions to the user's device. Users can then view the suggested products through a messenger app or a dedicated app. Specifically, product suggestions are sent to the user's device via an API and displayed in the user interface.

[0167] Input: Product proposal

[0168] Output: Product suggestions displayed on the user's terminal

[0169] Step 5:

[0170] Users review the suggested products and purchase them if necessary. This process involves users receiving product suggestions, selecting items of interest, and proceeding with the purchase. Specifically, they select products through the user interface and click the purchase button.

[0171] Input: Product suggestions displayed on the user's terminal

[0172] Output: User purchasing behavior

[0173] In this way, the system automatically suggests products based on the user's characteristics and preferences, enabling users to easily find the products that are best suited to them.

[0174] (Example 2)

[0175] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0176] Conventional information processing systems struggled to provide automated information and communication that fully reflected user characteristics. Furthermore, creating avatar accounts that could perform tasks such as SNS posting, message sending, and web searches on behalf of users was also difficult. This resulted in increased user burden and inefficient information processing.

[0177] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for learning user characteristics and processing information on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on the messenger app; means for coordinating with a generative AI model to pre-train it with the conversation history of the messenger app and the purchase history of online shopping to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; means for pre-processing data stored in the database, cleaning and tokenizing text data; and means for inputting prompt sentences to the generative AI model and sending the generated message. This enables automated information provision and communication that reflects the user's characteristics.

[0178] "User characteristics" refer to individual features such as user behavior patterns, preferences, and tastes.

[0179] "Information processing" refers to a series of operations such as data collection, analysis, storage, retrieval, and generation.

[0180] An "avatar account" refers to a virtual account that operates online on behalf of a user.

[0181] A "generative AI model" refers to an artificial intelligence model that generates new information or content based on given data.

[0182] "Pre-training" refers to the process of training a model using data in advance for a specific task.

[0183] "Being able to converse" refers to having the ability to engage in natural conversations on behalf of the user.

[0184] "Push-type" refers to a method of automatically providing information without waiting for a user request.

[0185] A "database" refers to a system for efficiently storing, managing, and retrieving data.

[0186] "Preprocessing" refers to the initial steps taken to convert data into a format suitable for analysis and learning.

[0187] "Cleaning" refers to the process of removing unnecessary information and noise from data.

[0188] "Tokenization" refers to the process of dividing text data into words or phrases.

[0189] A "prompt statement" refers to an input statement used to cause a generative AI model to generate a specific output.

[0190] This invention is a system that learns user characteristics and processes information on behalf of the user. Specific embodiments of this system are described below.

[0191] The server receives chat history from messenger apps and purchase history from online shopping provided by the user. This data is used as pre-training data to learn user characteristics. The server stores this data in a database. The database used is a relational database such as MySQL® or PostgreSQL.

[0192] Next, the server preprocesses the stored data. This preprocessing includes cleaning the text data (removing unnecessary characters and noise) and tokenization (dividing it into words and phrases). These processes are performed using Python's NLTK and spaCy libraries.

[0193] The pre-processed data is input into a generative AI model (for example, OpenAI's GPT-4®). The server uses this generative AI model to learn the user's characteristics. This learning process incorporates the user's conversation patterns and purchasing tendencies into the model. Once learning is complete, the server generates a model that embodies the user's characteristics.

[0194] The server generates an avatar account with the user's characteristics based on a trained model. This avatar account can then converse on behalf of the user within the messenger app. Specifically, the avatar account inputs prompt text into the generated AI model and sends the generated message to the friend.

[0195] As a concrete example, consider a scenario where a user provides their conversation history with a friend. For instance, the following prompt might be input into the AI ​​model:

[0196] Prompt: "Generate a message for a user to send to a friend. The user recently purchased a new smartphone and wants to tell their friend about their experience using it."

[0197] The generative AI model generates messages like the following:

[0198] Generated message: "Hi friend! I recently bought a new smartphone and it's really easy to use. The camera is especially amazing; it takes really beautiful photos. If you're thinking about getting a new smartphone, I highly recommend it!"

[0199] In this way, avatar accounts can send messages to friends that reflect the user's characteristics. Furthermore, avatar accounts can also make social media posts and perform web searches. This reduces the user's burden and enables efficient information processing.

[0200] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0201] Step 1: Data Collection

[0202] The server receives conversation history from messenger apps and purchase history from online shopping from users. This data is used as pre-training data to learn user characteristics. The input is conversation history and purchase history provided by the user, and the output is raw data stored in the database. Specifically, the server receives data uploaded by the user and stores it in the database.

[0203] Step 2: Data Preprocessing

[0204] The server preprocesses the stored data. This preprocessing includes cleaning the text data (removing unnecessary characters and noise) and tokenizing it (dividing it into words and phrases). The input is the raw data stored in the database, and the output is the preprocessed, clean data. Specifically, the server uses Python's NLTK and spaCy libraries to clean and tokenize the text data.

[0205] Step 3: Trait Learning

[0206] The server inputs pre-processed data into a generative AI model to learn user characteristics. The input is clean, pre-processed data, and the output is a trained model that reflects user characteristics. Specifically, the server uses a generative AI model (for example, OpenAI's GPT-4) to learn user conversation patterns and purchasing tendencies.

[0207] Step 4: Create an avatar account

[0208] The server generates an avatar account with the user's characteristics based on a pre-trained model. The input is the pre-trained model, and the output is the avatar account. Specifically, the server creates an avatar account that converses on behalf of the user in the messenger app based on the output of the generated AI model.

[0209] Step 5: Avatar Account Operation

[0210] The avatar account inputs a prompt into a generative AI model and sends the generated message to a friend. The input is the prompt, and the output is the generated message. Specifically, the avatar account inputs a prompt into the generative AI model like the following:

[0211] Prompt: "Generate a message for a user to send to a friend. The user recently purchased a new smartphone and wants to tell their friend about their experience using it."

[0212] The generative AI model generates messages like the following:

[0213] Generated message: "Hi friend! I recently bought a new smartphone and it's really easy to use. The camera is especially amazing; it takes really beautiful photos. If you're thinking about getting a new smartphone, I highly recommend it!"

[0214] The avatar account sends this message to its friends. This enables automated information sharing and communication that reflects the user's characteristics.

[0215] (Application Example 2)

[0216] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0217] Traditional social networking services (SNS) and messaging apps required users to gather information, post, and send messages themselves, which was time-consuming and laborious. Furthermore, there was a lack of effective means to deliver advertisements that reflected user characteristics and preferences. Therefore, there is a need to improve user convenience while maximizing the effectiveness of advertising.

[0218] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for learning user characteristics and performing SNS / web searches / information organization on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on a messenger app; means for collaborating with a generative AI to pre-train the AI ​​with messenger app talk history and internet search tool shopping purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; and means for the avatar, which has learned the user's characteristics, to post advertisements on SNS and messenger apps on behalf of the user. This makes it possible to automatically collect and post information on behalf of the user, improving user convenience and enabling effective advertising delivery that reflects the user's characteristics and preferences.

[0219] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and tastes.

[0220] "SNS" is an abbreviation for Social Networking Service, which refers to an online platform for users to share information and communicate with other users.

[0221] "Web search" refers to the act of finding information on the internet using a search engine.

[0222] "Information organization" refers to the act of classifying and organizing collected information.

[0223] A "messenger app" refers to an application that allows users to send and receive text messages, images, videos, and other content.

[0224] An "avatar account" refers to a virtual account that acts as a digital representation of a user online.

[0225] "Generative AI" refers to artificial intelligence that generates new information or content based on given data.

[0226] "Pre-training" refers to the process of training a model in advance using relevant data before performing a specific task.

[0227] "Being able to converse" refers to having the ability to engage in natural conversations with users.

[0228] "Push marketing" refers to a method of automatically providing information before the user requests it.

[0229] "Posting an advertisement" refers to the act of publishing information about a specific product or service on social media or messaging apps.

[0230] The system for implementing this invention learns user characteristics and performs tasks such as SNS, web searches, and information organization on behalf of the user. Specifically, it provides an avatar account that acts as a surrogate for the user on a messenger app, and by linking with a generative AI to pre-train the avatar with messenger app chat history and internet search tool shopping purchase history, it realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[0231] The server first retrieves the user's conversation history and purchase history from messenger apps. This data is collected using APIs from messenger apps and shopping sites. Next, the server uses generative AI to learn the user's characteristics based on this data. Specifically, the conversation history and purchase history are input as prompts into the generative AI model to learn the user's characteristics.

[0232] After learning is complete, the server generates an avatar account with the user's characteristics, and this avatar posts advertisements on social media and messaging apps on the user's behalf. This enables automatic information gathering and posting on behalf of the user, improving user convenience and allowing for effective ad delivery that reflects the user's characteristics and preferences.

[0233] The hardware used includes servers and user terminals (smartphones, tablets, PCs, etc.). The software includes messenger apps, shopping site APIs, and generative AI (for example, OpenAI APIs).

[0234] As a concrete example, consider a case where the user ID is "user123" and the advertisement content is "Check out this amazing product!". In this case, the server will operate as follows:

[0235] 1. The server retrieves the conversation history and purchase history of user ID "user123".

[0236] 2. The server uses generative AI to learn the characteristics of "user123" based on the acquired data.

[0237] 3. The server uses the trained user model to post an advertisement on social media saying, "Check out this amazing product!"

[0238] Examples of prompts to input into a generative AI model:

[0239] Train a model based on the following data: [Conversation history and purchase history data]

[0240] This prompt is used to train the generative AI model on user characteristics.

[0241] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0242] Step 1:

[0243] The server retrieves the user's conversation and purchase history from messenger apps. Specifically, it uses APIs from messenger apps and shopping sites to collect conversation and purchase history associated with the user ID. The input is the user ID, and the output is conversation and purchase history data.

[0244] Step 2:

[0245] The server uses generative AI to learn user characteristics based on acquired conversation and purchase history. Specifically, it inputs conversation and purchase history as prompts into the generative AI model to learn user characteristics. The input is conversation and purchase history data, and the output is a model that reflects the user's characteristics.

[0246] Step 3:

[0247] The server generates avatar accounts based on the learned user model. Specifically, it uses generative AI to create avatar accounts that possess the user's characteristics and provides them to the messenger app. The input is the user model, and the output is the avatar account.

[0248] Step 4:

[0249] The server uses avatar accounts to post advertisements on social media and messaging apps. Specifically, it generates ad content that reflects the user's characteristics and posts it using the APIs of the social media and messaging apps. The input is the user model and ad content, and the output is the ad post on the social media or messaging app.

[0250] Step 5:

[0251] The server monitors the effectiveness of the ads and updates the user model as needed. Specifically, it collects data such as ad click-through rates and engagement rates, and uses generative AI to retrain the user model. The input is the ad performance data, and the output is the updated user model.

[0252] (Example 3)

[0253] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0254] Traditional information systems and search engines have struggled to adequately reflect user preferences and tastes, making it difficult to provide users with the most relevant information. Furthermore, the effort required for users to search for and organize information themselves made efficient information gathering difficult.

[0255] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for learning user characteristics and providing information and generating search results on behalf of the user; means for providing a virtual account on the messaging application that acts as a proxy for the user; means for working in cooperation with a generative AI to pre-train messaging application talk history and internet search tool purchase history to realize an "interactive" and "push-type" virtual account with the same characteristics as the user; means for collecting user behavior data and analyzing the data using natural language processing technology; means for generating a profile that reflects the user's preferences and tastes based on the analysis results; means for the virtual account to automatically provide information and perform searches based on the generated profile; and means for generating customized information and search results based on the virtual account's behavior results. This makes it possible to generate optimal information and search results that reflect the user's preferences and tastes.

[0256] "User characteristics" refer to individual features such as a user's preferences, tastes, behavioral patterns, and interests.

[0257] "Information provision" refers to the act of providing useful information to users.

[0258] "Search results" refer to a list of information returned in response to a user's search on the internet.

[0259] A "messaging application" refers to software that allows users to send and receive text messages and multimedia messages.

[0260] A "virtual account" refers to a digital account that acts on behalf of a user.

[0261] "Generative AI" refers to artificial intelligence that has the ability to generate new information and content based on data.

[0262] "Pre-training" refers to the process by which a system learns using the user's past data.

[0263] "Interactive" refers to a system having the ability to engage in natural conversations with users.

[0264] "Push-type" refers to a system that automatically provides information without waiting for a user request.

[0265] "Behavioral data" refers to data related to user actions (e.g., social media posts, search history, message sending, etc.).

[0266] "Natural language processing technology" refers to the technology used to understand, analyze, and generate human language.

[0267] A "profile" refers to a collection of data generated based on a user's characteristics and behavioral patterns.

[0268] "Customized information" refers to information that has been individually tailored based on the user's characteristics and preferences.

[0269] Modes for carrying out the invention

[0270] This invention is a system that learns user characteristics and provides information and generates search results on behalf of the user. Specific embodiments of this system are described below.

[0271] 1. Generating the system program

[0272] The server generates programs that provide information and search results that reflect the user's preferences and tastes. These programs have the functionality to manage virtual accounts that automatically perform user actions such as posting on social media, sending messages, and performing web searches.

[0273] 2. Explanation of the program's processing

[0274] The server performs the following processes using the generated program.

[0275] 1. Data collection:

[0276] The server collects data such as the user's SNS posts, message transmissions, and web search history. To collect this data, the server obtains data from each SNS platform and search engine through APIs. Specifically, the Twitter API and Google (registered trademark) Search API are used.

[0277] 2. Data analysis:

[0278] The server uses natural language processing (NLP) technology to analyze the collected data. Specifically, software such as the Google Cloud Natural Language API and IBM Watson (registered trademark) Natural Language Understanding is used.

[0279] 3. Generation of user profiles:

[0280] The server generates a profile that reflects the user's preferences and tastes based on the analysis results. This profile includes topics and keywords that the user is interested in.

[0281] 4. Simulation of virtual account behavior:

[0282] Based on the generated user profile, the server automatically makes the virtual account perform SNS posts, message transmissions, and web searches. For this simulation, a generative AI model (e.g., OpenAI's GPT-4) is used.

[0283] 5. Information provision and generation of search results:

[0284] The server generates customized information and search results for users based on the actions of their virtual accounts. This information is provided in a format that best suits the user's preferences and tastes.

[0285] 3. Specific Examples and Examples of Prompt Statements

[0286] Specific example:

[0287] Let's say user A is interested in travel. The server extracts keywords such as "travel," "tourist destinations," and "hotels" from user A's social media posts and web search history. Based on this, a virtual account automatically makes travel-related social media posts and searches for travel blogs. Finally, it provides user A with information on recommended tourist destinations and hotels.

[0288] Example of a prompt:

[0289] "To provide user A with travel-related information they might be interested in, please suggest recommended tourist destinations and hotels based on their social media posts and web search history."

[0290] In this way, the server realizes a system that generates information and search results that reflect the user's preferences and tastes. The flow of specific processing in Example 3 will be explained using Figure 15.

[0291] Step 1: Data Collection

[0292] The server collects data such as users' social media posts, message transmissions, and web search history. Specifically, it uses the Twitter API to retrieve users' tweets and the Google Search API to collect users' search history. The input is the user's account information, and the output is the collected behavioral data.

[0293] Step 2: Data Analysis

[0294] The server uses natural language processing (NLP) techniques to analyze the collected data. Specifically, it uses the Google Cloud Natural Language API to extract keywords from users' tweets and search history and perform sentiment analysis. The input is collected behavioral data, and the output is the analyzed keywords and sentiment information.

[0295] Step 3: Generate User Profile

[0296] The server generates a profile that reflects the user's preferences and tastes based on the analysis results. Specifically, it lists topics and keywords that the user is interested in, based on the extracted keywords and sentiment analysis results. The input is the analyzed keywords and sentiment information, and the output is the generated user profile.

[0297] Step 4: Virtual Account Behavior Simulation

[0298] Based on the generated user profile, the server automatically performs social media posting, message sending, and web searches using a virtual account. Specifically, it uses a generative AI model (e.g., OpenAI's GPT-4) to generate social media posts based on the user's interests and search for relevant websites. The input is the user profile, and the output is the result of the virtual account's actions.

[0299] Step 5: Information provision and search result generation

[0300] The server generates customized information and search results for users based on the actions of virtual accounts. Specifically, it organizes the information collected by virtual accounts and provides users with recommendations for tourist destinations and hotels. The input is the actions of virtual accounts, and the output is customized information and search results.

[0301] (Application Example 3)

[0302] Next, Application Example 3 of Embodiment Example 3 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart device 14 is referred to as a "terminal".

[0303] In a conventional system, it has been difficult to generate information provision and search results that accurately reflect the preferences and tastes of users. In addition, there was a problem that users had to perform SNS postings, message transmissions, and web searches themselves, which took time and effort. Furthermore, the recommendation function for proposing products optimal for users was insufficient, and the purchasing motivation of users could not be enhanced. In order to solve these problems, there is a demand for a system that automates user behavior, learns user characteristics, and provides optimal information and product proposals.

[0304] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0305] In this invention, the server includes means for learning user characteristics and performing SNS, web searches, information organization, etc. on behalf of the user, means for providing an avatar account that serves as the user's alter ego on the messenger app, means for cooperating with generative AI to pre-learn the messenger app conversation history and internet search tool shopping purchase history to realize a "conversable" and "push-type" avatar account having the same characteristics as the user, means for analyzing the user's SNS postings, message transmissions, and web search history to learn the user's preferences and tastes, means for proposing products optimal for the user, means for automatically posting product reviews and questions, and means for performing recommendations based on the purchase history. As a result, it becomes possible to automate user behavior, learn user characteristics, and provide optimal information and product proposals.

[0306] "User characteristics" refers to individual characteristics such as the preferences and tastes of the user and the user's behavior pattern.

[0307] "SNS" is an abbreviation for Social Networking Service, which refers to a platform for users to interact with other users online.

[0308] "Web search" refers to the act of finding information on the internet using a search engine.

[0309] "Information organization" refers to the process of classifying and organizing collected data and information to make it easier to use.

[0310] A "messenger app" refers to application software used to send and receive text messages, images, videos, and other similar content.

[0311] An "avatar account" refers to a virtual account that acts as a digital representation of a user online.

[0312] "Generative AI" refers to artificial intelligence technology that generates new information and content based on data.

[0313] "Pre-training" refers to the process of training a model in advance using relevant data before performing a specific task.

[0314] "Push marketing" refers to a method of automatically providing information before the user requests it.

[0315] "SNS posting" refers to the act of publishing content such as text, images, and videos on social networking services.

[0316] "Sending a message" refers to the act of sending text messages, images, videos, etc., to other users.

[0317] "Web search history" refers to a record of searches a user has performed on the internet in the past.

[0318] "Product recommendation" refers to the act of recommending appropriate products based on the user's preferences and tastes.

[0319] A "product review" refers to the act of writing an evaluation or comment about a product that has been purchased.

[0320] "Recommendation" refers to a system that recommends appropriate products or services based on a user's past behavior and preferences.

[0321] The system for implementing this invention learns user characteristics and performs tasks such as SNS, web searches, and information organization on behalf of the user. The system provides an avatar account that acts as a surrogate for the user on a messenger app and operates in conjunction with a generative AI. A specific embodiment of the system is described below.

[0322] System Configuration

[0323] The system consists of the following main components:

[0324] 1. User device: A device such as a smartphone or tablet with a messenger app installed.

[0325] 2. Server: Collects and analyzes user data and provides information and product suggestions using generative AI models.

[0326] 3. Generative AI Models: AI models that analyze users' social media posts, message transmissions, and web search history to learn user preferences and tastes.

[0327] Program processing

[0328] The server implements the system using the following methods.

[0329] 1. Learning User Characteristics: The server collects users' social media posts, message transmissions, and web search history, and learns user characteristics using a generative AI model. Specifically, it converts text data into TF-IDF vectors and analyzes user preferences and tastes.

[0330] 2. Provision of Avatar Accounts: An avatar account will be created on the messenger app to act as a digital representation of the user, and it will post on social media and send messages on behalf of the user.

[0331] 3. Information Provision and Product Recommendations: The server provides optimal information and product recommendations based on the user's characteristics. Specifically, it makes recommendations based on the user's purchase history and search history.

[0332] 4. Automated posting: Avatar accounts automatically post product reviews and questions on behalf of users.

[0333] Hardware and software to be used

[0334] Hardware: User devices (smartphones, tablets), servers

[0335] Software: Messenger apps, generative AI models (e.g., scikit-learn's TfidfVectorizer, cosine_similarity)

[0336] Specific example

[0337] For example, if a user posts on social media that they "want a new smartphone," the system collects this information and learns the user's preferences. Then, if the user frequently searches for "high-performance laptops" on the web, the system uses this information to suggest the best products. Furthermore, the avatar account automatically posts product reviews and questions, organizing information on behalf of the user.

[0338] Example of a prompt

[0339] Create an application that analyzes users' social media posts, messages, and web search history to learn their preferences and tastes, and then recommends the most suitable products. User data will be retrieved via an API, and text data will be analyzed using TF-IDF vectors. Cosine similarity will be used for product recommendations.

[0340] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[0341] Step 1:

[0342] The user's device collects their social media posts, message transmissions, and web search history. This data serves as input for learning the user's preferences and tastes. Specifically, it collects text posts made by the user on social media, text messages sent via messaging apps, and URLs and search queries from their web search history.

[0343] Step 2:

[0344] The server inputs the collected user data into a generative AI model. The generative AI model uses this data to learn user characteristics. Specifically, it converts text data into TF-IDF vectors and analyzes user preferences and tastes. The input data is in text format, and the output is vector data representing user characteristics.

[0345] Step 3:

[0346] The server creates an avatar account on the messenger app that acts as a digital representation of the user. This avatar account makes social media posts and sends messages on behalf of the user. Specifically, it posts text generated based on the user's characteristics. The input data is a vector of the user's characteristics, and the output is the text of the social media post or message.

[0347] Step 4:

[0348] The server provides optimal information and product suggestions based on the user's characteristics. Specifically, it makes recommendations based on the user's purchase history and search history. The input data consists of the user's characteristic vector, purchase history, and search history, and the output is a list of recommended products.

[0349] Step 5:

[0350] The avatar account automatically posts product reviews and questions on behalf of the user. Specifically, it generates reviews and questions for recommended products and posts them from the avatar account. The input data is a list of recommended products, and the output is the text of the product reviews and questions.

[0351] Step 6:

[0352] The server continuously monitors user behavior, collects new data, and updates the generative AI model. This ensures that user characteristics are always up-to-date. The input data is newly collected user data, and the output is an updated user characteristic vector.

[0353] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0354] "Example of form 1"

[0355] One embodiment of the present invention is a system that incorporates an emotion engine. This system recognizes the user's emotions and adjusts the behavior of the avatar account based on those emotions. Specifically, when the user is feeling happy, the avatar account posts cheerful topics that reflect that emotion on social media. Also, when the user is sad, the avatar account sends comforting messages that reflect that emotion to their friends.

[0356] "Example of form 2"

[0357] Furthermore, systems that incorporate an emotion engine can recognize a user's emotions and adjust the information provided and search results based on those emotions. For example, when a user is feeling stressed, the system will provide information on relaxation and meditation. When a user is feeling happy, the system will provide search results that reflect that emotion, such as enjoyable events and news.

[0358] "Example of form 3"

[0359] Furthermore, a system incorporating an emotion engine can recognize a user's emotions and automatically perform actions on the avatar account based on those emotions. For example, when a user is angry, the avatar account automatically posts on social media that reflects that emotion. Also, when a user is surprised, the avatar account automatically reacts to a friend's post that reflects that emotion.

[0360] The following describes the processing flow for each example of the form.

[0361] "Example of form 1"

[0362] Step 1: The emotion engine recognizes the user's emotions.

[0363] Step 2: Adjust the avatar account's behavior based on the recognized emotions.

[0364] Step 3: For example, when a user is feeling happy, the avatar account posts fun topics on social media that reflect that emotion.

[0365] Step 4: Also, when a user is sad, the avatar account will send comforting messages to their friends that reflect that emotion.

[0366] "Example of form 2"

[0367] Step 1: The emotion engine recognizes the user's emotions.

[0368] Step 2: Adjust information and search results based on the recognized emotions.

[0369] Step 3: For example, when a user is feeling stressed, the system provides information about relaxation and meditation.

[0370] Step 4: When the user is feeling happy, the system will also provide search results with enjoyable events and news that reflect that emotion.

[0371] "Example of form 3"

[0372] Step 1: The emotion engine recognizes the user's emotions.

[0373] Step 2: Automate actions for the avatar account based on recognized emotions.

[0374] Step 3: For example, when a user is feeling angry, the avatar account automatically posts on social media that reflects that emotion.

[0375] Step 4: Also, when a user is surprised, the avatar account automatically reacts to their friend's post in a way that reflects that emotion.

[0376] (Example 1)

[0377] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0378] Conventional information processing systems struggled to fully reflect user characteristics and emotions during automation. When performing tasks such as posting on social media, sending messages, or searching for information on behalf of users, they failed to accurately reflect user intentions and feelings. Furthermore, they were insufficient in providing information and generating search results that reflected user preferences and tastes. This resulted in reduced user convenience and limited the system's usefulness.

[0379] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes means for learning user characteristics and processing information on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on the messenger app; means for working in cooperation with a generative AI to pre-train the AI ​​with messenger app talk history, internet search history, and purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; and means for recognizing the user's emotions using an emotion engine and adjusting the avatar account's behavior based on those emotions. This enables automation that accurately reflects the user's characteristics and emotions, thereby improving user convenience.

[0380] "User characteristics" refer to individual features such as a user's behavioral patterns, preferences, tastes, and emotions.

[0381] "Information processing" refers to a series of operations such as data collection, analysis, generation, and provision.

[0382] A "messenger app" refers to software that allows users to send and receive text messages and multimedia messages.

[0383] An "avatar account" refers to a virtual account that functions as a substitute for a user and acts on their behalf.

[0384] "Generative AI" refers to artificial intelligence that has the ability to generate new information and content based on data.

[0385] "Pre-training" refers to the process of learning using data in advance for a specific task.

[0386] An "emotion engine" refers to software that analyzes a user's emotions and generates appropriate responses based on those emotions.

[0387] "Push-type" refers to a method of automatically providing information and notifications without waiting for a user request.

[0388] This invention relates to a system that learns user characteristics and processes information on behalf of the user. This system provides an avatar account that acts as a surrogate for the user on a messenger app and learns the user's characteristics in cooperation with a generative AI. It also uses an emotion engine to recognize the user's emotions and adjusts the avatar account's behavior based on those emotions.

[0389] Hardware and software to be used

[0390] Hardware: Servers, user terminals (smartphones, PCs)

[0391] Software: Messenger apps, internet search tools, generative AI models (e.g., GPT-4), emotion engines

[0392] Data processing and data calculation

[0393] Data Collection: The server periodically collects users' messenger app chat history, internet search history, and purchase history. Specifically, the server retrieves chat history from messenger apps via APIs and search history from internet search tools. It also collects purchase history using shopping site APIs.

[0394] Data Pre-training: The server pre-trains a generative AI model using the collected data. During this process, it learns user characteristics and preferences. Specifically, the server pre-processes the collected data and tokenizes the text data. Then, it inputs the data into the generative AI model to learn user characteristics and preferences.

[0395] Emotion Recognition: The server uses an emotion engine to recognize the user's emotions in real time. Specifically, the server analyzes the latest message received from the messenger app and inputs it into the emotion engine. The emotion engine analyzes the user's emotions from the content and tone of the message and identifies emotions such as "joy," "sadness," and "anger."

[0396] Action Generation: The server combines a generative AI model and an emotion engine to generate actions based on the user's emotions. Specifically, the server inputs prompt text into the generative AI model based on the emotion information obtained from the emotion engine. For example, it inputs the prompt text, "Generate a fun topic to post on social media when the user is happy," and retrieves the generated text.

[0397] Action Execution: The server executes the generated action. Specifically, the server posts the generated text to messenger apps or social media. For example, when the user is happy, it posts a generated cheerful topic to social media, and when the user is sad, it sends generated words of comfort to their friends.

[0398] Specific example

[0399] Example 1: When a user sends "I'm so happy today!" via a messenger app, the emotion engine recognizes "joy." The generative AI model then posts fun topics to social media based on the user's past chat history.

[0400] Example 2: If a user sends "I'm tired today," the emotion engine recognizes "fatigue" and "sadness." The generative AI model then sends words of comfort to the user's friend.

[0401] Example of a prompt

[0402] "Generate fun topics that users will post on social media when they're happy."

[0403] "Generate comforting messages for users to send to their friends when they are feeling sad."

[0404] The above describes embodiments for carrying out the present invention.

[0405] The flow of the specific processing in Example 1 will be explained using Figure 17.

[0406] Step 1: Data Collection

[0407] The server periodically collects users' messenger app chat history, internet search history, and purchase history. Specifically, the server retrieves chat history from messenger apps via APIs and search history from internet search tools. It also collects purchase history using APIs from shopping sites.

[0408] Input: Messenger app chat history, internet search history, purchase history

[0409] Output: Collected user data

[0410] Specific operations: The server calls the messenger app's API every day at 2 AM to retrieve chat history. It also collects internet search history through browser extensions and retrieves purchase history from shopping site APIs once a week.

[0411] Step 2: Data pre-training

[0412] The server uses the collected data to pre-train a generative AI model. In this process, it learns the user's characteristics and preferences.

[0413] Input: Collected user data

[0414] Output: Pre-trained generative AI model

[0415] Specific operation: The server stores the collected data in JSON format and places it in a database. It reads the data from the database and tokenizes the text data. Then, it inputs the data into a generative AI model to learn user characteristics and preferences.

[0416] Step 3: Emotion Recognition

[0417] The server uses an emotion engine to recognize the user's emotions in real time. Specifically, the server analyzes the latest message received from the messenger app and inputs it into the emotion engine.

[0418] Input: Latest messenger app message

[0419] Output: User sentiment information

[0420] Specific operation: The server retrieves the latest message sent by the user via the messenger app in real time and inputs it into the emotion engine. The emotion engine analyzes the user's emotions from the content and tone of the message and identifies emotions such as "joy," "sadness," and "anger."

[0421] Step 4: Generate Action

[0422] The server combines a generative AI model and an emotion engine to generate actions based on the user's emotions. Specifically, the server inputs prompt text into the generative AI model based on the emotional information obtained from the emotion engine.

[0423] Input: User sentiment information

[0424] Output: Generated action text

[0425] Specific operation: Based on the emotional information obtained from the emotion engine, the server inputs a prompt message to the generative AI model saying, "Generate fun topics to post on social media when the user is happy," and retrieves the generated text.

[0426] Step 5: Execute Action

[0427] The server executes the generated actions. Specifically, the server posts the generated text to messenger apps or social media.

[0428] Input: Generated action text

[0429] Output: Actions performed (e.g., SNS post, message sent)

[0430] Specific operation: The server posts the generated text to messenger apps and social media. For example, when the user is happy, it posts a generated cheerful topic to social media, and when the user is sad, it sends generated words of comfort to their friends.

[0431] (Application Example 1)

[0432] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0433] Conventional information processing systems struggled to provide information and recommend products that adequately reflected user characteristics and emotions, thus failing to increase user satisfaction. Furthermore, users had to spend considerable time searching for and organizing information themselves, making efficient information acquisition difficult. Additionally, the inability to respond appropriately to user emotions led to a decline in the quality of the user experience.

[0434] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0435] This invention includes a server that includes means for learning user characteristics and processing information on behalf of the user, means for providing an avatar account that acts as a surrogate for the user on a messenger app, means for collaborating with a generative AI to pre-train the AI ​​with messenger app chat history and internet search tool purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user, means for recognizing the user's emotions and adjusting the avatar account's behavior based on those emotions, and means for recommending products based on the user's emotions. This makes it possible to provide information and recommend products that reflect the user's characteristics and emotions, thereby increasing user satisfaction. It also eliminates the effort the user has to spend searching for and organizing information themselves, enabling efficient information acquisition. Furthermore, it allows for appropriate responses in accordance with the user's emotions, thereby improving the quality of the user experience.

[0436] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and emotions.

[0437] "Information processing" refers to a series of operations such as data collection, organization, analysis, and provision.

[0438] A "messenger app" refers to a software application used to send and receive text messages, images, audio, and other data.

[0439] An "avatar account" refers to a virtual account that acts as a digital representation of a user online.

[0440] "Generative AI" refers to artificial intelligence that has the ability to generate new information and content based on data.

[0441] "Pre-training" refers to the process of training data in advance for a specific task.

[0442] "Emotion recognition" refers to technology that identifies emotions from a user's text, voice, and other data.

[0443] "Product recommendation" refers to the act of suggesting appropriate products based on the user's characteristics and emotions.

[0444] "Push marketing" refers to a method of automatically providing information or notifications to users before they request them.

[0445] The system for implementing this invention learns user characteristics and processes information on behalf of the user. Specifically, it provides an avatar account that acts as a surrogate for the user on a messenger app and pre-trains it with a generative AI using messenger app chat history and internet search tool purchase history. This realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[0446] Hardware and software to be used

[0447] Hardware: Smartphone

[0448] Software: Python, Transformers library

[0449] Processing flow

[0450] 1. Emotion Recognition: The user's input text is fed into an emotion recognition model (Transformers' sentiment-analysis pipeline) to recognize the user's emotions. This allows us to identify what emotions the user is currently experiencing.

[0451] 2. Product Recommendations: Based on recognized emotions, products are recommended based on the user's purchase and search history. For example, if the user is happy, special sale information is recommended; if they are sad, products that help them relax are recommended.

[0452] 3. Displaying Recommended Products: Display recommended products to the user. This makes it easy for users to find products that match their mood.

[0453] Specific example

[0454] If a user types "I'm feeling very happy today!", the app will recommend "Sale Item 1" and "Sale Item 2".

[0455] If a user enters "I'm feeling sad today," the app will recommend "Relaxation Product 1" or "Relaxation Product 2."

[0456] Example of a prompt

[0457] Create a Python program that recognizes user emotions and recommends products based on those emotions. The program will input user text into an emotion recognition model to recognize the user's emotions. Based on the recognized emotions, it will recommend products based on the user's purchase and search history. For example, if the user is happy, it will recommend special sale information; if they are sad, it will recommend relaxing products.

[0458] In this way, it becomes possible to provide information and recommend products that reflect the characteristics and emotions of users, thereby increasing user satisfaction. Furthermore, it eliminates the need for users to search for and organize information themselves, enabling efficient information acquisition. In addition, it allows for appropriate responses tailored to the user's emotions, improving the quality of the user experience.

[0459] The flow of a specific process in Application Example 1 will be explained using Figure 18.

[0460] Step 1:

[0461] The user enters text into the messenger app.

[0462] Input: User's text message

[0463] Output: Input text message

[0464] Specific action: The user enters text containing emotions, such as "I'm feeling very happy today!", into the messenger app.

[0465] Step 2:

[0466] The device sends the entered text message to the emotion recognition model.

[0467] Input: Entered text message

[0468] Output: Input data for the emotion recognition model

[0469] Specific operation: The terminal uses the Transformers library's sentiment-analysis pipeline to send the input text message to the sentiment recognition model.

[0470] Step 3:

[0471] The server uses an emotion recognition model to recognize the user's emotions.

[0472] Input: Input data for the emotion recognition model

[0473] Output: Recognized emotion (e.g., joy, sadness)

[0474] Specific operation: The server runs an emotion recognition model to determine the user's emotion from the input text message. For example, it recognizes the emotion "joy" from the text "I'm feeling very happy today!".

[0475] Step 4:

[0476] The server recommends products based on the emotions it perceives.

[0477] Input: Recognized emotions, user purchase history, search history

[0478] Output: Recommended product list

[0479] Specific operation: The server refers to the user's purchase and search history and selects appropriate products based on the recognized emotions. For example, for the emotion of "joy," it recommends products that include special sale information.

[0480] Step 5:

[0481] The server sends a list of recommended products to the terminal.

[0482] Input: Recommended Conlist

[0483] Output: Product list data to the terminal

[0484] Specific operation: The server generates a list of recommended products and sends it to the terminal.

[0485] Step 6:

[0486] The device displays a list of recommended products to the user.

[0487] Input: Product list data to the terminal

[0488] Output: Product list displayed to the user

[0489] Specific operation: The device displays a list of received products to the user, allowing the user to review the products. For example, "Sale Item 1" and "Sale Item 2" might be displayed.

[0490] By following these steps, product recommendations based on user emotions can be implemented, thereby increasing user satisfaction.

[0491] (Example 2)

[0492] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0493] In today's information society, users need to process and manage vast amounts of information. However, this is time-consuming and laborious, placing a burden on users. Furthermore, there is a lack of information provision based on users' emotions and preferences, failing to fully meet their needs. In addition, there is no means to process information on behalf of users when they are absent or busy, making efficient information management a challenge.

[0494] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0495] In this invention, the server includes means for learning user characteristics and processing information on behalf of the user; means for providing a virtual account that acts as a surrogate for the user on a communication application; means for collaborating with a generative AI to pre-train it with communication application talk history and e-commerce tool purchase history to realize a "conversational" and "push-type" virtual account with the same characteristics as the user; and means for analyzing the user's emotions using an emotion analysis engine and adjusting information provision and search results based on those emotions. This reduces the burden on the user, enables information provision based on the user's emotions and preferences, and realizes efficient information management.

[0496] "User characteristics" refer to individual features such as a user's behavioral patterns, preferences, and emotions.

[0497] "Information processing" refers to a series of operations such as data collection, analysis, organization, and provision.

[0498] A "communication application" refers to software used to send and receive messages between users.

[0499] A "virtual account" refers to a digital proxy account created to process information on behalf of a user.

[0500] "Generative AI" refers to artificial intelligence that has the ability to generate new information or responses based on data.

[0501] "Pre-training" refers to the process of training a model in advance using relevant data before performing a specific task.

[0502] An "emotion analysis engine" refers to software that analyzes a user's emotions from data such as text and audio.

[0503] "Information provision" refers to the act of presenting necessary information to users.

[0504] "Search results" refer to the list of information that a user obtains by using a search engine.

[0505] "Push marketing" refers to a method of automatically providing information before the user requests it.

[0506] This invention is a system that learns user characteristics and processes information on behalf of the user. Specifically, it provides a virtual account that acts as a surrogate for the user on a communication application and learns the user's characteristics in cooperation with a generative AI. It also analyzes the user's emotions using an emotion analysis engine and adjusts the information provided and search results based on those emotions.

[0507] Hardware and software to be used

[0508] The server uses the following hardware and software:

[0509] Hardware: High-performance server machine

[0510] Software: Python, pandas, numpy, generative AI (e.g., GPT-4), sentiment analysis engine (e.g., IBM Watson's Tone Analyzer)

[0511] Data collection and preprocessing

[0512] The server retrieves conversation history using the user's communication application API and purchase history using the e-commerce tool API. This data is stored in a secure database. Next, the data is preprocessed using the Python pandas library, specifically by removing unnecessary information and normalizing the data.

[0513] Feature extraction and model learning

[0514] The server extracts features from pre-processed data. For example, it uses natural language processing (NLP) techniques to extract user tone and frequently used phrases from conversation history. From purchase history, it analyzes user purchasing patterns and preferences. Next, it uses generative AI (GPT-4) to learn user characteristics. The pre-processed data and extracted features are input into the model to learn user conversation patterns and purchasing tendencies.

[0515] Creating a virtual account

[0516] The server generates a virtual account on the communication application that possesses the user's characteristics based on the learning results. Using the communication application's API, a new account is created and configured to reflect the user's characteristics. This virtual account can post information, send messages, and search for information on behalf of the user.

[0517] Emotion analysis and information provision

[0518] The server uses an emotion analysis engine to analyze the user's emotions. When a user sends a message through a communication application, the server analyzes the tone of the message and provides relaxation information if the user is feeling stressed. If the user is feeling happy, it provides search results that reflect that emotion, such as enjoyable events and news.

[0519] Specific example

[0520] User A provides conversation history with friend B via a communication application and purchase history via an e-commerce tool. The server collects this data and learns user A's characteristics using a generative AI (GPT-4). After learning is complete, the server creates a virtual account with user A's characteristics on the communication application. This virtual account can continue the conversation with friend B on behalf of user A.

[0521] Example of a prompt

[0522] "Based on User A's conversation history in communication applications and purchase history in e-commerce tools, learn User A's characteristics and create a virtual account that will engage in conversations and post information on User A's behalf."

[0523] In this way, the burden on users is reduced, information can be provided based on users' emotions and preferences, and efficient information management can be achieved.

[0524] The flow of the specific processing in Example 2 will be explained using Figure 19.

[0525] Step 1:

[0526] The server retrieves the conversation history using the user's communication application API. It takes the user's authentication information and API endpoint as input and obtains conversation history data as output. Specifically, the server sends an API request and receives the conversation history data in JSON format as a response.

[0527] Step 2:

[0528] The server retrieves purchase history using the API of an e-commerce tool. It uses user authentication information and the API endpoint as input and obtains purchase history data as output. Specifically, the server sends an API request and receives purchase history data in JSON format as a response.

[0529] Step 3:

[0530] The server preprocesses the acquired conversation history and purchase history data. It takes conversation history data and purchase history data as input and obtains preprocessed data as output. Specifically, the server uses the Python pandas library to remove unnecessary information and normalize the data.

[0531] Step 4:

[0532] The server extracts features from pre-processed data. It uses pre-processed conversation history data and purchase history data as input, and outputs extracted feature data. Specifically, the server uses natural language processing (NLP) techniques to extract conversational tones and frequently used phrases, and analyzes purchase patterns.

[0533] Step 5:

[0534] The server learns user characteristics using generative AI (GPT-4). It uses extracted feature data as input and obtains a trained model as output. Specifically, the server inputs feature data into the model and learns user conversation patterns and purchasing tendencies.

[0535] Step 6:

[0536] The server generates a virtual account on the communication application that possesses the user's characteristics based on the training results. It uses the trained model and the communication application's API endpoint as input, and obtains the virtual account as output. Specifically, the server sends an API request, creates a new account, and configures it to have the user's characteristics.

[0537] Step 7:

[0538] The server analyzes the user's emotions using an emotion analysis engine. It takes the user's message data as input and obtains the emotion analysis results as output. Specifically, the server inputs the message data into the emotion analysis engine and analyzes the emotional tone.

[0539] Step 8:

[0540] The server adjusts the information provided and search results based on the sentiment analysis results. It uses sentiment analysis results and user characteristic data as input, and outputs adjusted information and search results. Specifically, the server provides relaxation information and fun event information based on the sentiment analysis results.

[0541] (Application Example 2)

[0542] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0543] Traditional systems struggled to provide information and recommend content that adequately reflected user characteristics and emotions, making it difficult to offer services that met user needs. Furthermore, creating avatar accounts that could perform tasks like SNS posting, messaging, and web searches on behalf of users was challenging. This resulted in decreased user convenience and satisfaction.

[0544] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0545] This invention includes a server that learns user characteristics and performs SNS, web searches, and information organization on behalf of the user; a server that provides an avatar account representing the user on a messenger app; a server that collaborates with a generative AI to pre-train the AI ​​with messenger app chat history and internet search tool shopping purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; a server that analyzes the user's emotions and adjusts information provision and search results based on those emotions; and a server that recommends content based on the user's characteristics and emotions. This makes it possible to provide information and recommend content that reflects the user's characteristics and emotions, and to realize an avatar account that performs SNS posting, message sending, and web searches on behalf of the user.

[0546] "User characteristics" refer to the individual user's preferences, tastes, and behavioral patterns extracted from their activity history, purchase history, conversation history, etc.

[0547] An "avatar account" refers to a virtual account that automatically performs actions such as posting on social media, sending messages, and performing web searches on behalf of the user.

[0548] "Generative AI" refers to artificial intelligence technology that generates new information and content based on user data.

[0549] "Pre-learning" refers to the process by which generative AI learns the user's characteristics based on the user's past behavior and purchase history.

[0550] "Sentiment analysis" refers to a technology that analyzes a user's emotional state based on their conversation history and behavioral data.

[0551] "Information provision" refers to the act of providing information that is appropriate for the user, based on the user's characteristics and emotions.

[0552] "Content recommendation" refers to the act of recommending content that is suitable for a user based on their characteristics and emotions.

[0553] "Push marketing" refers to a method of automatically providing information or content to users before they request it.

[0554] The system for implementing this invention has the function of learning user characteristics and performing tasks such as SNS, web searches, and information organization on behalf of the user. The system provides an avatar account that acts as a surrogate for the user on a messenger app and pre-trains the avatar account in conjunction with a generative AI using the user's messenger app chat history and internet search tool shopping purchase history. This realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[0555] The server has the ability to analyze user emotions and adjust the information provided and search results based on those emotions. Furthermore, it also has the ability to recommend content based on user characteristics and emotions.

[0556] Hardware and software to be used

[0557] Hardware: Servers, user terminals (smartphones, tablets, PCs, etc.)

[0558] Software: Messenger apps, generative AI models (e.g., OpenAI GPT-3), sentiment analysis libraries (e.g., NLTK, TextBlob), content recommendation engines (e.g., TENSORFLOW® Recommenders)

[0559] Data processing and data calculation

[0560] 1. Data Acquisition: The user's device sends conversation history from messenger apps and purchase history from internet search tools to the server.

[0561] 2. Learning User Characteristics: The server uses a generative AI model to learn user characteristics from the acquired data.

[0562] 3. Sentiment Analysis: The server uses a sentiment analysis library to analyze the user's emotions from their conversation history.

[0563] 4. Information Provision and Content Recommendation: The server provides appropriate information and content to the user's device based on the user's characteristics and emotions.

[0564] Specific example

[0565] For example, if a user tells a friend via a messenger app that they are stressed out because work has been so busy lately, the server retrieves this conversation history and uses a sentiment analysis library to analyze whether the user is experiencing stress. Furthermore, it checks the user's purchase history to see if they have bought relaxation-related books and uses a generative AI model to learn the user's characteristics.

[0566] Example of a prompt

[0567] User's conversation history: I've been really busy with work lately and it's been stressing me out.

[0568] User's purchase history: Purchased books related to relaxation.

[0569] Learn about this user's characteristics.

[0570] By inputting this prompt into a generative AI model, it can learn the user's characteristics and recommend appropriate relaxation content.

[0571] The flow of a specific process in Application Example 2 will be explained using Figure 20.

[0572] Step 1:

[0573] The user's device sends the conversation history from the messenger app and the purchase history from the internet search tool to the server.

[0574] Input: User conversation history, purchase history

[0575] Output: User data sent to the server

[0576] Specific operation: The user's device retrieves conversation history using the messenger app's API and purchase history using the internet search tool's API. This data is then sent to the server.

[0577] Step 2:

[0578] The server inputs the received user data into a generative AI model to learn the user's characteristics.

[0579] Input: User data sent to the server

[0580] Output: Learned user characteristics

[0581] Specific operation: The server generates prompts for a generative AI model (e.g., OpenAI GPT-3) and learns the user's characteristics based on the user's conversation history and purchase history. An example of a prompt is: "User's conversation history: I've been stressed out lately because I've been busy with work. User's purchase history: I bought a book about relaxation. Please learn the characteristics of this user."

[0582] Step 3:

[0583] The server uses a sentiment analysis library to analyze the user's emotions from their conversation history.

[0584] Input: User conversation history

[0585] Output: Analyzed user emotional state

[0586] Specific operation: The server uses sentiment analysis libraries (e.g., NLTK, TextBlob) to analyze the user's conversation history and identify the emotions the user is feeling (e.g., stress, joy).

[0587] Step 4:

[0588] The server recommends appropriate information and content based on the user's characteristics and emotions.

[0589] Input: Learned user characteristics, analyzed user emotional state

[0590] Output: Recommended information and content

[0591] Specific operation: The server uses a content recommendation engine (e.g., TensorFlow Recommenders) to select appropriate information and content based on the user's characteristics and emotions, and sends it to the user's device.

[0592] Step 5:

[0593] The user terminal displays recommendation information and content received from the server.

[0594] Input: Recommended information or content

[0595] Output: Information and content displayed to the user

[0596] Specific operation: The user's device displays information and content received from the server on a messenger app or browser and provides it to the user.

[0597] (Example 3)

[0598] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0599] Traditional information systems and search engines have failed to adequately reflect user characteristics and preferences, making it difficult to provide users with the most relevant information. Furthermore, they have not incorporated information delivery or automated actions that consider user emotions, thus failing to increase user satisfaction.

[0600] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for providing an avatar account that represents the user, means for coordinating with a generative AI to pre-train the user's behavior history and realize an avatar account with the same characteristics as the user, and means for analyzing the user's emotions in real time using an emotion recognition engine and automatically performing actions of the avatar account based on the results. This makes it possible to provide information and generate search results that reflect the user's characteristics and preferences, and further enables the automation of actions that take the user's emotions into consideration.

[0601] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and interests.

[0602] An "avatar account" refers to a virtual account that acts as a digital representation of a user online.

[0603] "Generative AI" refers to artificial intelligence that has the ability to generate new information and content based on data.

[0604] "Pre-learning" refers to the process of pre-training the system with user behavior history and data.

[0605] An "emotion recognition engine" refers to a technology that analyzes a user's emotions based on their facial expressions, tone of voice, and other factors.

[0606] "Real-time" refers to the instantaneous collection and processing of data.

[0607] "Information provision" refers to the act of presenting useful information to users.

[0608] "Search results" refer to the list of related information displayed when a user performs a search.

[0609] "Action automation" refers to a system automatically performing specific actions on behalf of the user.

[0610] This invention is a system that learns user characteristics and provides information and generates search results on behalf of the user. The system provides an avatar account that acts as a surrogate for the user and pre-trains the avatar account with the user's behavioral history in cooperation with a generative AI. It also uses an emotion recognition engine to analyze the user's emotions in real time and automatically performs actions for the avatar account based on the results.

[0611] Hardware and software to be used

[0612] Hardware: Servers, terminals (PCs, smartphones)

[0613] Software: Generative AI models (e.g., GPT-4), emotion recognition engines (e.g., Affectiva), social media APIs (e.g., Twitter API, Facebook API)

[0614] Data processing and data calculation

[0615] 1. Data collection:

[0616] The server collects data such as users' social media posts, message transmissions, and web search history. Specifically, it uses the Twitter API and Facebook API to retrieve users' posts and search history.

[0617] The device collects real-time emotional data from the user using an emotion recognition engine. It analyzes the user's facial expressions and voice tone using the device's camera and microphone.

[0618] 2. Data Analysis:

[0619] The server inputs the collected data into an AI model to learn user characteristics and preferences. Specifically, it analyzes users' posts and search history to identify topics of interest.

[0620] The generative AI model learns user characteristics based on the input data and generates a profile.

[0621] 3. Information provision and search result generation:

[0622] The server generates appropriate information and search results based on the user's learned characteristics and preferences. Specifically, it provides news articles and product information that the user is likely to be interested in.

[0623] The generative AI model generates relevant information based on the user's profile.

[0624] 4. Emotion Recognition and Avatar Account Behavior:

[0625] The device uses an emotion recognition engine to analyze the user's emotions in real time and sends the results to the server.

[0626] Based on the results of the emotion recognition engine, the server instructs the avatar account to automatically perform actions that reflect the user's emotions.

[0627] Specific example

[0628] Example 1: When a user frequently posts on social media about "travel"

[0629] The server learns the user's travel preferences and prioritizes providing travel-related information and search results.

[0630] Example prompt for a generative AI model: "This user frequently posts about travel. Please provide travel-related information."

[0631] Example 2: When the user is angry

[0632] The device uses an emotion recognition engine to detect when the user is feeling angry and sends the result to the server.

[0633] The server sends an instruction to the avatar account saying, "The user is feeling angry. Please generate a social media post that reflects that anger."

[0634] The generation AI model creates "SNS posts for angry users," and the avatar account automatically posts them.

[0635] In this way, the system takes into account the user's characteristics and emotions to provide optimal information and automate actions. The flow of a specific process in Example 3 will be explained using Figure 21.

[0636] Step 1: Data Collection

[0637] The server collects data such as users' social media posts, message transmissions, and web search history. Specifically, it uses the Twitter API and Facebook API to retrieve users' posts and search history.

[0638] Input: User data obtained from the SNS API (post content, search history, etc.)

[0639] Output: Collected user data

[0640] Specific operation: The server periodically calls the SNS API to retrieve the user's latest posts.

[0641] Step 2: Collecting emotional data

[0642] The device collects real-time emotional data from the user using an emotion recognition engine. It analyzes the user's facial expressions and voice tone using the device's camera and microphone.

[0643] Input: Real-time user facial and voice data acquired from the device's camera and microphone.

[0644] Output: User emotion data analyzed by the emotion recognition engine

[0645] Specific operation: The device collects facial expression data via the camera while the user is using the smartphone and sends it to the emotion recognition engine.

[0646] Step 3: Data Analysis

[0647] The server inputs the collected data into an AI model to learn user characteristics and preferences. Specifically, it analyzes users' posts and search history to identify topics of interest.

[0648] Input: Collected user data (post content, search history, etc.)

[0649] Output: A profile that reflects the user's characteristics and preferences.

[0650] Specific operation: The server inputs the collected SNS post data into a generating AI model and analyzes it using the prompt message, "What topics do users frequently post about?"

[0651] Step 4: Information provision and search result generation

[0652] The server generates appropriate information and search results based on the user's learned characteristics and preferences. Specifically, it provides news articles and product information that the user is likely to be interested in.

[0653] Input: A profile that reflects the user's characteristics and preferences.

[0654] Output: Information and search results provided to the user

[0655] Specific operation: Based on a profile indicating that the user is interested in travel, the server searches for travel-related news articles and provides them to the user.

[0656] Step 5: Emotion Recognition and Avatar Account Behavior

[0657] The device uses an emotion recognition engine to analyze the user's emotions in real time and sends the results to the server.

[0658] Based on the results of the emotion recognition engine, the server instructs the avatar account to automatically perform actions that reflect the user's emotions.

[0659] Input: User emotion data analyzed by the emotion recognition engine

[0660] Output: Automated actions performed by the avatar account (e.g., social media posts, message sending)

[0661] Specific operation: The device detects that the user is feeling angry using an emotion recognition engine and sends the result to the server. The server sends an instruction to the avatar account saying, "The user is feeling angry. Please generate a social media post that reflects that anger." The generation AI model generates a social media post for the angry user, and the avatar account automatically posts it.

[0662] (Application Example 3)

[0663] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0664] Traditional avatar account systems were insufficient in providing information and generating search results that reflected user preferences and tastes, and were unable to automatically perform actions based on user emotions. Furthermore, they lacked the ability to recommend optimal content based on user emotions and preferences, making improving the user experience a challenge.

[0665] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for learning user characteristics and performing SNS / web searches / information organization on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on a messenger app; means for collaborating with a generative AI to pre-train the AI ​​with messenger app talk history and internet search tool shopping purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; means for recognizing the user's emotions in real time using an emotion engine and automatically performing actions of the avatar account based on those emotions; and means for automatically recommending optimal content based on the user's emotions and preferences. This makes it possible to provide information, generate search results, and recommend optimal content based on the user's emotions and preferences.

[0666] "User characteristics" refer to individual features of a user, such as their behavioral history, preferences, tastes, and emotions.

[0667] An "avatar account" refers to a virtual account that acts as a digital representation of a user online.

[0668] "Generative AI" refers to artificial intelligence that learns user behavior and characteristics and generates information based on that.

[0669] "Messenger app chat history" refers to a user's conversation history within a messenger app.

[0670] "Internet search tool shopping purchase history" refers to the shopping history of a user who has used an internet search tool.

[0671] "Pre-training" refers to the process by which generative AI learns in advance using data from users' past behavior.

[0672] An "emotion engine" refers to a system that recognizes a user's emotions in real time and makes decisions based on those emotions.

[0673] "Content" refers to information and entertainment such as videos, articles, and music.

[0674] "Recommendation" refers to the act of selecting and presenting the most suitable content based on the user's characteristics and emotions.

[0675] The system for implementing this invention has the ability to learn user characteristics and perform tasks such as SNS, web searches, and information organization on behalf of the user. Specifically, it provides an avatar account that acts as a surrogate for the user on a messenger app and pre-trains it using messenger app chat history, internet search tool shopping purchase history, etc., in conjunction with a generative AI. Through this pre-training, it realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[0676] Furthermore, it uses an emotion engine to recognize the user's emotions in real time and automatically controls the avatar account's actions based on those emotions. It also includes a feature that automatically recommends the most suitable content based on the user's emotions and preferences.

[0677] Program Processing Description

[0678] The server runs sentiment recognition and content recommendation programs using Python. TextBlob is used for sentiment recognition to analyze user text data. To learn user preferences, the text data is vectorized using scikit-learn's TfidfVectorizer, and the optimal content is recommended by calculating cosine similarity.

[0679] Hardware and software to be used

[0680] Hardware: Smartphone

[0681] Software: Python, TextBlob, scikit-learn

[0682] Specific example

[0683] For example, if a user posts "I'm so tired today," sentiment analysis will detect negative emotions. In this case, the system will recommend relaxing music or videos.

[0684] Example of a prompt

[0685] User post: "I'm so tired today."

[0686] User preferences: ["Likes movies", "Likes music", "Likes traveling"]

[0687] Recommended content: ["Relaxing music", "Exciting movies", "Travel blogs"]

[0688] By inputting this prompt into the AI ​​generation model, it is possible to recommend the most suitable content based on the user's emotions and preferences.

[0689] The flow of the specific processing in Application Example 3 will be explained using Figure 22.

[0690] Step 1:

[0691] The server collects the user's messenger app chat history and internet search tool shopping purchase history. This data is entered as the user's behavioral history. The server stores this data in text format in preparation for later processing.

[0692] Step 2:

[0693] The server learns user characteristics using the collected data. Specifically, it uses the Python scikit-learn library and TfidfVectorizer to vectorize text data. This vectorized data is output as features representing user preferences.

[0694] Step 3:

[0695] The server uses TextBlob to recognize user emotions in real time. It receives text data posted by users in messenger apps as input and performs emotion analysis using TextBlob. As a result of the emotion analysis, positive, negative, or neutral emotion scores are output.

[0696] Step 4:

[0697] The server uses an emotion engine to determine the avatar account's actions based on the user's emotions. It receives an emotion score as input and is configured to recommend relaxing content for negative emotions and exciting content for positive emotions. This configuration is output as a guideline for the avatar account's actions.

[0698] Step 5:

[0699] The server recommends the most suitable content based on the user's preferences and emotions. Specifically, it takes features representing the user's preferences and an emotion score as input, and uses scikit-learn's cosine similarity calculation to select the most appropriate content. This recommended content is then output as information provided to the user.

[0700] Step 6:

[0701] The device displays content recommended by the server to the user. The user can view the recommended content on their smartphone screen and use it as needed. In this step, the recommended content is displayed on the user's device.

[0702] Step 7:

[0703] Users engage with recommended content and submit feedback to the server. The server receives this feedback and stores it in a database to inform future recommendations. This feedback is used to further learn about user characteristics.

[0704] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0705] The data generation model 58 is a form of so-called generative AI (Artificial Intelligence).

[0706] One example of a data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. Data generation model 58 is

[0707] This is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images.

[0708] The data generation model 58 performs inference on the input inference data according to instructions indicated by prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0709] Other examples of generative AI include Gemini® (registered trademark) (Internet search). <url: https: gemini.google.com ?hl="ja">) are some examples.

[0710] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0711] [Second Embodiment]

[0712] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0713] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0714] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0715] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0716] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0717] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0718] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0719] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0720] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0721] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0722] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0723] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0724] "Example of form 1"

[0725] One embodiment of the present invention provides a system that learns user characteristics and performs tasks such as SNS, web searches, and information organization on behalf of the user. Specifically, an avatar account that acts as a surrogate for the user is provided on a messenger app. This avatar account pre-learns the user's messenger app chat history and internet search tool shopping purchase history, and by linking with a generative AI, it realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[0726] "Example of form 2"

[0727] As a concrete example, user A provides pre-training data such as their conversation history with friend B on a messenger app and their Amazon purchase history. Based on this data, the system works in conjunction with a generative AI to learn user A's characteristics. After learning is complete, the system provides an avatar account on the messenger app that acts as a digital representation of user A. This avatar account has the same characteristics as user A and is a "conversational," "push-type" avatar account that can perform actions such as posting on social media, sending messages, and performing web searches on behalf of user A.

[0728] "Example of form 3"

[0729] Furthermore, in another embodiment of the present invention, a function is provided to generate information and search results that reflect the user's preferences and tastes. Specifically, an avatar account is provided that automatically performs actions such as posting on social media, sending messages, and performing web searches. This avatar account can learn the user's characteristics and generate information and search results that reflect them.

[0730] The following describes the processing flow for each example of the form.

[0731] "Example of form 1"

[0732] Step 1: The user provides the system with their chat history with friends via a messenger app and their Amazon purchase history.

[0733] Step 2: Based on the provided data, the system works in conjunction with generative AI to learn the user's characteristics.

[0734] Step 3: After learning is complete, the system will provide the user with an avatar account on the messenger app, which will serve as their digital counterpart.

[0735] Step 4: This avatar account becomes a "conversational" and "push-type" avatar account with the same characteristics as the user, and can perform actions such as posting on social media, sending messages, and performing web searches on behalf of the user.

[0736] "Example of form 2"

[0737] Step 1: Users provide the system with behavioral data such as social media posts, message sending, and web searches.

[0738] Step 2: The system learns the user's characteristics based on the provided data.

[0739] Step 3: After learning is complete, the system will provide an avatar account that will serve as a digital representation of the user.

[0740] Step 4: This avatar account can generate information and search results that reflect the user's characteristics.

[0741] (Example 1)

[0742] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0743] In today's information society, users need to efficiently collect and organize vast amounts of information. However, doing so manually is time-consuming, laborious, and inefficient. Furthermore, there is a demand for information tailored to user characteristics and preferences, but conventional systems struggle to adequately achieve this. Additionally, there is a lack of means to centrally manage and utilize the history and characteristics of multiple information and communication services and search tools used by users. A system is needed to address these challenges.

[0744] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0745] In this invention, the server includes means for learning user characteristics and performing information communication services, information retrieval, and information organization on behalf of the user; means for providing a virtual account that acts as a surrogate for the user on an information communication application; means for collaborating with a generative AI to pre-train it on the history of information communication applications, internet search tools, and purchase history to realize a "conversational" and "push-type" virtual account with the same characteristics as the user; means for pre-processing collected data and extracting user characteristics; means for training a generative AI model using the extracted characteristics; and means for collecting relevant information based on the user's interests and preferences and sending push notifications through the virtual account. This makes it possible to provide information and generate search results based on the user's characteristics and preferences, and can significantly improve the efficiency of the user's information gathering and organization.

[0746] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and interests.

[0747] "Information and communication services" refers to communication methods such as messaging, social networking services (SNS), and email that are provided via the internet.

[0748] "Information retrieval" refers to the act of finding specific information on the internet.

[0749] "Information organization" refers to the act of classifying, organizing, and making usable information.

[0750] An "information and communication application" refers to software that allows users to send and receive messages and share information.

[0751] A "virtual account" refers to a digital avatar that operates on behalf of a user within an information and communication application.

[0752] "Generative AI" refers to artificial intelligence technology that learns user characteristics and generates responses and actions similar to those of the user.

[0753] "Pre-training" refers to the process of training a generative AI model in advance using collected data.

[0754] "Conversational" means that a virtual account has the ability to engage in natural conversations with the user.

[0755] "Push notifications" refer to a function that automatically notifies users of relevant information based on their interests and preferences.

[0756] "Preprocessing" refers to the process of preparing collected data into a format that is easy to analyze.

[0757] "Feature extraction" refers to the process of extracting characteristics such as user interests and preferences from pre-processed data.

[0758] "Training" refers to the process of using extracted characteristics to train a generative AI model.

[0759] "Push notifications" refer to a function that sends information to users in real time.

[0760] This invention is a system that learns user characteristics and performs information communication services, information retrieval, information organization, etc., on behalf of the user. Specific embodiments of this system are described below.

[0761] 1. System Overview

[0762] The server learns user characteristics and provides a system to perform information communication services, information retrieval, and information organization on behalf of the user. This system provides a virtual account that acts as a surrogate for the user on the information communication application and operates in conjunction with a generative AI.

[0763] 2. Hardware and software to be used

[0764] The server uses the following hardware and software:

[0765] Hardware: High-performance server machines, database servers

[0766] Software: Information and communication applications, generative AI models, data analysis tools, push notification systems

[0767] 3. Data Collection and Pre-training

[0768] The server collects the user's information and communication application history, internet search tool history, and purchase history. This data serves as foundational data for learning user characteristics. The collected data is preprocessed, with unnecessary data being removed and data normalization performed.

[0769] Next, user characteristics are extracted from the pre-processed data. Text data is analyzed using natural language processing techniques to identify user interests and preferences. A generative AI model is then trained using these extracted characteristics. This pre-training allows the generative AI model to understand user characteristics and generate responses similar to those of the user.

[0770] 4. Creating and configuring virtual accounts

[0771] The server uses a pre-trained generative AI model to generate a virtual account that reflects the user's characteristics. This virtual account can converse on behalf of the user within information and communication applications. Furthermore, the virtual account has the functionality to push relevant information based on the user's interests and preferences.

[0772] 5. Specific Examples

[0773] Example 1: Travel planning

[0774] When a user is planning a trip with a friend using an information and communication application, the server provides information on suitable destinations and accommodations based on past travel and search history. For example, it might suggest places the user has visited in the past that their friend might be interested in.

[0775] Example 2: Providing shopping information

[0776] If a user is interested in products from a particular brand, the server collects information about new products from that brand and sends push notifications through a virtual account. For example, it might immediately notify the user when a new product from a brand they have previously purchased is released.

[0777] 6. Example of a prompt statement

[0778] "Create a virtual account that provides information about travel destinations the user is interested in, based on their information and communication application history and internet search history. Also, implement a function to send push notifications with the latest information about those travel destinations."

[0779] In this way, a system is realized that learns the characteristics of users and performs information and communication services, information retrieval, and information organization on behalf of the user.

[0780] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0781] Step 1: Data Collection

[0782] The server collects the user's information and communication application history, internet search tool history, and purchase history.

[0783] Input: User's information, communication application history, internet search history, purchase history

[0784] Data processing: The collected data is stored in a database for centralized management.

[0785] Output: User activity history data stored in the database

[0786] Specific operation: The server retrieves data from information and communication applications and internet search tools via APIs and stores it in a database.

[0787] Step 2: Data preprocessing

[0788] The server preprocesses the collected data.

[0789] Input: User activity history data stored in the database

[0790] Data processing: Deletion of unnecessary data, data normalization, tokenization of text data.

[0791] Output: Preprocessed clean dataset

[0792] Specific operation: The server uses a data cleansing tool to remove noisy data and prepares the text data in a format that is easy to analyze.

[0793] Step 3: Feature Extraction

[0794] The server extracts user characteristics from the pre-processed data.

[0795] Input: Preprocessed clean dataset

[0796] Data Processing: Analyze text data using natural language processing techniques to identify user interests and preferences.

[0797] Output: Feature vector representing user characteristics

[0798] Specific operation: The server uses a natural language processing library to analyze text data and extract user interests and preferences.

[0799] Step 4: Training the Generative AI Model

[0800] The server uses the extracted features to train a generative AI model.

[0801] Input: Feature vector representing user characteristics

[0802] Data processing: Input feature vectors into a generative AI model and train the model.

[0803] Output: Generative AI model that learned user characteristics

[0804] Specific operation: The server trains a generative AI model using a machine learning framework to learn the user's characteristics.

[0805] Step 5: Create a virtual account

[0806] The server uses a pre-trained generative AI model to generate virtual accounts that reflect the user's characteristics.

[0807] Input: Generative AI model that has learned user characteristics

[0808] Data processing: Create virtual accounts using a generative AI model and register them in an information and communication application.

[0809] Output: Virtual account on information and communication application

[0810] Specific operation: The server generates virtual accounts using a generated AI model and registers them with the information and communication application via an API.

[0811] Step 6: Implementing push notifications

[0812] The server collects relevant information based on the user's interests and sends push notifications through a virtual account.

[0813] Input: Data on user interests and preferences, external information sources

[0814] Data processing: Collect relevant information and format it in a way that is suitable for the user.

[0815] Output: Push notification message

[0816] Specific operation: The server collects relevant information from external information sources and sends push notifications to the user through a virtual account.

[0817] In this way, a system is realized that learns the characteristics of users and performs information and communication services, information retrieval, and information organization on behalf of the user.

[0818] (Application Example 1)

[0819] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0820] Modern consumers find it difficult and time-consuming to find the best product for them from the vast amount of information available. Furthermore, the lack of personalized product recommendations based on user characteristics and preferences makes it difficult for consumers to find suitable products. Additionally, the absence of systems that handle social media, web searches, and information organization on behalf of users means that users must gather information themselves, posing a significant challenge.

[0821] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0822] In this invention, the server includes means for learning user characteristics and performing SNS, web searches, information organization, etc. on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on a messenger app; means for collaborating with a generative AI to pre-train the AI ​​with messenger app chat history and internet search tool shopping purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; means for learning the user's purchase history and search history and automatically suggesting products that match the user's preferences; and means for suggesting products based on the user's history using a generative AI model. As a result, users can easily find products that are best suited to them and can save time and effort in gathering information.

[0823] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and purchase history.

[0824] "SNS" is an abbreviation for Social Networking Service, which is an online platform for users to share information and communicate with other users.

[0825] "Web search" refers to the act of finding information on the internet using a search engine.

[0826] "Information organization" refers to the act of classifying collected information and compiling it into an easily understandable format.

[0827] A "messenger app" is application software that allows users to send and receive text messages, images, audio, and other data.

[0828] An "avatar account" is a virtual account that functions as a substitute for the user, handling communication and information gathering on their behalf.

[0829] "Generative AI" refers to artificial intelligence technology that generates new information and suggestions based on user input and historical data.

[0830] "Pre-learning" refers to the process by which a system learns from a user's past behavior history and data in advance.

[0831] "Push-type" refers to a system that automatically provides information or suggestions before the user requests them.

[0832] "Purchase history" refers to a record of products that a user has purchased in the past.

[0833] "Search history" refers to a record of keywords and phrases that a user has previously searched for on a search engine.

[0834] A "generative AI model" refers to an artificial intelligence algorithm that generates new information and suggestions based on a user's historical data.

[0835] As an embodiment of this invention, a system is provided that learns user characteristics and performs SNS, web searches, information organization, etc., on behalf of the user. Specifically, an avatar account that acts as a surrogate of the user is provided on a messenger app, and in cooperation with a generative AI, the avatar is pre-trained with the user's messenger app chat history, internet search tool, and shopping purchase history. This avatar account realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[0836] The server has the ability to learn from the user's purchase and search history and automatically suggest products that match the user's preferences. It uses a generative AI model to suggest products based on the user's history. This system makes it easy for users to find the products that are best suited to them, saving them the trouble of gathering information themselves.

[0837] Hardware and software to be used

[0838] Hardware: Servers, user terminals (smartphones, tablets, PCs, etc.)

[0839] Software: Messenger apps, generative AI models (e.g., OpenAI GPT-3)

[0840] Data processing and data calculation

[0841] The server collects the user's messenger app chat history, internet search tool data, and shopping purchase history, and uses this data for pre-training. Using a generative AI model, it analyzes the user's historical data and learns the user's characteristics and preferences. This allows it to automatically suggest products that match the user's preferences.

[0842] Specific example

[0843] If a user has previously purchased a "smartphone" or "wireless earphones" and has searched for "the latest smartphone" or "high-quality earphones," the generative AI model will suggest products related to "the latest smartphone" or "high-quality earphones."

[0844] Example of a prompt

[0845] User purchase history: ["Smartphone", "Wireless earphones"]

[0846] User's search history: ["Latest smartphone", "High-quality earphones"]

[0847] Please suggest products that would be suitable for this user.

[0848] In this way, a personal shopping assistant can be realized that automatically suggests products that match the user's preferences.

[0849] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0850] Step 1:

[0851] The server collects users' messenger app chat history, internet search tool data, and shopping purchase history. This data serves as input for learning user characteristics and preferences. Specifically, it retrieves data from each platform via APIs and stores it in a database.

[0852] Input: Messenger app chat history, internet search tools, shopping purchase history

[0853] Output: User history data stored in the database

[0854] Step 2:

[0855] The server pre-trains on the collected user history data. In this pre-training process, a generative AI model is used to analyze the data and learn user characteristics and preferences. Specifically, the history data is input into the AI ​​model to extract user behavior patterns and preferences.

[0856] Input: User history data stored in the database

[0857] Output: A model that reflects the user's characteristics and preferences.

[0858] Step 3:

[0859] The server uses a generative AI model to suggest products based on the user's history. In this process, prompt sentences are generated based on the user's purchase and search history to suggest the most suitable products, and these are input into the AI ​​model. The AI ​​model then generates product suggestions based on these prompt sentences.

[0860] Input: A model that reflects the user's characteristics and preferences, and prompt text.

[0861] Output: Product proposal

[0862] Step 4:

[0863] The server sends the generated product suggestions to the user's device. Users can then view the suggested products through a messenger app or a dedicated app. Specifically, product suggestions are sent to the user's device via an API and displayed in the user interface.

[0864] Input: Product proposal

[0865] Output: Product suggestions displayed on the user's terminal

[0866] Step 5:

[0867] Users review the suggested products and purchase them if necessary. This process involves users receiving product suggestions, selecting items of interest, and proceeding with the purchase. Specifically, they select products through the user interface and click the purchase button.

[0868] Input: Product suggestions displayed on the user's terminal

[0869] Output: User purchasing behavior

[0870] In this way, the system automatically suggests products based on the user's characteristics and preferences, enabling users to easily find the products that are best suited to them.

[0871] (Example 2)

[0872] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0873] Conventional information processing systems struggled to provide automated information and communication that fully reflected user characteristics. Furthermore, creating avatar accounts that could perform tasks such as SNS posting, message sending, and web searches on behalf of users was also difficult. This resulted in increased user burden and inefficient information processing.

[0874] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for learning user characteristics and processing information on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on the messenger app; means for coordinating with a generative AI model to pre-train it with the conversation history of the messenger app and the purchase history of online shopping to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; means for pre-processing data stored in the database, cleaning and tokenizing text data; and means for inputting prompt sentences to the generative AI model and sending the generated message. This enables automated information provision and communication that reflects the user's characteristics.

[0875] "User characteristics" refer to individual features such as user behavior patterns, preferences, and tastes.

[0876] "Information processing" refers to a series of operations such as data collection, analysis, storage, retrieval, and generation.

[0877] An "avatar account" refers to a virtual account that operates online on behalf of a user.

[0878] A "generative AI model" refers to an artificial intelligence model that generates new information or content based on given data.

[0879] "Pre-training" refers to the process of training a model using data in advance for a specific task.

[0880] "Being able to converse" refers to having the ability to engage in natural conversations on behalf of the user.

[0881] "Push-type" refers to a method of automatically providing information without waiting for a user request.

[0882] A "database" refers to a system for efficiently storing, managing, and retrieving data.

[0883] "Preprocessing" refers to the initial steps taken to convert data into a format suitable for analysis and learning.

[0884] "Cleaning" refers to the process of removing unnecessary information and noise from data.

[0885] "Tokenization" refers to the process of dividing text data into words or phrases.

[0886] A "prompt statement" refers to an input statement used to cause a generative AI model to generate a specific output.

[0887] This invention is a system that learns user characteristics and processes information on behalf of the user. Specific embodiments of this system are described below.

[0888] The server receives conversation history from messenger apps and purchase history from online shopping provided by the user. This data is used as pre-training data to learn user characteristics. The server stores this data in a database. The database used is a relational database such as MySQL or PostgreSQL.

[0889] Next, the server preprocesses the stored data. This preprocessing includes cleaning the text data (removing unnecessary characters and noise) and tokenization (dividing it into words and phrases). These processes are performed using Python's NLTK and spaCy libraries.

[0890] The pre-processed data is input into a generative AI model (for example, OpenAI's GPT-4). The server uses this generative AI model to learn user characteristics. This learning process incorporates user conversation patterns and purchasing tendencies into the model. Once learning is complete, the server generates a model that embodies the user's characteristics.

[0891] The server generates an avatar account with the user's characteristics based on a trained model. This avatar account can then converse on behalf of the user within the messenger app. Specifically, the avatar account inputs prompt text into the generated AI model and sends the generated message to the friend.

[0892] As a concrete example, consider a scenario where a user provides their conversation history with a friend. For instance, the following prompt might be input into the AI ​​model:

[0893] Prompt: "Generate a message for a user to send to a friend. The user recently purchased a new smartphone and wants to tell their friend about their experience using it."

[0894] The generative AI model generates messages like the following:

[0895] Generated message: "Hi friend! I recently bought a new smartphone and it's really easy to use. The camera is especially amazing; it takes really beautiful photos. If you're thinking about getting a new smartphone, I highly recommend it!"

[0896] In this way, avatar accounts can send messages to friends that reflect the user's characteristics. Furthermore, avatar accounts can also make social media posts and perform web searches. This reduces the user's burden and enables efficient information processing.

[0897] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0898] Step 1: Data Collection

[0899] The server receives conversation history from messenger apps and purchase history from online shopping from users. This data is used as pre-training data to learn user characteristics. The input is conversation history and purchase history provided by the user, and the output is raw data stored in the database. Specifically, the server receives data uploaded by the user and stores it in the database.

[0900] Step 2: Data Preprocessing

[0901] The server preprocesses the stored data. This preprocessing includes cleaning the text data (removing unnecessary characters and noise) and tokenizing it (dividing it into words and phrases). The input is the raw data stored in the database, and the output is the preprocessed, clean data. Specifically, the server uses Python's NLTK and spaCy libraries to clean and tokenize the text data.

[0902] Step 3: Trait Learning

[0903] The server inputs pre-processed data into a generative AI model to learn user characteristics. The input is clean, pre-processed data, and the output is a trained model that reflects user characteristics. Specifically, the server uses a generative AI model (for example, OpenAI's GPT-4) to learn user conversation patterns and purchasing tendencies.

[0904] Step 4: Create an avatar account

[0905] The server generates an avatar account with the user's characteristics based on a pre-trained model. The input is the pre-trained model, and the output is the avatar account. Specifically, the server creates an avatar account that converses on behalf of the user in the messenger app based on the output of the generated AI model.

[0906] Step 5: Avatar Account Operation

[0907] The avatar account inputs a prompt into a generative AI model and sends the generated message to a friend. The input is the prompt, and the output is the generated message. Specifically, the avatar account inputs a prompt into the generative AI model like the following:

[0908] Prompt: "Generate a message for a user to send to a friend. The user recently purchased a new smartphone and wants to tell their friend about their experience using it."

[0909] The generative AI model generates messages like the following:

[0910] Generated message: "Hi friend! I recently bought a new smartphone and it's really easy to use. The camera is especially amazing; it takes really beautiful photos. If you're thinking about getting a new smartphone, I highly recommend it!"

[0911] The avatar account sends this message to its friends. This enables automated information sharing and communication that reflects the user's characteristics.

[0912] (Application Example 2)

[0913] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0914] Traditional social networking services (SNS) and messaging apps required users to gather information, post, and send messages themselves, which was time-consuming and laborious. Furthermore, there was a lack of effective means to deliver advertisements that reflected user characteristics and preferences. Therefore, there is a need to improve user convenience while maximizing the effectiveness of advertising.

[0915] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for learning user characteristics and performing SNS / web searches / information organization on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on a messenger app; means for collaborating with a generative AI to pre-train the AI ​​with messenger app talk history and internet search tool shopping purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; and means for the avatar, which has learned the user's characteristics, to post advertisements on SNS and messenger apps on behalf of the user. This makes it possible to automatically collect and post information on behalf of the user, improving user convenience and enabling effective advertising delivery that reflects the user's characteristics and preferences.

[0916] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and tastes.

[0917] "SNS" is an abbreviation for Social Networking Service, which refers to an online platform for users to share information and communicate with other users.

[0918] "Web search" refers to the act of finding information on the internet using a search engine.

[0919] "Information organization" refers to the act of classifying and organizing collected information.

[0920] A "messenger app" refers to an application that allows users to send and receive text messages, images, videos, and other content.

[0921] An "avatar account" refers to a virtual account that acts as a digital representation of a user online.

[0922] "Generative AI" refers to artificial intelligence that generates new information or content based on given data.

[0923] "Pre-training" refers to the process of training a model in advance using relevant data before performing a specific task.

[0924] "Being able to converse" refers to having the ability to engage in natural conversations with users.

[0925] "Push marketing" refers to a method of automatically providing information before the user requests it.

[0926] "Posting an advertisement" refers to the act of publishing information about a specific product or service on social media or messaging apps.

[0927] The system for implementing this invention learns user characteristics and performs tasks such as SNS, web searches, and information organization on behalf of the user. Specifically, it provides an avatar account that acts as a surrogate for the user on a messenger app, and by linking with a generative AI to pre-train the avatar with messenger app chat history and internet search tool shopping purchase history, it realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[0928] The server first retrieves the user's conversation history and purchase history from messenger apps. This data is collected using APIs from messenger apps and shopping sites. Next, the server uses generative AI to learn the user's characteristics based on this data. Specifically, the conversation history and purchase history are input as prompts into the generative AI model to learn the user's characteristics.

[0929] After learning is complete, the server generates an avatar account with the user's characteristics, and this avatar posts advertisements on social media and messaging apps on the user's behalf. This enables automatic information gathering and posting on behalf of the user, improving user convenience and allowing for effective ad delivery that reflects the user's characteristics and preferences.

[0930] The hardware used includes servers and user terminals (smartphones, tablets, PCs, etc.). The software includes messenger apps, shopping site APIs, and generative AI (for example, OpenAI APIs).

[0931] As a concrete example, consider a case where the user ID is "user123" and the advertisement content is "Check out this amazing product!". In this case, the server will operate as follows:

[0932] 1. The server retrieves the conversation history and purchase history of user ID "user123".

[0933] 2. The server uses generative AI to learn the characteristics of "user123" based on the acquired data.

[0934] 3. The server uses the trained user model to post an advertisement on social media saying, "Check out this amazing product!"

[0935] Examples of prompts to input into a generative AI model:

[0936] Train a model based on the following data: [Conversation history and purchase history data]

[0937] This prompt is used to train the generative AI model on user characteristics.

[0938] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0939] Step 1:

[0940] The server retrieves the user's conversation and purchase history from messenger apps. Specifically, it uses APIs from messenger apps and shopping sites to collect conversation and purchase history associated with the user ID. The input is the user ID, and the output is conversation and purchase history data.

[0941] Step 2:

[0942] The server uses generative AI to learn user characteristics based on acquired conversation and purchase history. Specifically, it inputs conversation and purchase history as prompts into the generative AI model to learn user characteristics. The input is conversation and purchase history data, and the output is a model that reflects the user's characteristics.

[0943] Step 3:

[0944] The server generates avatar accounts based on the learned user model. Specifically, it uses generative AI to create avatar accounts that possess the user's characteristics and provides them to the messenger app. The input is the user model, and the output is the avatar account.

[0945] Step 4:

[0946] The server uses avatar accounts to post advertisements on social media and messaging apps. Specifically, it generates ad content that reflects the user's characteristics and posts it using the APIs of the social media and messaging apps. The input is the user model and ad content, and the output is the ad post on the social media or messaging app.

[0947] Step 5:

[0948] The server monitors the effectiveness of the ads and updates the user model as needed. Specifically, it collects data such as ad click-through rates and engagement rates, and uses generative AI to retrain the user model. The input is the ad performance data, and the output is the updated user model.

[0949] (Example 3)

[0950] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0951] Traditional information systems and search engines have struggled to adequately reflect user preferences and tastes, making it difficult to provide users with the most relevant information. Furthermore, the effort required for users to search for and organize information themselves made efficient information gathering difficult.

[0952] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for learning user characteristics and providing information and generating search results on behalf of the user; means for providing a virtual account on the messaging application that acts as a proxy for the user; means for working in cooperation with a generative AI to pre-train messaging application talk history and internet search tool purchase history to realize an "interactive" and "push-type" virtual account with the same characteristics as the user; means for collecting user behavior data and analyzing the data using natural language processing technology; means for generating a profile that reflects the user's preferences and tastes based on the analysis results; means for the virtual account to automatically provide information and perform searches based on the generated profile; and means for generating customized information and search results based on the virtual account's behavior results. This makes it possible to generate optimal information and search results that reflect the user's preferences and tastes.

[0953] "User characteristics" refer to individual features such as a user's preferences, tastes, behavioral patterns, and interests.

[0954] "Information provision" refers to the act of providing useful information to users.

[0955] "Search results" refer to a list of information returned in response to a user's search on the internet.

[0956] A "messaging application" refers to software that allows users to send and receive text messages and multimedia messages.

[0957] A "virtual account" refers to a digital account that acts on behalf of a user.

[0958] "Generative AI" refers to artificial intelligence that has the ability to generate new information and content based on data.

[0959] "Pre-training" refers to the process by which a system learns using the user's past data.

[0960] "Interactive" refers to a system having the ability to engage in natural conversations with users.

[0961] "Push-type" refers to a system that automatically provides information without waiting for a user request.

[0962] "Behavioral data" refers to data related to user actions (e.g., social media posts, search history, message sending, etc.).

[0963] "Natural language processing technology" refers to the technology used to understand, analyze, and generate human language.

[0964] A "profile" refers to a collection of data generated based on a user's characteristics and behavioral patterns.

[0965] "Customized information" refers to information that has been individually tailored based on the user's characteristics and preferences.

[0966] Modes for carrying out the invention

[0967] This invention is a system that learns user characteristics and provides information and generates search results on behalf of the user. Specific embodiments of this system are described below.

[0968] 1. Generating the system program

[0969] The server generates programs that provide information and search results that reflect the user's preferences and tastes. These programs have the functionality to manage virtual accounts that automatically perform user actions such as posting on social media, sending messages, and performing web searches.

[0970] 2. Explanation of the program's processing

[0971] The server uses the generated program to perform the following operations.

[0972] 1. Data collection:

[0973] The server collects data such as users' social media posts, message transmissions, and web search history. This data collection is done by obtaining data from various social media platforms and search engines via APIs. Specifically, it uses the Twitter API and the Google Search API.

[0974] 2. Data Analysis:

[0975] The server uses natural language processing (NLP) techniques to analyze the collected data. Specifically, it uses software such as Google Cloud Natural Language API and IBM Watson Natural Language Understanding.

[0976] 3. Generating a user profile:

[0977] The server generates a profile that reflects the user's preferences and tastes based on the analysis results. This profile includes topics and keywords that the user is interested in.

[0978] 4. Simulation of virtual account behavior:

[0979] Based on the generated user profiles, the server automatically performs tasks such as posting to social media, sending messages, and performing web searches using virtual accounts. This simulation uses a generative AI model (for example, OpenAI's GPT-4).

[0980] 5. Information provision and search result generation:

[0981] The server generates customized information and search results for users based on the actions of their virtual accounts. This information is provided in a format that best suits the user's preferences and tastes.

[0982] 3. Specific Examples and Examples of Prompt Statements

[0983] Specific example:

[0984] Let's say user A is interested in travel. The server extracts keywords such as "travel," "tourist destinations," and "hotels" from user A's social media posts and web search history. Based on this, a virtual account automatically makes travel-related social media posts and searches for travel blogs. Finally, it provides user A with information on recommended tourist destinations and hotels.

[0985] Example of a prompt:

[0986] "To provide user A with travel-related information they might be interested in, please suggest recommended tourist destinations and hotels based on their social media posts and web search history."

[0987] In this way, the server realizes a system that generates information and search results that reflect the user's preferences and tastes. The flow of specific processing in Example 3 will be explained using Figure 15.

[0988] Step 1: Data Collection

[0989] The server collects data such as users' social media posts, message transmissions, and web search history. Specifically, it uses the Twitter API to retrieve users' tweets and the Google Search API to collect users' search history. The input is the user's account information, and the output is the collected behavioral data.

[0990] Step 2: Data Analysis

[0991] The server uses natural language processing (NLP) techniques to analyze the collected data. Specifically, it uses the Google Cloud Natural Language API to extract keywords from users' tweets and search history and perform sentiment analysis. The input is collected behavioral data, and the output is the analyzed keywords and sentiment information.

[0992] Step 3: Generate User Profile

[0993] The server generates a profile that reflects the user's preferences and tastes based on the analysis results. Specifically, it lists topics and keywords that the user is interested in, based on the extracted keywords and sentiment analysis results. The input is the analyzed keywords and sentiment information, and the output is the generated user profile.

[0994] Step 4: Virtual Account Behavior Simulation

[0995] Based on the generated user profile, the server automatically performs social media posting, message sending, and web searches using a virtual account. Specifically, it uses a generative AI model (e.g., OpenAI's GPT-4) to generate social media posts based on the user's interests and search for relevant websites. The input is the user profile, and the output is the result of the virtual account's actions.

[0996] Step 5: Information provision and search result generation

[0997] The server generates customized information and search results for users based on the actions of virtual accounts. Specifically, it organizes the information collected by virtual accounts and provides users with recommendations for tourist destinations and hotels. The input is the actions of virtual accounts, and the output is customized information and search results.

[0998] (Application Example 3)

[0999] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[1000] Traditional systems have struggled to accurately reflect user preferences and tastes in providing information and generating search results. Furthermore, users had to manually post on social media, send messages, and perform web searches, which was time-consuming and laborious. Additionally, recommendation features were insufficient to suggest the most suitable products, failing to increase user purchasing intent. To address these challenges, there is a need for a system that automates user behavior, learns user characteristics, and provides optimal information and product recommendations.

[1001] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1002] In this invention, the server includes means for learning user characteristics and performing SNS, web searches, information organization, etc. on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on a messenger app; means for collaborating with a generative AI to pre-train the AI ​​with messenger app chat history and internet search tool shopping purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; means for analyzing the user's SNS posts, message sending, and web search history to learn the user's preferences and tastes; means for suggesting the most suitable products to the user; means for automatically posting product reviews and questions; and means for making recommendations based on purchase history. This makes it possible to automate user behavior and learn user characteristics to provide optimal information and product suggestions.

[1003] "User characteristics" refer to individual features such as a user's preferences, tastes, and behavioral patterns.

[1004] "SNS" is an abbreviation for Social Networking Service, which refers to a platform for users to interact with other users online.

[1005] "Web search" refers to the act of finding information on the internet using a search engine.

[1006] "Information organization" refers to the process of classifying and organizing collected data and information to make it easier to use.

[1007] A "messenger app" refers to application software used to send and receive text messages, images, videos, and other similar content.

[1008] An "avatar account" refers to a virtual account that acts as a digital representation of a user online.

[1009] "Generative AI" refers to artificial intelligence technology that generates new information and content based on data.

[1010] "Pre-training" refers to the process of training a model in advance using relevant data before performing a specific task.

[1011] "Push marketing" refers to a method of automatically providing information before the user requests it.

[1012] "SNS posting" refers to the act of publishing content such as text, images, and videos on social networking services.

[1013] "Sending a message" refers to the act of sending text messages, images, videos, etc., to other users.

[1014] "Web search history" refers to a record of searches a user has performed on the internet in the past.

[1015] "Product recommendation" refers to the act of recommending appropriate products based on the user's preferences and tastes.

[1016] A "product review" refers to the act of writing an evaluation or comment about a product that has been purchased.

[1017] "Recommendation" refers to a system that recommends appropriate products or services based on a user's past behavior and preferences.

[1018] The system for implementing this invention learns user characteristics and performs tasks such as SNS, web searches, and information organization on behalf of the user. The system provides an avatar account that acts as a surrogate for the user on a messenger app and operates in conjunction with a generative AI. A specific embodiment of the system is described below.

[1019] System Configuration

[1020] The system consists of the following main components:

[1021] 1. User device: A device such as a smartphone or tablet with a messenger app installed.

[1022] 2. Server: Collects and analyzes user data and provides information and product suggestions using generative AI models.

[1023] 3. Generative AI Models: AI models that analyze users' social media posts, message transmissions, and web search history to learn user preferences and tastes.

[1024] Program processing

[1025] The server implements the system using the following methods.

[1026] 1. Learning User Characteristics: The server collects users' social media posts, message transmissions, and web search history, and learns user characteristics using a generative AI model. Specifically, it converts text data into TF-IDF vectors and analyzes user preferences and tastes.

[1027] 2. Provision of Avatar Accounts: An avatar account will be created on the messenger app to act as a digital representation of the user, and it will post on social media and send messages on behalf of the user.

[1028] 3. Information Provision and Product Recommendations: The server provides optimal information and product recommendations based on the user's characteristics. Specifically, it makes recommendations based on the user's purchase history and search history.

[1029] 4. Automated posting: Avatar accounts automatically post product reviews and questions on behalf of users.

[1030] Hardware and software to be used

[1031] Hardware: User devices (smartphones, tablets), servers

[1032] Software: Messenger apps, generative AI models (e.g., scikit-learn's TfidfVectorizer, cosine_similarity)

[1033] Specific example

[1034] For example, if a user posts on social media that they "want a new smartphone," the system collects this information and learns the user's preferences. Then, if the user frequently searches for "high-performance laptops" on the web, the system uses this information to suggest the best products. Furthermore, the avatar account automatically posts product reviews and questions, organizing information on behalf of the user.

[1035] Example of a prompt

[1036] Create an application that analyzes users' social media posts, messages, and web search history to learn their preferences and tastes, and then recommends the most suitable products. User data will be retrieved via an API, and text data will be analyzed using TF-IDF vectors. Cosine similarity will be used for product recommendations.

[1037] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[1038] Step 1:

[1039] The user's device collects their social media posts, message transmissions, and web search history. This data serves as input for learning the user's preferences and tastes. Specifically, it collects text posts made by the user on social media, text messages sent via messaging apps, and URLs and search queries from their web search history.

[1040] Step 2:

[1041] The server inputs the collected user data into a generative AI model. The generative AI model uses this data to learn user characteristics. Specifically, it converts text data into TF-IDF vectors and analyzes user preferences and tastes. The input data is in text format, and the output is vector data representing user characteristics.

[1042] Step 3:

[1043] The server creates an avatar account on the messenger app that acts as a digital representation of the user. This avatar account makes social media posts and sends messages on behalf of the user. Specifically, it posts text generated based on the user's characteristics. The input data is a vector of the user's characteristics, and the output is the text of the social media post or message.

[1044] Step 4:

[1045] The server provides optimal information and product suggestions based on the user's characteristics. Specifically, it makes recommendations based on the user's purchase history and search history. The input data consists of the user's characteristic vector, purchase history, and search history, and the output is a list of recommended products.

[1046] Step 5:

[1047] The avatar account automatically posts product reviews and questions on behalf of the user. Specifically, it generates reviews and questions for recommended products and posts them from the avatar account. The input data is a list of recommended products, and the output is the text of the product reviews and questions.

[1048] Step 6:

[1049] The server continuously monitors user behavior, collects new data, and updates the generative AI model. This ensures that user characteristics are always up-to-date. The input data is newly collected user data, and the output is an updated user characteristic vector.

[1050] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1051] "Example of form 1"

[1052] One embodiment of the present invention is a system that incorporates an emotion engine. This system recognizes the user's emotions and adjusts the behavior of the avatar account based on those emotions. Specifically, when the user is feeling happy, the avatar account posts cheerful topics that reflect that emotion on social media. Also, when the user is sad, the avatar account sends comforting messages that reflect that emotion to their friends.

[1053] "Example of form 2"

[1054] Furthermore, systems that incorporate an emotion engine can recognize a user's emotions and adjust the information provided and search results based on those emotions. For example, when a user is feeling stressed, the system will provide information on relaxation and meditation. When a user is feeling happy, the system will provide search results that reflect that emotion, such as enjoyable events and news.

[1055] "Example of form 3"

[1056] Furthermore, a system incorporating an emotion engine can recognize a user's emotions and automatically perform actions on the avatar account based on those emotions. For example, when a user is angry, the avatar account automatically posts on social media that reflects that emotion. Also, when a user is surprised, the avatar account automatically reacts to a friend's post that reflects that emotion.

[1057] The following describes the processing flow for each example of the form.

[1058] "Example of form 1"

[1059] Step 1: The emotion engine recognizes the user's emotions.

[1060] Step 2: Adjust the avatar account's behavior based on the recognized emotions.

[1061] Step 3: For example, when a user is feeling happy, the avatar account posts fun topics on social media that reflect that emotion.

[1062] Step 4: Also, when a user is sad, the avatar account will send comforting messages to their friends that reflect that emotion.

[1063] "Example of form 2"

[1064] Step 1: The emotion engine recognizes the user's emotions.

[1065] Step 2: Adjust information and search results based on the recognized emotions.

[1066] Step 3: For example, when a user is feeling stressed, the system provides information about relaxation and meditation.

[1067] Step 4: When the user is feeling happy, the system will also provide search results with enjoyable events and news that reflect that emotion.

[1068] "Example of form 3"

[1069] Step 1: The emotion engine recognizes the user's emotions.

[1070] Step 2: Automate actions for the avatar account based on recognized emotions.

[1071] Step 3: For example, when a user is feeling angry, the avatar account automatically posts on social media that reflects that emotion.

[1072] Step 4: Also, when a user is surprised, the avatar account automatically reacts to their friend's post in a way that reflects that emotion.

[1073] (Example 1)

[1074] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1075] Conventional information processing systems struggled to fully reflect user characteristics and emotions during automation. When performing tasks such as posting on social media, sending messages, or searching for information on behalf of users, they failed to accurately reflect user intentions and feelings. Furthermore, they were insufficient in providing information and generating search results that reflected user preferences and tastes. This resulted in reduced user convenience and limited the system's usefulness.

[1076] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes means for learning user characteristics and processing information on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on the messenger app; means for working in cooperation with a generative AI to pre-train the AI ​​with messenger app talk history, internet search history, and purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; and means for recognizing the user's emotions using an emotion engine and adjusting the avatar account's behavior based on those emotions. This enables automation that accurately reflects the user's characteristics and emotions, thereby improving user convenience.

[1077] "User characteristics" refer to individual features such as a user's behavioral patterns, preferences, tastes, and emotions.

[1078] "Information processing" refers to a series of operations such as data collection, analysis, generation, and provision.

[1079] A "messenger app" refers to software that allows users to send and receive text messages and multimedia messages.

[1080] An "avatar account" refers to a virtual account that functions as a substitute for a user and acts on their behalf.

[1081] "Generative AI" refers to artificial intelligence that has the ability to generate new information and content based on data.

[1082] "Pre-training" refers to the process of learning using data in advance for a specific task.

[1083] An "emotion engine" refers to software that analyzes a user's emotions and generates appropriate responses based on those emotions.

[1084] "Push-type" refers to a method of automatically providing information and notifications without waiting for a user request.

[1085] This invention relates to a system that learns user characteristics and processes information on behalf of the user. This system provides an avatar account that acts as a surrogate for the user on a messenger app and learns the user's characteristics in cooperation with a generative AI. It also uses an emotion engine to recognize the user's emotions and adjusts the avatar account's behavior based on those emotions.

[1086] Hardware and software to be used

[1087] Hardware: Servers, user terminals (smartphones, PCs)

[1088] Software: Messenger apps, internet search tools, generative AI models (e.g., GPT-4), emotion engines

[1089] Data processing and data calculation

[1090] Data Collection: The server periodically collects users' messenger app chat history, internet search history, and purchase history. Specifically, the server retrieves chat history from messenger apps via APIs and search history from internet search tools. It also collects purchase history using shopping site APIs.

[1091] Data Pre-training: The server pre-trains a generative AI model using the collected data. During this process, it learns user characteristics and preferences. Specifically, the server pre-processes the collected data and tokenizes the text data. Then, it inputs the data into the generative AI model to learn user characteristics and preferences.

[1092] Emotion Recognition: The server uses an emotion engine to recognize the user's emotions in real time. Specifically, the server analyzes the latest message received from the messenger app and inputs it into the emotion engine. The emotion engine analyzes the user's emotions from the content and tone of the message and identifies emotions such as "joy," "sadness," and "anger."

[1093] Action Generation: The server combines a generative AI model and an emotion engine to generate actions based on the user's emotions. Specifically, the server inputs prompt text into the generative AI model based on the emotion information obtained from the emotion engine. For example, it inputs the prompt text, "Generate a fun topic to post on social media when the user is happy," and retrieves the generated text.

[1094] Action Execution: The server executes the generated action. Specifically, the server posts the generated text to messenger apps or social media. For example, when the user is happy, it posts a generated cheerful topic to social media, and when the user is sad, it sends generated words of comfort to their friends.

[1095] Specific example

[1096] Example 1: When a user sends "I'm so happy today!" via a messenger app, the emotion engine recognizes "joy." The generative AI model then posts fun topics to social media based on the user's past chat history.

[1097] Example 2: If a user sends "I'm tired today," the emotion engine recognizes "fatigue" and "sadness." The generative AI model then sends words of comfort to the user's friend.

[1098] Example of a prompt

[1099] "Generate fun topics that users will post on social media when they're happy."

[1100] "Generate comforting messages for users to send to their friends when they are feeling sad."

[1101] The above describes embodiments for carrying out the present invention.

[1102] The flow of the specific processing in Example 1 will be explained using Figure 17.

[1103] Step 1: Data Collection

[1104] The server periodically collects users' messenger app chat history, internet search history, and purchase history. Specifically, the server retrieves chat history from messenger apps via APIs and search history from internet search tools. It also collects purchase history using APIs from shopping sites.

[1105] Input: Messenger app chat history, internet search history, purchase history

[1106] Output: Collected user data

[1107] Specific operations: The server calls the messenger app's API every day at 2 AM to retrieve chat history. It also collects internet search history through browser extensions and retrieves purchase history from shopping site APIs once a week.

[1108] Step 2: Data pre-training

[1109] The server uses the collected data to pre-train a generative AI model. In this process, it learns the user's characteristics and preferences.

[1110] Input: Collected user data

[1111] Output: Pre-trained generative AI model

[1112] Specific operation: The server stores the collected data in JSON format and places it in a database. It reads the data from the database and tokenizes the text data. Then, it inputs the data into a generative AI model to learn user characteristics and preferences.

[1113] Step 3: Emotion Recognition

[1114] The server uses an emotion engine to recognize the user's emotions in real time. Specifically, the server analyzes the latest message received from the messenger app and inputs it into the emotion engine.

[1115] Input: Latest messenger app message

[1116] Output: User sentiment information

[1117] Specific operation: The server retrieves the latest message sent by the user via the messenger app in real time and inputs it into the emotion engine. The emotion engine analyzes the user's emotions from the content and tone of the message and identifies emotions such as "joy," "sadness," and "anger."

[1118] Step 4: Generate Action

[1119] The server combines a generative AI model and an emotion engine to generate actions based on the user's emotions. Specifically, the server inputs prompt text into the generative AI model based on the emotional information obtained from the emotion engine.

[1120] Input: User sentiment information

[1121] Output: Generated action text

[1122] Specific operation: Based on the emotional information obtained from the emotion engine, the server inputs a prompt message to the generative AI model saying, "Generate fun topics to post on social media when the user is happy," and retrieves the generated text.

[1123] Step 5: Execute Action

[1124] The server executes the generated actions. Specifically, the server posts the generated text to messenger apps or social media.

[1125] Input: Generated action text

[1126] Output: Actions performed (e.g., SNS post, message sent)

[1127] Specific operation: The server posts the generated text to messenger apps and social media. For example, when the user is happy, it posts a generated cheerful topic to social media, and when the user is sad, it sends generated words of comfort to their friends.

[1128] (Application Example 1)

[1129] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1130] Conventional information processing systems struggled to provide information and recommend products that adequately reflected user characteristics and emotions, thus failing to increase user satisfaction. Furthermore, users had to spend considerable time searching for and organizing information themselves, making efficient information acquisition difficult. Additionally, the inability to respond appropriately to user emotions led to a decline in the quality of the user experience.

[1131] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1132] This invention includes a server that includes means for learning user characteristics and processing information on behalf of the user, means for providing an avatar account that acts as a surrogate for the user on a messenger app, means for collaborating with a generative AI to pre-train the AI ​​with messenger app chat history and internet search tool purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user, means for recognizing the user's emotions and adjusting the avatar account's behavior based on those emotions, and means for recommending products based on the user's emotions. This makes it possible to provide information and recommend products that reflect the user's characteristics and emotions, thereby increasing user satisfaction. It also eliminates the effort the user has to spend searching for and organizing information themselves, enabling efficient information acquisition. Furthermore, it allows for appropriate responses in accordance with the user's emotions, thereby improving the quality of the user experience.

[1133] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and emotions.

[1134] "Information processing" refers to a series of operations such as data collection, organization, analysis, and provision.

[1135] A "messenger app" refers to a software application used to send and receive text messages, images, audio, and other data.

[1136] An "avatar account" refers to a virtual account that acts as a digital representation of a user online.

[1137] "Generative AI" refers to artificial intelligence that has the ability to generate new information and content based on data.

[1138] "Pre-training" refers to the process of training data in advance for a specific task.

[1139] "Emotion recognition" refers to technology that identifies emotions from a user's text, voice, and other data.

[1140] "Product recommendation" refers to the act of suggesting appropriate products based on the user's characteristics and emotions.

[1141] "Push marketing" refers to a method of automatically providing information or notifications to users before they request them.

[1142] The system for implementing this invention learns user characteristics and processes information on behalf of the user. Specifically, it provides an avatar account that acts as a surrogate for the user on a messenger app and pre-trains it with a generative AI using messenger app chat history and internet search tool purchase history. This realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[1143] Hardware and software to be used

[1144] Hardware: Smartphone

[1145] Software: Python, Transformers library

[1146] Processing flow

[1147] 1. Emotion Recognition: The user's input text is fed into an emotion recognition model (Transformers' sentiment-analysis pipeline) to recognize the user's emotions. This allows us to identify what emotions the user is currently experiencing.

[1148] 2. Product Recommendations: Based on recognized emotions, products are recommended based on the user's purchase and search history. For example, if the user is happy, special sale information is recommended; if they are sad, products that help them relax are recommended.

[1149] 3. Displaying Recommended Products: Display recommended products to the user. This makes it easy for users to find products that match their mood.

[1150] Specific example

[1151] If a user types "I'm feeling very happy today!", the app will recommend "Sale Item 1" and "Sale Item 2".

[1152] If a user enters "I'm feeling sad today," the app will recommend "Relaxation Product 1" or "Relaxation Product 2."

[1153] Example of a prompt

[1154] Create a Python program that recognizes user emotions and recommends products based on those emotions. The program will input user text into an emotion recognition model to recognize the user's emotions. Based on the recognized emotions, it will recommend products based on the user's purchase and search history. For example, if the user is happy, it will recommend special sale information; if they are sad, it will recommend relaxing products.

[1155] In this way, it becomes possible to provide information and recommend products that reflect the characteristics and emotions of users, thereby increasing user satisfaction. Furthermore, it eliminates the need for users to search for and organize information themselves, enabling efficient information acquisition. In addition, it allows for appropriate responses tailored to the user's emotions, improving the quality of the user experience.

[1156] The flow of a specific process in Application Example 1 will be explained using Figure 18.

[1157] Step 1:

[1158] The user enters text into the messenger app.

[1159] Input: User's text message

[1160] Output: Input text message

[1161] Specific action: The user enters text containing emotions, such as "I'm feeling very happy today!", into the messenger app.

[1162] Step 2:

[1163] The device sends the entered text message to the emotion recognition model.

[1164] Input: Entered text message

[1165] Output: Input data for the emotion recognition model

[1166] Specific operation: The terminal uses the Transformers library's sentiment-analysis pipeline to send the input text message to the sentiment recognition model.

[1167] Step 3:

[1168] The server uses an emotion recognition model to recognize the user's emotions.

[1169] Input: Input data for the emotion recognition model

[1170] Output: Recognized emotion (e.g., joy, sadness)

[1171] Specific operation: The server runs an emotion recognition model to determine the user's emotion from the input text message. For example, it recognizes the emotion "joy" from the text "I'm feeling very happy today!".

[1172] Step 4:

[1173] The server recommends products based on the emotions it perceives.

[1174] Input: Recognized emotions, user purchase history, search history

[1175] Output: Recommended product list

[1176] Specific operation: The server refers to the user's purchase and search history and selects appropriate products based on the recognized emotions. For example, for the emotion of "joy," it recommends products that include special sale information.

[1177] Step 5:

[1178] The server sends a list of recommended products to the terminal.

[1179] Input: Recommended Conlist

[1180] Output: Product list data to the terminal

[1181] Specific operation: The server generates a list of recommended products and sends it to the terminal.

[1182] Step 6:

[1183] The device displays a list of recommended products to the user.

[1184] Input: Product list data to the terminal

[1185] Output: Product list displayed to the user

[1186] Specific operation: The device displays a list of received products to the user, allowing the user to review the products. For example, "Sale Item 1" and "Sale Item 2" might be displayed.

[1187] By following these steps, product recommendations based on user emotions can be implemented, thereby increasing user satisfaction.

[1188] (Example 2)

[1189] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1190] In today's information society, users need to process and manage vast amounts of information. However, this is time-consuming and laborious, placing a burden on users. Furthermore, there is a lack of information provision based on users' emotions and preferences, failing to fully meet their needs. In addition, there is no means to process information on behalf of users when they are absent or busy, making efficient information management a challenge.

[1191] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1192] In this invention, the server includes means for learning user characteristics and processing information on behalf of the user; means for providing a virtual account that acts as a surrogate for the user on a communication application; means for collaborating with a generative AI to pre-train it with communication application talk history and e-commerce tool purchase history to realize a "conversational" and "push-type" virtual account with the same characteristics as the user; and means for analyzing the user's emotions using an emotion analysis engine and adjusting information provision and search results based on those emotions. This reduces the burden on the user, enables information provision based on the user's emotions and preferences, and realizes efficient information management.

[1193] "User characteristics" refer to individual features such as a user's behavioral patterns, preferences, and emotions.

[1194] "Information processing" refers to a series of operations such as data collection, analysis, organization, and provision.

[1195] A "communication application" refers to software used to send and receive messages between users.

[1196] A "virtual account" refers to a digital proxy account created to process information on behalf of a user.

[1197] "Generative AI" refers to artificial intelligence that has the ability to generate new information or responses based on data.

[1198] "Pre-training" refers to the process of training a model in advance using relevant data before performing a specific task.

[1199] An "emotion analysis engine" refers to software that analyzes a user's emotions from data such as text and audio.

[1200] "Information provision" refers to the act of presenting necessary information to users.

[1201] "Search results" refer to the list of information that a user obtains by using a search engine.

[1202] "Push marketing" refers to a method of automatically providing information before the user requests it.

[1203] This invention is a system that learns user characteristics and processes information on behalf of the user. Specifically, it provides a virtual account that acts as a surrogate for the user on a communication application and learns the user's characteristics in cooperation with a generative AI. It also analyzes the user's emotions using an emotion analysis engine and adjusts the information provided and search results based on those emotions.

[1204] Hardware and software to be used

[1205] The server uses the following hardware and software:

[1206] Hardware: High-performance server machine

[1207] Software: Python, pandas, numpy, generative AI (e.g., GPT-4), sentiment analysis engine (e.g., IBM Watson's Tone Analyzer)

[1208] Data collection and preprocessing

[1209] The server retrieves conversation history using the user's communication application API and purchase history using the e-commerce tool API. This data is stored in a secure database. Next, the data is preprocessed using the Python pandas library, specifically by removing unnecessary information and normalizing the data.

[1210] Feature extraction and model learning

[1211] The server extracts features from pre-processed data. For example, it uses natural language processing (NLP) techniques to extract user tone and frequently used phrases from conversation history. From purchase history, it analyzes user purchasing patterns and preferences. Next, it uses generative AI (GPT-4) to learn user characteristics. The pre-processed data and extracted features are input into the model to learn user conversation patterns and purchasing tendencies.

[1212] Creating a virtual account

[1213] The server generates a virtual account on the communication application that possesses the user's characteristics based on the learning results. Using the communication application's API, a new account is created and configured to reflect the user's characteristics. This virtual account can post information, send messages, and search for information on behalf of the user.

[1214] Emotion analysis and information provision

[1215] The server uses an emotion analysis engine to analyze the user's emotions. When a user sends a message through a communication application, the server analyzes the tone of the message and provides relaxation information if the user is feeling stressed. If the user is feeling happy, it provides search results that reflect that emotion, such as enjoyable events and news.

[1216] Specific example

[1217] User A provides conversation history with friend B via a communication application and purchase history via an e-commerce tool. The server collects this data and learns user A's characteristics using a generative AI (GPT-4). After learning is complete, the server creates a virtual account with user A's characteristics on the communication application. This virtual account can continue the conversation with friend B on behalf of user A.

[1218] Example of a prompt

[1219] "Based on User A's conversation history in communication applications and purchase history in e-commerce tools, learn User A's characteristics and create a virtual account that will engage in conversations and post information on User A's behalf."

[1220] In this way, the burden on users is reduced, information can be provided based on users' emotions and preferences, and efficient information management can be achieved.

[1221] The flow of the specific processing in Example 2 will be explained using Figure 19.

[1222] Step 1:

[1223] The server retrieves the conversation history using the user's communication application API. It takes the user's authentication information and API endpoint as input and obtains conversation history data as output. Specifically, the server sends an API request and receives the conversation history data in JSON format as a response.

[1224] Step 2:

[1225] The server retrieves purchase history using the API of an e-commerce tool. It uses user authentication information and the API endpoint as input and obtains purchase history data as output. Specifically, the server sends an API request and receives purchase history data in JSON format as a response.

[1226] Step 3:

[1227] The server preprocesses the acquired conversation history and purchase history data. It takes conversation history data and purchase history data as input and obtains preprocessed data as output. Specifically, the server uses the Python pandas library to remove unnecessary information and normalize the data.

[1228] Step 4:

[1229] The server extracts features from pre-processed data. It uses pre-processed conversation history data and purchase history data as input, and outputs extracted feature data. Specifically, the server uses natural language processing (NLP) techniques to extract conversational tones and frequently used phrases, and analyzes purchase patterns.

[1230] Step 5:

[1231] The server learns user characteristics using generative AI (GPT-4). It uses extracted feature data as input and obtains a trained model as output. Specifically, the server inputs feature data into the model and learns user conversation patterns and purchasing tendencies.

[1232] Step 6:

[1233] The server generates a virtual account on the communication application that possesses the user's characteristics based on the training results. It uses the trained model and the communication application's API endpoint as input, and obtains the virtual account as output. Specifically, the server sends an API request, creates a new account, and configures it to have the user's characteristics.

[1234] Step 7:

[1235] The server analyzes the user's emotions using an emotion analysis engine. It takes the user's message data as input and obtains the emotion analysis results as output. Specifically, the server inputs the message data into the emotion analysis engine and analyzes the emotional tone.

[1236] Step 8:

[1237] The server adjusts the information provided and search results based on the sentiment analysis results. It uses sentiment analysis results and user characteristic data as input, and outputs adjusted information and search results. Specifically, the server provides relaxation information and fun event information based on the sentiment analysis results.

[1238] (Application Example 2)

[1239] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[1240] Traditional systems struggled to provide information and recommend content that adequately reflected user characteristics and emotions, making it difficult to offer services that met user needs. Furthermore, creating avatar accounts that could perform tasks like SNS posting, messaging, and web searches on behalf of users was challenging. This resulted in decreased user convenience and satisfaction.

[1241] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1242] This invention includes a server that learns user characteristics and performs SNS, web searches, and information organization on behalf of the user; a server that provides an avatar account representing the user on a messenger app; a server that collaborates with a generative AI to pre-train the AI ​​with messenger app chat history and internet search tool shopping purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; a server that analyzes the user's emotions and adjusts information provision and search results based on those emotions; and a server that recommends content based on the user's characteristics and emotions. This makes it possible to provide information and recommend content that reflects the user's characteristics and emotions, and to realize an avatar account that performs SNS posting, message sending, and web searches on behalf of the user.

[1243] "User characteristics" refer to the individual user's preferences, tastes, and behavioral patterns extracted from their activity history, purchase history, conversation history, etc.

[1244] An "avatar account" refers to a virtual account that automatically performs actions such as posting on social media, sending messages, and performing web searches on behalf of the user.

[1245] "Generative AI" refers to artificial intelligence technology that generates new information and content based on user data.

[1246] "Pre-learning" refers to the process by which generative AI learns the user's characteristics based on the user's past behavior and purchase history.

[1247] "Sentiment analysis" refers to a technology that analyzes a user's emotional state based on their conversation history and behavioral data.

[1248] "Information provision" refers to the act of providing information that is appropriate for the user, based on the user's characteristics and emotions.

[1249] "Content recommendation" refers to the act of recommending content that is suitable for a user based on their characteristics and emotions.

[1250] "Push marketing" refers to a method of automatically providing information or content to users before they request it.

[1251] The system for implementing this invention has the function of learning user characteristics and performing tasks such as SNS, web searches, and information organization on behalf of the user. The system provides an avatar account that acts as a surrogate for the user on a messenger app and pre-trains the avatar account in conjunction with a generative AI using the user's messenger app chat history and internet search tool shopping purchase history. This realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[1252] The server has the ability to analyze user emotions and adjust the information provided and search results based on those emotions. Furthermore, it also has the ability to recommend content based on user characteristics and emotions.

[1253] Hardware and software to be used

[1254] Hardware: Servers, user terminals (smartphones, tablets, PCs, etc.)

[1255] Software: Messenger apps, generative AI models (e.g., OpenAI GPT-3), sentiment analysis libraries (e.g., NLTK, TextBlob), content recommendation engines (e.g., TensorFlow Recommenders)

[1256] Data processing and data calculation

[1257] 1. Data Acquisition: The user's device sends conversation history from messenger apps and purchase history from internet search tools to the server.

[1258] 2. Learning User Characteristics: The server uses a generative AI model to learn user characteristics from the acquired data.

[1259] 3. Sentiment Analysis: The server uses a sentiment analysis library to analyze the user's emotions from their conversation history.

[1260] 4. Information Provision and Content Recommendation: The server provides appropriate information and content to the user's device based on the user's characteristics and emotions.

[1261] Specific example

[1262] For example, if a user tells a friend via a messenger app that they are stressed out because work has been so busy lately, the server retrieves this conversation history and uses a sentiment analysis library to analyze whether the user is experiencing stress. Furthermore, it checks the user's purchase history to see if they have bought relaxation-related books and uses a generative AI model to learn the user's characteristics.

[1263] Example of a prompt

[1264] User's conversation history: I've been really busy with work lately and it's been stressing me out.

[1265] User's purchase history: Purchased books related to relaxation.

[1266] Learn about this user's characteristics.

[1267] By inputting this prompt into a generative AI model, it can learn the user's characteristics and recommend appropriate relaxation content.

[1268] The flow of a specific process in Application Example 2 will be explained using Figure 20.

[1269] Step 1:

[1270] The user's device sends the conversation history from the messenger app and the purchase history from the internet search tool to the server.

[1271] Input: User conversation history, purchase history

[1272] Output: User data sent to the server

[1273] Specific operation: The user's device retrieves conversation history using the messenger app's API and purchase history using the internet search tool's API. This data is then sent to the server.

[1274] Step 2:

[1275] The server inputs the received user data into a generative AI model to learn the user's characteristics.

[1276] Input: User data sent to the server

[1277] Output: Learned user characteristics

[1278] Specific operation: The server generates prompts for a generative AI model (e.g., OpenAI GPT-3) and learns the user's characteristics based on the user's conversation history and purchase history. An example of a prompt is: "User's conversation history: I've been stressed out lately because I've been busy with work. User's purchase history: I bought a book about relaxation. Please learn the characteristics of this user."

[1279] Step 3:

[1280] The server uses a sentiment analysis library to analyze the user's emotions from their conversation history.

[1281] Input: User conversation history

[1282] Output: Analyzed user emotional state

[1283] Specific operation: The server uses sentiment analysis libraries (e.g., NLTK, TextBlob) to analyze the user's conversation history and identify the emotions the user is feeling (e.g., stress, joy).

[1284] Step 4:

[1285] The server recommends appropriate information and content based on the user's characteristics and emotions.

[1286] Input: Learned user characteristics, analyzed user emotional state

[1287] Output: Recommended information and content

[1288] Specific operation: The server uses a content recommendation engine (e.g., TensorFlow Recommenders) to select appropriate information and content based on the user's characteristics and emotions, and sends it to the user's device.

[1289] Step 5:

[1290] The user terminal displays recommendation information and content received from the server.

[1291] Input: Recommended information or content

[1292] Output: Information and content displayed to the user

[1293] Specific operation: The user's device displays information and content received from the server on a messenger app or browser and provides it to the user.

[1294] (Example 3)

[1295] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1296] Traditional information systems and search engines have failed to adequately reflect user characteristics and preferences, making it difficult to provide users with the most relevant information. Furthermore, they have not incorporated information delivery or automated actions that consider user emotions, thus failing to increase user satisfaction.

[1297] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for providing an avatar account that represents the user, means for coordinating with a generative AI to pre-train the user's behavior history and realize an avatar account with the same characteristics as the user, and means for analyzing the user's emotions in real time using an emotion recognition engine and automatically performing actions of the avatar account based on the results. This makes it possible to provide information and generate search results that reflect the user's characteristics and preferences, and further enables the automation of actions that take the user's emotions into consideration.

[1298] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and interests.

[1299] An "avatar account" refers to a virtual account that acts as a digital representation of a user online.

[1300] "Generative AI" refers to artificial intelligence that has the ability to generate new information and content based on data.

[1301] "Pre-learning" refers to the process of pre-training the system with user behavior history and data.

[1302] An "emotion recognition engine" refers to a technology that analyzes a user's emotions based on their facial expressions, tone of voice, and other factors.

[1303] "Real-time" refers to the instantaneous collection and processing of data.

[1304] "Information provision" refers to the act of presenting useful information to users.

[1305] "Search results" refer to the list of related information displayed when a user performs a search.

[1306] "Action automation" refers to a system automatically performing specific actions on behalf of the user.

[1307] This invention is a system that learns user characteristics and provides information and generates search results on behalf of the user. The system provides an avatar account that acts as a surrogate for the user and pre-trains the avatar account with the user's behavioral history in cooperation with a generative AI. It also uses an emotion recognition engine to analyze the user's emotions in real time and automatically performs actions for the avatar account based on the results.

[1308] Hardware and software to be used

[1309] Hardware: Servers, terminals (PCs, smartphones)

[1310] Software: Generative AI models (e.g., GPT-4), emotion recognition engines (e.g., Affectiva), social media APIs (e.g., Twitter API, Facebook API)

[1311] Data processing and data calculation

[1312] 1. Data collection:

[1313] The server collects data such as users' social media posts, message transmissions, and web search history. Specifically, it uses the Twitter API and Facebook API to retrieve users' posts and search history.

[1314] The device collects real-time emotional data from the user using an emotion recognition engine. It analyzes the user's facial expressions and voice tone using the device's camera and microphone.

[1315] 2. Data Analysis:

[1316] The server inputs the collected data into an AI model to learn user characteristics and preferences. Specifically, it analyzes users' posts and search history to identify topics of interest.

[1317] The generative AI model learns user characteristics based on the input data and generates a profile.

[1318] 3. Information provision and search result generation:

[1319] The server generates appropriate information and search results based on the user's learned characteristics and preferences. Specifically, it provides news articles and product information that the user is likely to be interested in.

[1320] The generative AI model generates relevant information based on the user's profile.

[1321] 4. Emotion Recognition and Avatar Account Behavior:

[1322] The device uses an emotion recognition engine to analyze the user's emotions in real time and sends the results to the server.

[1323] Based on the results of the emotion recognition engine, the server instructs the avatar account to automatically perform actions that reflect the user's emotions.

[1324] Specific example

[1325] Example 1: When a user frequently posts on social media about "travel"

[1326] The server learns the user's travel preferences and prioritizes providing travel-related information and search results.

[1327] Example prompt for a generative AI model: "This user frequently posts about travel. Please provide travel-related information."

[1328] Example 2: When the user is angry

[1329] The device uses an emotion recognition engine to detect when the user is feeling angry and sends the result to the server.

[1330] The server sends an instruction to the avatar account saying, "The user is feeling angry. Please generate a social media post that reflects that anger."

[1331] The generation AI model creates "SNS posts for angry users," and the avatar account automatically posts them.

[1332] In this way, the system takes into account the user's characteristics and emotions to provide optimal information and automate actions. The flow of a specific process in Example 3 will be explained using Figure 21.

[1333] Step 1: Data Collection

[1334] The server collects data such as users' social media posts, message transmissions, and web search history. Specifically, it uses the Twitter API and Facebook API to retrieve users' posts and search history.

[1335] Input: User data obtained from the SNS API (post content, search history, etc.)

[1336] Output: Collected user data

[1337] Specific operation: The server periodically calls the SNS API to retrieve the user's latest posts.

[1338] Step 2: Collecting emotional data

[1339] The device collects real-time emotional data from the user using an emotion recognition engine. It analyzes the user's facial expressions and voice tone using the device's camera and microphone.

[1340] Input: Real-time user facial and voice data acquired from the device's camera and microphone.

[1341] Output: User emotion data analyzed by the emotion recognition engine

[1342] Specific operation: The device collects facial expression data via the camera while the user is using the smartphone and sends it to the emotion recognition engine.

[1343] Step 3: Data Analysis

[1344] The server inputs the collected data into an AI model to learn user characteristics and preferences. Specifically, it analyzes users' posts and search history to identify topics of interest.

[1345] Input: Collected user data (post content, search history, etc.)

[1346] Output: A profile that reflects the user's characteristics and preferences.

[1347] Specific operation: The server inputs the collected SNS post data into a generating AI model and analyzes it using the prompt message, "What topics do users frequently post about?"

[1348] Step 4: Information provision and search result generation

[1349] The server generates appropriate information and search results based on the user's learned characteristics and preferences. Specifically, it provides news articles and product information that the user is likely to be interested in.

[1350] Input: A profile that reflects the user's characteristics and preferences.

[1351] Output: Information and search results provided to the user

[1352] Specific operation: Based on a profile indicating that the user is interested in travel, the server searches for travel-related news articles and provides them to the user.

[1353] Step 5: Emotion Recognition and Avatar Account Behavior

[1354] The device uses an emotion recognition engine to analyze the user's emotions in real time and sends the results to the server.

[1355] Based on the results of the emotion recognition engine, the server instructs the avatar account to automatically perform actions that reflect the user's emotions.

[1356] Input: User emotion data analyzed by the emotion recognition engine

[1357] Output: Automated actions performed by the avatar account (e.g., social media posts, message sending)

[1358] Specific operation: The device detects that the user is feeling angry using an emotion recognition engine and sends the result to the server. The server sends an instruction to the avatar account saying, "The user is feeling angry. Please generate a social media post that reflects that anger." The generation AI model generates a social media post for the angry user, and the avatar account automatically posts it.

[1359] (Application Example 3)

[1360] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[1361] Traditional avatar account systems were insufficient in providing information and generating search results that reflected user preferences and tastes, and were unable to automatically perform actions based on user emotions. Furthermore, they lacked the ability to recommend optimal content based on user emotions and preferences, making improving the user experience a challenge.

[1362] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for learning user characteristics and performing SNS / web searches / information organization on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on a messenger app; means for collaborating with a generative AI to pre-train the AI ​​with messenger app talk history and internet search tool shopping purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; means for recognizing the user's emotions in real time using an emotion engine and automatically performing actions of the avatar account based on those emotions; and means for automatically recommending optimal content based on the user's emotions and preferences. This makes it possible to provide information, generate search results, and recommend optimal content based on the user's emotions and preferences.

[1363] "User characteristics" refer to individual features of a user, such as their behavioral history, preferences, tastes, and emotions.

[1364] An "avatar account" refers to a virtual account that acts as a digital representation of a user online.

[1365] "Generative AI" refers to artificial intelligence that learns user behavior and characteristics and generates information based on that.

[1366] "Messenger app chat history" refers to a user's conversation history within a messenger app.

[1367] "Internet search tool shopping purchase history" refers to the shopping history of a user who has used an internet search tool.

[1368] "Pre-training" refers to the process by which generative AI learns in advance using data from users' past behavior.

[1369] An "emotion engine" refers to a system that recognizes a user's emotions in real time and makes decisions based on those emotions.

[1370] "Content" refers to information and entertainment such as videos, articles, and music.

[1371] "Recommendation" refers to the act of selecting and presenting the most suitable content based on the user's characteristics and emotions.

[1372] The system for implementing this invention has the ability to learn user characteristics and perform tasks such as SNS, web searches, and information organization on behalf of the user. Specifically, it provides an avatar account that acts as a surrogate for the user on a messenger app and pre-trains it using messenger app chat history, internet search tool shopping purchase history, etc., in conjunction with a generative AI. Through this pre-training, it realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[1373] Furthermore, it uses an emotion engine to recognize the user's emotions in real time and automatically controls the avatar account's actions based on those emotions. It also includes a feature that automatically recommends the most suitable content based on the user's emotions and preferences.

[1374] Program Processing Description

[1375] The server runs sentiment recognition and content recommendation programs using Python. TextBlob is used for sentiment recognition to analyze user text data. To learn user preferences, the text data is vectorized using scikit-learn's TfidfVectorizer, and the optimal content is recommended by calculating cosine similarity.

[1376] Hardware and software to be used

[1377] Hardware: Smartphone

[1378] Software: Python, TextBlob, scikit-learn

[1379] Specific example

[1380] For example, if a user posts "I'm so tired today," sentiment analysis will detect negative emotions. In this case, the system will recommend relaxing music or videos.

[1381] Example of a prompt

[1382] User post: "I'm so tired today."

[1383] User preferences: ["Likes movies", "Likes music", "Likes traveling"]

[1384] Recommended content: ["Relaxing music", "Exciting movies", "Travel blogs"]

[1385] By inputting this prompt into the AI ​​generation model, it is possible to recommend the most suitable content based on the user's emotions and preferences.

[1386] The flow of the specific processing in Application Example 3 will be explained using Figure 22.

[1387] Step 1:

[1388] The server collects the user's messenger app chat history and internet search tool shopping purchase history. This data is entered as the user's behavioral history. The server stores this data in text format in preparation for later processing.

[1389] Step 2:

[1390] The server learns user characteristics using the collected data. Specifically, it uses the Python scikit-learn library and TfidfVectorizer to vectorize text data. This vectorized data is output as features representing user preferences.

[1391] Step 3:

[1392] The server uses TextBlob to recognize user emotions in real time. It receives text data posted by users in messenger apps as input and performs emotion analysis using TextBlob. As a result of the emotion analysis, positive, negative, or neutral emotion scores are output.

[1393] Step 4:

[1394] The server uses an emotion engine to determine the avatar account's actions based on the user's emotions. It receives an emotion score as input and is configured to recommend relaxing content for negative emotions and exciting content for positive emotions. This configuration is output as a guideline for the avatar account's actions.

[1395] Step 5:

[1396] The server recommends the most suitable content based on the user's preferences and emotions. Specifically, it takes features representing the user's preferences and an emotion score as input, and uses scikit-learn's cosine similarity calculation to select the most appropriate content. This recommended content is then output as information provided to the user.

[1397] Step 6:

[1398] The device displays content recommended by the server to the user. The user can view the recommended content on their smartphone screen and use it as needed. In this step, the recommended content is displayed on the user's device.

[1399] Step 7:

[1400] Users engage with recommended content and submit feedback to the server. The server receives this feedback and stores it in a database to inform future recommendations. This feedback is used to further learn about user characteristics.

[1401] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1402] The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1403] Other examples of generative AI include Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are some examples.

[1404] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1405] [Third Embodiment]

[1406] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1407] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1408] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1409] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1410] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1411] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1412] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1413] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1414] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1415] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1416] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1417] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[1418] "Example of form 1"

[1419] One embodiment of the present invention provides a system that learns user characteristics and performs tasks such as SNS, web searches, and information organization on behalf of the user. Specifically, an avatar account that acts as a surrogate for the user is provided on a messenger app. This avatar account pre-learns the user's messenger app chat history and internet search tool shopping purchase history, and by linking with a generative AI, it realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[1420] "Example of form 2"

[1421] As a concrete example, user A provides pre-training data such as their conversation history with friend B on a messenger app and their Amazon purchase history. Based on this data, the system works in conjunction with a generative AI to learn user A's characteristics. After learning is complete, the system provides an avatar account on the messenger app that acts as a digital representation of user A. This avatar account has the same characteristics as user A and is a "conversational," "push-type" avatar account that can perform actions such as posting on social media, sending messages, and performing web searches on behalf of user A.

[1422] "Example of form 3"

[1423] Furthermore, in another embodiment of the present invention, a function is provided to generate information and search results that reflect the user's preferences and tastes. Specifically, an avatar account is provided that automatically performs actions such as posting on social media, sending messages, and performing web searches. This avatar account can learn the user's characteristics and generate information and search results that reflect them.

[1424] The following describes the processing flow for each example of the form.

[1425] "Example of form 1"

[1426] Step 1: The user provides the system with their chat history with friends via a messenger app and their Amazon purchase history.

[1427] Step 2: Based on the provided data, the system works in conjunction with generative AI to learn the user's characteristics.

[1428] Step 3: After learning is complete, the system will provide the user with an avatar account on the messenger app, which will serve as their digital counterpart.

[1429] Step 4: This avatar account becomes a "conversational" and "push-type" avatar account with the same characteristics as the user, and can perform actions such as posting on social media, sending messages, and performing web searches on behalf of the user.

[1430] "Example of form 2"

[1431] Step 1: Users provide the system with behavioral data such as social media posts, message sending, and web searches.

[1432] Step 2: The system learns the user's characteristics based on the provided data.

[1433] Step 3: After learning is complete, the system will provide an avatar account that will serve as a digital representation of the user.

[1434] Step 4: This avatar account can generate information and search results that reflect the user's characteristics.

[1435] (Example 1)

[1436] Next, we will describe Embodiment 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1437] In today's information society, users need to efficiently collect and organize vast amounts of information. However, doing so manually is time-consuming, laborious, and inefficient. Furthermore, there is a demand for information tailored to user characteristics and preferences, but conventional systems struggle to adequately achieve this. Additionally, there is a lack of means to centrally manage and utilize the history and characteristics of multiple information and communication services and search tools used by users. A system is needed to address these challenges.

[1438] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1439] In this invention, the server includes means for learning user characteristics and performing information communication services, information retrieval, and information organization on behalf of the user; means for providing a virtual account that acts as a surrogate for the user on an information communication application; means for collaborating with a generative AI to pre-train it on the history of information communication applications, internet search tools, and purchase history to realize a "conversational" and "push-type" virtual account with the same characteristics as the user; means for pre-processing collected data and extracting user characteristics; means for training a generative AI model using the extracted characteristics; and means for collecting relevant information based on the user's interests and preferences and sending push notifications through the virtual account. This makes it possible to provide information and generate search results based on the user's characteristics and preferences, and can significantly improve the efficiency of the user's information gathering and organization.

[1440] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and interests.

[1441] "Information and communication services" refers to communication methods such as messaging, social networking services (SNS), and email that are provided via the internet.

[1442] "Information retrieval" refers to the act of finding specific information on the internet.

[1443] "Information organization" refers to the act of classifying, organizing, and making usable information.

[1444] An "information and communication application" refers to software that allows users to send and receive messages and share information.

[1445] A "virtual account" refers to a digital avatar that operates on behalf of a user within an information and communication application.

[1446] "Generative AI" refers to artificial intelligence technology that learns user characteristics and generates responses and actions similar to those of the user.

[1447] "Pre-training" refers to the process of training a generative AI model in advance using collected data.

[1448] "Conversational" means that a virtual account has the ability to engage in natural conversations with the user.

[1449] "Push notifications" refer to a function that automatically notifies users of relevant information based on their interests and preferences.

[1450] "Preprocessing" refers to the process of preparing collected data into a format that is easy to analyze.

[1451] "Feature extraction" refers to the process of extracting characteristics such as user interests and preferences from pre-processed data.

[1452] "Training" refers to the process of using extracted characteristics to train a generative AI model.

[1453] "Push notifications" refer to a function that sends information to users in real time.

[1454] This invention is a system that learns user characteristics and performs information communication services, information retrieval, information organization, etc., on behalf of the user. Specific embodiments of this system are described below.

[1455] 1. System Overview

[1456] The server learns user characteristics and provides a system to perform information communication services, information retrieval, and information organization on behalf of the user. This system provides a virtual account that acts as a surrogate for the user on the information communication application and operates in conjunction with a generative AI.

[1457] 2. Hardware and software to be used

[1458] The server uses the following hardware and software:

[1459] Hardware: High-performance server machines, database servers

[1460] Software: Information and communication applications, generative AI models, data analysis tools, push notification systems

[1461] 3. Data Collection and Pre-training

[1462] The server collects the user's information and communication application history, internet search tool history, and purchase history. This data serves as foundational data for learning user characteristics. The collected data is preprocessed, with unnecessary data being removed and data normalization performed.

[1463] Next, user characteristics are extracted from the pre-processed data. Text data is analyzed using natural language processing techniques to identify user interests and preferences. A generative AI model is then trained using these extracted characteristics. This pre-training allows the generative AI model to understand user characteristics and generate responses similar to those of the user.

[1464] 4. Creating and configuring virtual accounts

[1465] The server uses a pre-trained generative AI model to generate a virtual account that reflects the user's characteristics. This virtual account can converse on behalf of the user within information and communication applications. Furthermore, the virtual account has the functionality to push relevant information based on the user's interests and preferences.

[1466] 5. Specific Examples

[1467] Example 1: Travel planning

[1468] When a user is planning a trip with a friend using an information and communication application, the server provides information on suitable destinations and accommodations based on past travel and search history. For example, it might suggest places the user has visited in the past that their friend might be interested in.

[1469] Example 2: Providing shopping information

[1470] If a user is interested in products from a particular brand, the server collects information about new products from that brand and sends push notifications through a virtual account. For example, it might immediately notify the user when a new product from a brand they have previously purchased is released.

[1471] 6. Example of a prompt statement

[1472] "Create a virtual account that provides information about travel destinations the user is interested in, based on their information and communication application history and internet search history. Also, implement a function to send push notifications with the latest information about those travel destinations."

[1473] In this way, a system is realized that learns the characteristics of users and performs information and communication services, information retrieval, and information organization on behalf of the user.

[1474] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1475] Step 1: Data Collection

[1476] The server collects the user's information and communication application history, internet search tool history, and purchase history.

[1477] Input: User's information, communication application history, internet search history, purchase history

[1478] Data processing: The collected data is stored in a database for centralized management.

[1479] Output: User activity history data stored in the database

[1480] Specific operation: The server retrieves data from information and communication applications and internet search tools via APIs and stores it in a database.

[1481] Step 2: Data preprocessing

[1482] The server preprocesses the collected data.

[1483] Input: User activity history data stored in the database

[1484] Data processing: Deletion of unnecessary data, data normalization, tokenization of text data.

[1485] Output: Preprocessed clean dataset

[1486] Specific operation: The server uses a data cleansing tool to remove noisy data and prepares the text data in a format that is easy to analyze.

[1487] Step 3: Feature Extraction

[1488] The server extracts user characteristics from the pre-processed data.

[1489] Input: Preprocessed clean dataset

[1490] Data Processing: Analyze text data using natural language processing techniques to identify user interests and preferences.

[1491] Output: Feature vector representing user characteristics

[1492] Specific operation: The server uses a natural language processing library to analyze text data and extract user interests and preferences.

[1493] Step 4: Training the Generative AI Model

[1494] The server uses the extracted features to train a generative AI model.

[1495] Input: Feature vector representing user characteristics

[1496] Data processing: Input feature vectors into a generative AI model and train the model.

[1497] Output: Generative AI model that learned user characteristics

[1498] Specific operation: The server trains a generative AI model using a machine learning framework to learn the user's characteristics.

[1499] Step 5: Create a virtual account

[1500] The server uses a pre-trained generative AI model to generate virtual accounts that reflect the user's characteristics.

[1501] Input: Generative AI model that has learned user characteristics

[1502] Data processing: Create virtual accounts using a generative AI model and register them in an information and communication application.

[1503] Output: Virtual account on information and communication application

[1504] Specific operation: The server generates virtual accounts using a generated AI model and registers them with the information and communication application via an API.

[1505] Step 6: Implementing push notifications

[1506] The server collects relevant information based on the user's interests and sends push notifications through a virtual account.

[1507] Input: Data on user interests and preferences, external information sources

[1508] Data processing: Collect relevant information and format it in a way that is suitable for the user.

[1509] Output: Push notification message

[1510] Specific operation: The server collects relevant information from external information sources and sends push notifications to the user through a virtual account.

[1511] In this way, a system is realized that learns the characteristics of users and performs information and communication services, information retrieval, and information organization on behalf of the user.

[1512] (Application Example 1)

[1513] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1514] Modern consumers find it difficult and time-consuming to find the best product for them from the vast amount of information available. Furthermore, the lack of personalized product recommendations based on user characteristics and preferences makes it difficult for consumers to find suitable products. Additionally, the absence of systems that handle social media, web searches, and information organization on behalf of users means that users must gather information themselves, posing a significant challenge.

[1515] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1516] In this invention, the server includes means for learning user characteristics and performing SNS, web searches, information organization, etc. on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on a messenger app; means for collaborating with a generative AI to pre-train the AI ​​with messenger app chat history and internet search tool shopping purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; means for learning the user's purchase history and search history and automatically suggesting products that match the user's preferences; and means for suggesting products based on the user's history using a generative AI model. As a result, users can easily find products that are best suited to them and can save time and effort in gathering information.

[1517] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and purchase history.

[1518] "SNS" is an abbreviation for Social Networking Service, which is an online platform for users to share information and communicate with other users.

[1519] "Web search" refers to the act of finding information on the internet using a search engine.

[1520] "Information organization" refers to the act of classifying collected information and compiling it into an easily understandable format.

[1521] A "messenger app" is application software that allows users to send and receive text messages, images, audio, and other data.

[1522] An "avatar account" is a virtual account that functions as a substitute for the user, handling communication and information gathering on their behalf.

[1523] "Generative AI" refers to artificial intelligence technology that generates new information and suggestions based on user input and historical data.

[1524] "Pre-learning" refers to the process by which a system learns from a user's past behavior history and data in advance.

[1525] "Push-type" refers to a system that automatically provides information or suggestions before the user requests them.

[1526] "Purchase history" refers to a record of products that a user has purchased in the past.

[1527] "Search history" refers to a record of keywords and phrases that a user has previously searched for on a search engine.

[1528] A "generative AI model" refers to an artificial intelligence algorithm that generates new information and suggestions based on a user's historical data.

[1529] As an embodiment of this invention, a system is provided that learns user characteristics and performs SNS, web searches, information organization, etc., on behalf of the user. Specifically, an avatar account that acts as a surrogate of the user is provided on a messenger app, and in cooperation with a generative AI, the avatar is pre-trained with the user's messenger app chat history, internet search tool, and shopping purchase history. This avatar account realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[1530] The server has the ability to learn from the user's purchase and search history and automatically suggest products that match the user's preferences. It uses a generative AI model to suggest products based on the user's history. This system makes it easy for users to find the products that are best suited to them, saving them the trouble of gathering information themselves.

[1531] Hardware and software to be used

[1532] Hardware: Servers, user terminals (smartphones, tablets, PCs, etc.)

[1533] Software: Messenger apps, generative AI models (e.g., OpenAI GPT-3)

[1534] Data processing and data calculation

[1535] The server collects the user's messenger app chat history, internet search tool data, and shopping purchase history, and uses this data for pre-training. Using a generative AI model, it analyzes the user's historical data and learns the user's characteristics and preferences. This allows it to automatically suggest products that match the user's preferences.

[1536] Specific example

[1537] If a user has previously purchased a "smartphone" or "wireless earphones" and has searched for "the latest smartphone" or "high-quality earphones," the generative AI model will suggest products related to "the latest smartphone" or "high-quality earphones."

[1538] Example of a prompt

[1539] User purchase history: ["Smartphone", "Wireless earphones"]

[1540] User's search history: ["Latest smartphone", "High-quality earphones"]

[1541] Please suggest products that would be suitable for this user.

[1542] In this way, a personal shopping assistant can be realized that automatically suggests products that match the user's preferences.

[1543] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1544] Step 1:

[1545] The server collects users' messenger app chat history, internet search tool data, and shopping purchase history. This data serves as input for learning user characteristics and preferences. Specifically, it retrieves data from each platform via APIs and stores it in a database.

[1546] Input: Messenger app chat history, internet search tools, shopping purchase history

[1547] Output: User history data stored in the database

[1548] Step 2:

[1549] The server pre-trains on the collected user history data. In this pre-training process, a generative AI model is used to analyze the data and learn user characteristics and preferences. Specifically, the history data is input into the AI ​​model to extract user behavior patterns and preferences.

[1550] Input: User history data stored in the database

[1551] Output: A model that reflects the user's characteristics and preferences.

[1552] Step 3:

[1553] The server uses a generative AI model to suggest products based on the user's history. In this process, prompt sentences are generated based on the user's purchase and search history to suggest the most suitable products, and these are input into the AI ​​model. The AI ​​model then generates product suggestions based on these prompt sentences.

[1554] Input: A model that reflects the user's characteristics and preferences, and prompt text.

[1555] Output: Product proposal

[1556] Step 4:

[1557] The server sends the generated product suggestions to the user's device. Users can then view the suggested products through a messenger app or a dedicated app. Specifically, product suggestions are sent to the user's device via an API and displayed in the user interface.

[1558] Input: Product proposal

[1559] Output: Product suggestions displayed on the user's terminal

[1560] Step 5:

[1561] Users review the suggested products and purchase them if necessary. This process involves users receiving product suggestions, selecting items of interest, and proceeding with the purchase. Specifically, they select products through the user interface and click the purchase button.

[1562] Input: Product suggestions displayed on the user's terminal

[1563] Output: User purchasing behavior

[1564] In this way, the system automatically suggests products based on the user's characteristics and preferences, enabling users to easily find the products that are best suited to them.

[1565] (Example 2)

[1566] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1567] Conventional information processing systems struggled to provide automated information and communication that fully reflected user characteristics. Furthermore, creating avatar accounts that could perform tasks such as SNS posting, message sending, and web searches on behalf of users was also difficult. This resulted in increased user burden and inefficient information processing.

[1568] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for learning user characteristics and processing information on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on the messenger app; means for coordinating with a generative AI model to pre-train it with the conversation history of the messenger app and the purchase history of online shopping to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; means for pre-processing data stored in the database, cleaning and tokenizing text data; and means for inputting prompt sentences to the generative AI model and sending the generated message. This enables automated information provision and communication that reflects the user's characteristics.

[1569] "User characteristics" refer to individual features such as user behavior patterns, preferences, and tastes.

[1570] "Information processing" refers to a series of operations such as data collection, analysis, storage, retrieval, and generation.

[1571] An "avatar account" refers to a virtual account that operates online on behalf of a user.

[1572] A "generative AI model" refers to an artificial intelligence model that generates new information or content based on given data.

[1573] "Pre-training" refers to the process of training a model using data in advance for a specific task.

[1574] "Being able to converse" refers to having the ability to engage in natural conversations on behalf of the user.

[1575] "Push-type" refers to a method of automatically providing information without waiting for a user request.

[1576] A "database" refers to a system for efficiently storing, managing, and retrieving data.

[1577] "Preprocessing" refers to the initial steps taken to convert data into a format suitable for analysis and learning.

[1578] "Cleaning" refers to the process of removing unnecessary information and noise from data.

[1579] "Tokenization" refers to the process of dividing text data into words or phrases.

[1580] A "prompt statement" refers to an input statement used to cause a generative AI model to generate a specific output.

[1581] This invention is a system that learns user characteristics and processes information on behalf of the user. Specific embodiments of this system are described below.

[1582] The server receives conversation history from messenger apps and purchase history from online shopping provided by the user. This data is used as pre-training data to learn user characteristics. The server stores this data in a database. The database used is a relational database such as MySQL or PostgreSQL.

[1583] Next, the server preprocesses the stored data. This preprocessing includes cleaning the text data (removing unnecessary characters and noise) and tokenization (dividing it into words and phrases). These processes are performed using Python's NLTK and spaCy libraries.

[1584] The pre-processed data is input into a generative AI model (for example, OpenAI's GPT-4). The server uses this generative AI model to learn user characteristics. This learning process incorporates user conversation patterns and purchasing tendencies into the model. Once learning is complete, the server generates a model that embodies the user's characteristics.

[1585] The server generates an avatar account with the user's characteristics based on a trained model. This avatar account can then converse on behalf of the user within the messenger app. Specifically, the avatar account inputs prompt text into the generated AI model and sends the generated message to the friend.

[1586] As a concrete example, consider a scenario where a user provides their conversation history with a friend. For instance, the following prompt might be input into the AI ​​model:

[1587] Prompt: "Generate a message for a user to send to a friend. The user recently purchased a new smartphone and wants to tell their friend about their experience using it."

[1588] The generative AI model generates messages like the following:

[1589] Generated message: "Hi friend! I recently bought a new smartphone and it's really easy to use. The camera is especially amazing; it takes really beautiful photos. If you're thinking about getting a new smartphone, I highly recommend it!"

[1590] In this way, avatar accounts can send messages to friends that reflect the user's characteristics. Furthermore, avatar accounts can also make social media posts and perform web searches. This reduces the user's burden and enables efficient information processing.

[1591] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1592] Step 1: Data Collection

[1593] The server receives conversation history from messenger apps and purchase history from online shopping from users. This data is used as pre-training data to learn user characteristics. The input is conversation history and purchase history provided by the user, and the output is raw data stored in the database. Specifically, the server receives data uploaded by the user and stores it in the database.

[1594] Step 2: Data Preprocessing

[1595] The server preprocesses the stored data. This preprocessing includes cleaning the text data (removing unnecessary characters and noise) and tokenizing it (dividing it into words and phrases). The input is the raw data stored in the database, and the output is the preprocessed, clean data. Specifically, the server uses Python's NLTK and spaCy libraries to clean and tokenize the text data.

[1596] Step 3: Trait Learning

[1597] The server inputs pre-processed data into a generative AI model to learn user characteristics. The input is clean, pre-processed data, and the output is a trained model that reflects user characteristics. Specifically, the server uses a generative AI model (for example, OpenAI's GPT-4) to learn user conversation patterns and purchasing tendencies.

[1598] Step 4: Create an avatar account

[1599] The server generates an avatar account with the user's characteristics based on a pre-trained model. The input is the pre-trained model, and the output is the avatar account. Specifically, the server creates an avatar account that converses on behalf of the user in the messenger app based on the output of the generated AI model.

[1600] Step 5: Avatar Account Operation

[1601] The avatar account inputs a prompt into a generative AI model and sends the generated message to a friend. The input is the prompt, and the output is the generated message. Specifically, the avatar account inputs a prompt into the generative AI model like the following:

[1602] Prompt: "Generate a message for a user to send to a friend. The user recently purchased a new smartphone and wants to tell their friend about their experience using it."

[1603] The generative AI model generates messages like the following:

[1604] Generated message: "Hi friend! I recently bought a new smartphone and it's really easy to use. The camera is especially amazing; it takes really beautiful photos. If you're thinking about getting a new smartphone, I highly recommend it!"

[1605] The avatar account sends this message to its friends. This enables automated information sharing and communication that reflects the user's characteristics.

[1606] (Application Example 2)

[1607] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as a "server," and the headset-type terminal 314 will be referred to as a "terminal."

[1608] Traditional social networking services (SNS) and messaging apps required users to gather information, post, and send messages themselves, which was time-consuming and laborious. Furthermore, there was a lack of effective means to deliver advertisements that reflected user characteristics and preferences. Therefore, there is a need to improve user convenience while maximizing the effectiveness of advertising.

[1609] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for learning user characteristics and performing SNS / web searches / information organization on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on a messenger app; means for collaborating with a generative AI to pre-train the AI ​​with messenger app talk history and internet search tool shopping purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; and means for the avatar, which has learned the user's characteristics, to post advertisements on SNS and messenger apps on behalf of the user. This makes it possible to automatically collect and post information on behalf of the user, improving user convenience and enabling effective advertising delivery that reflects the user's characteristics and preferences.

[1610] "User characteristics" refer to individual features such as a user's behavioral history, preferences, and tastes.

[1611] "SNS" is an abbreviation for Social Networking Service, which refers to an online platform for users to share information and communicate with other users.

[1612] "Web search" refers to the act of finding information on the internet using a search engine.

[1613] "Information organization" refers to the act of classifying and organizing collected information.

[1614] A "messenger app" refers to an application that allows users to send and receive text messages, images, videos, and other content.

[1615] An "avatar account" refers to a virtual account that acts as a digital representation of a user online.

[1616] "Generative AI" refers to artificial intelligence that generates new information or content based on given data.

[1617] "Pre-training" refers to the process of training a model in advance using relevant data before performing a specific task.

[1618] "Being able to converse" refers to having the ability to engage in natural conversations with users.

[1619] "Push marketing" refers to a method of automatically providing information before the user requests it.

[1620] "Posting an advertisement" refers to the act of publishing information about a specific product or service on social media or messaging apps.

[1621] The system for implementing this invention learns user characteristics and performs tasks such as SNS, web searches, and information organization on behalf of the user. Specifically, it provides an avatar account that acts as a surrogate for the user on a messenger app, and by linking with a generative AI to pre-train the avatar with messenger app chat history and internet search tool shopping purchase history, it realizes a "conversational" and "push-type" avatar account that has the same characteristics as the user.

[1622] The server first retrieves the user's conversation history and purchase history from messenger apps. This data is collected using APIs from messenger apps and shopping sites. Next, the server uses generative AI to learn the user's characteristics based on this data. Specifically, the conversation history and purchase history are input as prompts into the generative AI model to learn the user's characteristics.

[1623] After learning is complete, the server generates an avatar account with the user's characteristics, and this avatar posts advertisements on social media and messaging apps on the user's behalf. This enables automatic information gathering and posting on behalf of the user, improving user convenience and allowing for effective ad delivery that reflects the user's characteristics and preferences.

[1624] The hardware used includes servers and user terminals (smartphones, tablets, PCs, etc.). The software includes messenger apps, shopping site APIs, and generative AI (for example, OpenAI APIs).

[1625] As a concrete example, consider a case where the user ID is "user123" and the advertisement content is "Check out this amazing product!". In this case, the server will operate as follows:

[1626] 1. The server retrieves the conversation history and purchase history of user ID "user123".

[1627] 2. The server uses generative AI to learn the characteristics of "user123" based on the acquired data.

[1628] 3. The server uses the trained user model to post an advertisement on social media saying, "Check out this amazing product!"

[1629] Examples of prompts to input into a generative AI model:

[1630] Train a model based on the following data: [Conversation history and purchase history data]

[1631] This prompt is used to train the generative AI model on user characteristics.

[1632] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1633] Step 1:

[1634] The server retrieves the user's conversation and purchase history from messenger apps. Specifically, it uses APIs from messenger apps and shopping sites to collect conversation and purchase history associated with the user ID. The input is the user ID, and the output is conversation and purchase history data.

[1635] Step 2:

[1636] The server uses generative AI to learn user characteristics based on acquired conversation and purchase history. Specifically, it inputs conversation and purchase history as prompts into the generative AI model to learn user characteristics. The input is conversation and purchase history data, and the output is a model that reflects the user's characteristics.

[1637] Step 3:

[1638] The server generates avatar accounts based on the learned user model. Specifically, it uses generative AI to create avatar accounts that possess the user's characteristics and provides them to the messenger app. The input is the user model, and the output is the avatar account.

[1639] Step 4:

[1640] The server uses avatar accounts to post advertisements on social media and messaging apps. Specifically, it generates ad content that reflects the user's characteristics and posts it using the APIs of the social media and messaging apps. The input is the user model and ad content, and the output is the ad post on the social media or messaging app.

[1641] Step 5:

[1642] The server monitors the effectiveness of the ads and updates the user model as needed. Specifically, it collects data such as ad click-through rates and engagement rates, and uses generative AI to retrain the user model. The input is the ad performance data, and the output is the updated user model.

[1643] (Example 3)

[1644] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1645] Traditional information systems and search engines have struggled to adequately reflect user preferences and tastes, making it difficult to provide users with the most relevant information. Furthermore, the effort required for users to search for and organize information themselves made efficient information gathering difficult.

[1646] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for learning user characteristics and providing information and generating search results on behalf of the user; means for providing a virtual account on the messaging application that acts as a proxy for the user; means for working in cooperation with a generative AI to pre-train messaging application talk history and internet search tool purchase history to realize an "interactive" and "push-type" virtual account with the same characteristics as the user; means for collecting user behavior data and analyzing the data using natural language processing technology; means for generating a profile that reflects the user's preferences and tastes based on the analysis results; means for the virtual account to automatically provide information and perform searches based on the generated profile; and means for generating customized information and search results based on the virtual account's behavior results. This makes it possible to generate optimal information and search results that reflect the user's preferences and tastes.

[1647] "User characteristics" refer to individual features such as a user's preferences, tastes, behavioral patterns, and interests.

[1648] "Information provision" refers to the act of providing useful information to users.

[1649] "Search results" refer to a list of information returned in response to a user's search on the internet.

[1650] A "messaging application" refers to software that allows users to send and receive text messages and multimedia messages.

[1651] A "virtual account" refers to a digital account that acts on behalf of a user.

[1652] "Generative AI" refers to artificial intelligence that has the ability to generate new information and content based on data.

[1653] "Pre-training" refers to the process by which a system learns using the user's past data.

[1654] "Interactive" refers to a system having the ability to engage in natural conversations with users.

[1655] "Push-type" refers to a system that automatically provides information without waiting for a user request.

[1656] "Behavioral data" refers to data related to user actions (e.g., social media posts, search history, message sending, etc.).

[1657] "Natural language processing technology" refers to the technology used to understand, analyze, and generate human language.

[1658] A "profile" refers to a collection of data generated based on a user's characteristics and behavioral patterns.

[1659] "Customized information" refers to information that has been individually tailored based on the user's characteristics and preferences.

[1660] Modes for carrying out the invention

[1661] This invention is a system that learns user characteristics and provides information and generates search results on behalf of the user. Specific embodiments of this system are described below.

[1662] 1. Generating the system program

[1663] The server generates programs that provide information and search results that reflect the user's preferences and tastes. These programs have the functionality to manage virtual accounts that automatically perform user actions such as posting on social media, sending messages, and performing web searches.

[1664] 2. Explanation of the program's processing

[1665] The server uses the generated program to perform the following operations.

[1666] 1. Data collection:

[1667] The server collects data such as users' social media posts, message transmissions, and web search history. This data collection is done by obtaining data from various social media platforms and search engines via APIs. Specifically, it uses the Twitter API and the Google Search API.

[1668] 2. Data Analysis:

[1669] The server uses natural language processing (NLP) techniques to analyze the collected data. Specifically, it uses software such as Google Cloud Natural Language API and IBM Watson Natural Language Understanding.

[1670] 3. Generating a user profile:

[1671] The server generates a profile that reflects the user's preferences and tastes based on the analysis results. This profile includes topics and keywords that the user is interested in.

[1672] 4. Simulation of virtual account behavior:

[1673] Based on the generated user profiles, the server automatically performs tasks such as posting to social media, sending messages, and performing web searches using virtual accounts. This simulation uses a generative AI model (for example, OpenAI's GPT-4).

[1674] 5. Information provision and search result generation:

[1675] The server generates customized information and search results for users based on the actions of their virtual accounts. This information is provided in a format that best suits the user's preferences and tastes.

[1676] 3. Specific Examples and Examples of Prompt Statements

[1677] Specific example:

[1678] Let's say user A is interested in travel. The server extracts keywords such as "travel," "tourist destinations," and "hotels" from user A's social media posts and web search history. Based on this, a virtual account automatically makes travel-related social media posts and searches for travel blogs. Finally, it provides user A with information on recommended tourist destinations and hotels.

[1679] Example of a prompt:

[1680] "To provide user A with travel-related information they might be interested in, please suggest recommended tourist destinations and hotels based on their social media posts and web search history."

[1681] In this way, the server realizes a system that generates information and search results that reflect the user's preferences and tastes. The flow of specific processing in Example 3 will be explained using Figure 15.

[1682] Step 1: Data Collection

[1683] The server collects data such as users' social media posts, message transmissions, and web search history. Specifically, it uses the Twitter API to retrieve users' tweets and the Google Search API to collect users' search history. The input is the user's account information, and the output is the collected behavioral data.

[1684] Step 2: Data Analysis

[1685] The server uses natural language processing (NLP) techniques to analyze the collected data. Specifically, it uses the Google Cloud Natural Language API to extract keywords from users' tweets and search history and perform sentiment analysis. The input is collected behavioral data, and the output is the analyzed keywords and sentiment information.

[1686] Step 3: Generate User Profile

[1687] The server generates a profile that reflects the user's preferences and tastes based on the analysis results. Specifically, it lists topics and keywords that the user is interested in, based on the extracted keywords and sentiment analysis results. The input is the analyzed keywords and sentiment information, and the output is the generated user profile.

[1688] Step 4: Virtual Account Behavior Simulation

[1689] Based on the generated user profile, the server automatically performs social media posting, message sending, and web searches using a virtual account. Specifically, it uses a generative AI model (e.g., OpenAI's GPT-4) to generate social media posts based on the user's interests and search for relevant websites. The input is the user profile, and the output is the result of the virtual account's actions.

[1690] Step 5: Information provision and search result generation

[1691] The server generates customized information and search results for users based on the actions of virtual accounts. Specifically, it organizes the information collected by virtual accounts and provides users with recommendations for tourist destinations and hotels. The input is the actions of virtual accounts, and the output is customized information and search results.

[1692] (Application Example 3)

[1693] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1694] Traditional systems have struggled to accurately reflect user preferences and tastes in providing information and generating search results. Furthermore, users had to manually post on social media, send messages, and perform web searches, which was time-consuming and laborious. Additionally, recommendation features were insufficient to suggest the most suitable products, failing to increase user purchasing intent. To address these challenges, there is a need for a system that automates user behavior, learns user characteristics, and provides optimal information and product recommendations.

[1695] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1696] In this invention, the server includes means for learning user characteristics and performing SNS, web searches, information organization, etc. on behalf of the user; means for providing an avatar account that acts as a surrogate for the user on a messenger app; means for collaborating with a generative AI to pre-train the AI ​​with messenger app chat history and internet search tool shopping purchase history to realize a "conversational" and "push-type" avatar account with the same characteristics as the user; means for analyzing the user's SNS posts, message sending, and web search history to learn the user's preferences and tastes; means for suggesting the most suitable products to the user; means for automatically posting product reviews and questions; and means for making recommendations based on purchase history. This makes it possible to automate user behavior and learn user characteristics to provide optimal information and product suggestions.

[1697] "User characteristics" refer to individual features such as a user's preferences, tastes, and behavioral patterns.

[1698] "SNS" is an abbreviation for Social Networking Service, which refers to a platform for users to interact with other users online.

[1699] "Web search" refers to the act of finding information on the internet using a search engine.

[1700] "Information organization" refers to the process of classifying and organizing collected data and information to make it easier to use.

[1701] A "messenger app" refers to application software used to send and receive text messages, images, videos, and other similar content.

[1702] An "avatar account" refers to a virtual account that acts as a digital representation of a user online.

[1703] "Generative AI" refers to artificial intelligence technology that generates new information and content based on data. 【170...

Claims

1. A system that learns the characteristics of a user and performs message sending, information retrieval, and information organization in information and communication services on behalf of the user, A means for collecting information and communication application history, internet search tool usage history, and purchase history, A means for generating a virtual account that conducts conversations and push notifications that reflect the user's characteristics based on the output of a generative AI model that has been pre-trained on the history of the information and communication application, the usage history, and the purchase history, Means for registering the virtual account in the aforementioned information and communication application, A means for preprocessing the collected data and extracting the characteristics of the user, A means for training the generative AI model using the extracted user characteristics, A means for collecting information related to the user's interests and concerns based on the user's interests and concerns identified in the extraction of the user's characteristics by the means for extracting the user's characteristics, and for pushing notifications to the user through the virtual account, A means for causing the generative AI model to generate a message that reflects the user's characteristics for sending to the friend, based on the conversation history between the user and their friend, and for sending the message to the friend through the virtual account, A means for recognizing the user's emotions using an emotion engine and adjusting the behavior of the virtual account based on the recognized emotions, wherein the behavior of the virtual account includes sending messages to friends that reflect the recognized emotions. A system that includes this.

2. The system according to claim 1, which, through the aforementioned pre-training, is capable of generating information and search results that reflect the user's preferences and tastes.

3. The system according to claim 1, wherein the virtual account is capable of automatically performing actions such as posting to information and communication services, sending messages, and searching for information.

Images