System

Abstract

A system is provided.SOLUTION: A system comprising: means for obtaining user data; means for analyzing the obtained data and generating a vector representation; means for creating a digital clone of the user using the generated vector representation; and means for the digital clone to operate on a predetermined task.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technologies have had difficulty efficiently utilizing a user's life log data to perform tasks on behalf of the user, assist in love matching, and select and remind important notifications and emails. In particular, systems that separate the analysis of user data from its application often result in reduced efficiency and a limited user experience. The present invention aims to solve these problems and provide a system that maximizes the use of user data. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. Specifically, a system is provided that includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, and means for the digital clone to perform a predetermined task. This system acquires messages, emails, and search queries as user data and stores the generated vector representation in a database. The digital clone can also be used to answer surveys and interviews on behalf of the user, or to match users with each other. Furthermore, it is possible to identify content that is likely to interest the user from newly received messages and emails and remind the user of the content. These means enable consistent and efficient use of user data and an improved user experience.

[0006] "User data" refers to life log information generated by a user, and specifically includes messages, emails, search queries, and the like.

[0007] A "vector representation" is obtained by expressing acquired user data as a numerical vector and converting it into a format that can be processed by a machine learning model.

[0008] A "digital clone" is a computer program that is generated based on a vector representation of a user and performs certain tasks on behalf of the user.

[0009] A "survey" is a set of questions designed to gather specific information, to which a user or digital clone provides answers.

[0010] "Hearing" is the process of asking questions interactively to gather specific information, to which the user or digital clone responds.

[0011] "Love matching" is the process of evaluating compatibility between users or digital clones and suggesting partners.

[0012] "Remind" is a function that notifies the user of important notifications or information again, in order to draw the user's attention.

[0013] A "predetermined task" refers to a specific task or operation that the digital clone performs on behalf of the user.

[0014] "Natural language processing engine" is a general term for programs and models that analyze text data and understand and generate human language.

[0015] A "database" is an information system for systematically storing and managing vector representations and user data. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a system that efficiently utilizes user data to generate a digital clone to perform various tasks. The system includes a means for acquiring user data and converting it into a vector representation to generate a digital clone that captures the user's characteristics. The generated digital clone is then used for specific tasks such as answering questionnaires, conducting interviews, and matching relationships. The system also includes a means for providing notifications and email reminders to the user.

[0038] System program processing explanation

[0039] Retrieving User Data

[0040] The user logs in and agrees to provide LINE chat data, emails, search queries, etc. The device collects this data with the user's consent.

[0041] Analyzing data and generating vector representations

[0042] The server analyzes the collected user data and generates vector representations. This analysis uses a natural language processing engine to tokenize the data and preprocess it to understand its context. The preprocessed data is then fed into an embedding model to generate vector representations. The server stores the generated vectors in a database.

[0043] Creating a digital clone and performing tasks

[0044] If a user wishes to create a digital clone, the server uses the stored vector data to generate a digital clone that reflects the user's characteristics and performs specific tasks on the user's behalf.

[0045] For example, in the case of a survey, the server inputs the survey contents into the digital clone, generates and sends appropriate answers, and in the case of love matching, the server compares multiple digital clones, finds compatible pairs, and notifies the user of the results.

[0046] Notifications and reminders

[0047] The server identifies new messages and emails that the user is likely to be interested in, analyzes their importance, and then sends push notifications to the device to remind the user of the important content.

[0048] Specific examples

[0049] 1. The user logs in to the system and agrees to provide data. The device collects LINE chat data, emails, and search queries.

[0050] 2. The server analyzes the collected data, generates a vector representation, and stores it in a database.

[0051] 3. The user requests the creation of a digital clone, and the server uses the generated vector data to create the digital clone.

[0052] 4. To answer the survey, the server inputs the survey details into the digital clone, generates the appropriate answers, and sends them.

[0053] 5. For love matching, the server compares multiple clones and suggests compatible pairs.

[0054] 6. The server analyzes new messages and emails and sends important information via push notification to the device to remind the user.

[0055] Thus, the present invention provides a versatile system for effectively utilizing user data and enhancing the user experience.

[0056] The processing flow will be explained below.

[0057] Specific flow of program processing

[0058] Processing from acquiring user data to utilizing digital clones

[0059] Step 1:

[0060] A user logs in to the system.

[0061] The user enters their authentication information and accesses the login screen.

[0062] The server verifies the authentication information and establishes a session for the user.

[0063] Step 2:

[0064] The user agrees to provide the data.

[0065] The user reads the terms of use and selects a checkbox to agree to providing data.

[0066] The terminal transmits the user's consent to the server.

[0067] Step 3:

[0068] The terminal collects user data.

[0069] The device downloads the chat data through the LINE app's API.

[0070] The terminal retrieves received emails using the mail server's API.

[0071] The device collects search queries using the browser's history API.

[0072] Step 4:

[0073] The server parses the user data and generates a vector representation.

[0074] The server sends the collected messages, emails, and search queries to a natural language processing engine.

[0075] A natural language processing engine tokenizes the data and analyzes its contextual information.

[0076] The server inputs the preprocessed data into an embedding model (e.g., BERT, GPT) to generate vectors.

[0077] Step 5:

[0078] The server stores the generated vector in a database.

[0079] The server stores the vector data generated for each user ID in a database.

[0080] A database associates the vector data with a user profile.

[0081] Step 6:

[0082] A user wants to create a digital clone.

[0083] The user selects the digital cloning option on the settings page and sends a request to the server.

[0084] Step 7:

[0085] The server uses the stored vector data to generate a digital clone.

[0086] The server inputs the user's vector data into an LLM (Large Scale Language Model).

[0087] LLM generates a digital clone that reflects the user's characteristics.

[0088] Step 8:

[0089] The server uses a digital clone to perform tasks.

[0090] The server passes the survey results to the digital clone.

[0091] The digital clone generates appropriate answers to the questions and sends them to the survey system.

[0092] Step 9:

[0093] A user registers for a dating service.

[0094] Users enter profile information and preferences and submit them to the server.

[0095] Step 10:

[0096] The server compares multiple digital clones and finds compatible pairs.

[0097] The server analyzes the digital clones of all registered users and compares their feature vectors.

[0098] The server pairs compatible clones using a matching algorithm.

[0099] Step 11:

[0100] The server analyzes the importance of new messages and emails.

[0101] The device collects new notifications and emails in real time.

[0102] The server inputs the collected data into a machine learning model to calculate an importance score.

[0103] Step 12:

[0104] The server reminds the user of important content.

[0105] The server selects important notifications and emails and generates reminder content.

[0106] Send a push notification to the device and display a reminder to the user.

[0107] The above is the processing flow related to the system of the present invention. In this way, user data can be effectively utilized and functions suited to the user can be provided.

[0108] Example 1

[0109] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0110] In today's information society, the amount of data generated by users every day is steadily increasing. However, it is not easy to efficiently analyze this vast amount of data and provide services and notifications that meet user needs. In particular, there is a lack of systems that can generate digital clones that perform tasks on behalf of users and that can provide important notifications in a timely manner. There is a need for systems that can solve these technical challenges and improve user convenience.

[0111] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0112] In this invention, the server includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, means for the digital clone to perform a predetermined task, means for analyzing the content of newly arrived messages and emails and identifying important content, and means for sending the identified important content to the terminal by push notification. This makes it possible to effectively utilize user data and provide services that meet the user's needs.

[0113] "User Data" means information generated or provided by a user, including messages, emails, search queries, and the like.

[0114] "Analysis" refers to the process of analyzing collected data and converting it into meaningful information.

[0115] "Vector representation" is a format in which data is quantified and represented in a multidimensional space. This format makes it possible to efficiently handle the characteristics and relationships of data.

[0116] A "digital clone" is a virtual agent that reflects the user's characteristics and can perform specific tasks on the user's behalf.

[0117] "Tasks" refer to specific tasks or activities that digital clones perform, such as answering surveys or matching romantic partners.

[0118] "New messages" refers to messages or notifications that the user has recently received.

[0119] "Email" means a digital message sent or received over the Internet.

[0120] A "push notification" is an instantaneous notification sent from a server to a terminal, and is a means of quickly conveying important information to a user.

[0121] The present invention relates to a system that efficiently utilizes user data to generate a digital clone to perform various tasks. The system includes a means for acquiring user data and converting it into a vector representation to generate a digital clone that captures the user's characteristics. The generated digital clone is then used for specific tasks such as answering questionnaires, conducting interviews, and matching relationships. The system also includes a means for providing notifications and email reminders to the user.

[0122] Specifically, to obtain user data, users must log in to the system and agree to provide LINE chat data, emails, search queries, etc. The device collects this data with the user's consent. The collected data is encrypted and sent to the server.

[0123] The server then analyzes the collected user data and generates vector representations. This analysis involves using the Google NLP API to tokenize the data and preprocess it to understand its context. The preprocessed data is then fed into an embedding model (e.g., Word2Vec or BERT) to generate vector representations. The server then stores the generated vectors in a database (e.g., MySQL).

[0124] If a user wishes to create a digital clone, the server uses the stored vector data to generate a digital clone that reflects the user's characteristics and can perform specific tasks (e.g., answering surveys or matching relationships) on the user's behalf.

[0125] For example, in the case of a survey, the server inputs the survey contents into the digital clone, generates and sends appropriate answers, and in the case of love matching, the server compares multiple digital clones, finds compatible pairs, and notifies the user of the results.

[0126] The server also identifies content that may be of interest to the user from among newly received messages and emails, analyzes its importance, and sends push notifications to the device to remind the user of the identified important content.

[0127] Thus, the present invention provides a versatile system for effectively utilizing user data and improving user experience.

[0128] Examples of prompt statements

[0129] 1. Survey prompt:

[0130] "Please share your opinion on the following survey question: 'How do you like to spend your holidays?'"

[0131] 2. Romance Matching Prompt:

[0132] "What is your favorite movie genre and why?"

[0133] 3. Prompt for notification reminder:

[0134] "Check for the new message below: 'Tomorrow's meeting time has changed.'"

[0135] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0136] Step 1:

[0137] The user logs in to the system and agrees to provide data. User ID and password are required as input. The output is a successful login status and confirmation of data provision agreement. Specifically, the user enters authentication information into the login form and checks the checkbox for data provision.

[0138] Step 2:

[0139] With the user's consent, the device collects user data such as LINE chat data, emails, and search queries. Raw data provided by various apps and browsers is required as input. The collected data is encrypted and sent to the server as output. Specifically, the device collects data in the background, encrypts it with RSA, and uploads it to the server.

[0140] Step 3:

[0141] The server receives and decrypts the collected encrypted data. It requires encrypted user data as input and decrypts the decrypted text data as output. Specifically, the server uses an RSA decryption key to decrypt the received data and convert it into an analyzable format.

[0142] Step 4:

[0143] The server tokenizes and preprocesses the data using the Google NLP API. The input requires decrypted user data. The output is tokenized data. Specifically, the server calls the NLP API to split sentences and generate tokens.

[0144] Step 5:

[0145] The server inputs the preprocessed data into an embedding model (such as Word2Vec or BERT) to generate a vector representation. Tokenized data is required as input, and a vector representation is obtained as output. Specifically, the server calls the embedding model library to vectorize the data.

[0146] Step 6:

[0147] The server saves the generated vector to the database. It requires a vector representation as input and a confirmation of saving to the database as output. Specifically, the server connects to MySQL and uses the INSERT statement to insert the vector data into a table.

[0148] Step 7:

[0149] A user wishes to create a digital clone and submits a request. The input required is a clone creation request from the user. The output is a confirmation of receipt of the clone creation request. The specific operation is when the user clicks the clone creation button on a dedicated dashboard.

[0150] Step 8:

[0151] The server receives the request and generates a digital clone using the stored vector data. The input is vector data. The output is a digital clone. Specifically, the server reads the vector data and runs the clone generation algorithm.

[0152] Step 9:

[0153] The digital clone performs tasks such as answering surveys and matching. Survey questions and matching conditions are required as input. The output is the results of the task execution and matching. Specifically, the server inputs the survey questions into the clone, generates answers, and sends them.

[0154] Step 10:

[0155] The server identifies new messages or emails that are likely to be of interest to users and analyzes their importance. The input required is the new messages or emails. The output is the identified important content. Specifically, the server uses a text analysis algorithm to evaluate the importance of the messages.

[0156] Step 11:

[0157] The server sends a push notification to the device to remind the user of the identified important information. The input required is the identified important information. The output is a confirmation of the push notification. Specifically, the server sends the notification to the device using the Firebase Cloud Messaging (FCM) API.

[0158] (Application example 1)

[0159] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0160] Conventional virtual store shopping experiences often fail to fully address individual user preferences, potentially resulting in low user satisfaction. Furthermore, users' questions are not answered in real time, limiting the user experience. Therefore, there is a need for a system that can recommend products based on user preferences and quickly respond to user questions.

[0161] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0162] In this invention, the server includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, means for the digital clone to perform a predetermined task, means for recommending products based on the user's preferences, and means for responding to user questions, thereby enabling product recommendations based on the user's individual preferences, improving the shopping experience in the virtual store, and enabling quick responses to user questions.

[0163] "User Data" refers to any information about a user, including messages, emails, search queries, and other data.

[0164] "Vector representation" refers to the technique of converting text and other data into numerical vectors, a format that can be processed by a computer.

[0165] A "digital clone" is a virtual model that captures a user's characteristics and performs various tasks on their behalf.

[0166] "Prescribed tasks" refer to specific tasks that the digital clone will perform, such as answering surveys or matching romantic partners.

[0167] "Product recommendation tools" refers to algorithms and systems that suggest appropriate products based on a user's preferences and past behavior.

[0168] "Means for responding to questions" refers to a function for providing appropriate answers to inquiries and questions from users.

[0169] This invention is a system that utilizes user data to generate a digital clone that captures the user's characteristics and automatically executes various tasks. A specific embodiment of this system will be described below.

[0170] System Overview

[0171] The system mainly includes the following elements:

[0172] 1. How to obtain user data

[0173] 2. Data analysis and vector representation generation

[0174] 3. How to create a digital clone

[0175] 4. A means of accomplishing a specific task

[0176] 5. A way to recommend products based on user preferences

[0177] 6. Means of responding to user questions

[0178] Hardware and software used

[0179] Hardware: Smartphones, servers

[0180] Software: Python, scikit-learn (natural language processing engine), TfidfVectorizer (text vectorization), cosine_similarity (similarity calculation)

[0181] Program processing overview

[0182] 1. How to obtain user data

[0183] Users log in through a smartphone application and agree to provide data, which is then collected, including messages, emails, and search queries.

[0184] 2. Data analysis and vector representation generation

[0185] The server analyzes the collected user data, tokenizing it using TfidfVectorizer to generate vector representations that represent the user's characteristics in numerical form.

[0186] 3. How to create a digital clone

[0187] A digital clone is generated based on the analyzed vector data, and the digital clone can perform various tasks while reflecting the user's characteristics.

[0188] 4. A means of accomplishing a specific task

[0189] Digital clones can automatically perform tasks such as answering surveys and matching people for love.

[0190] 5. A way to recommend products based on user preferences

[0191] The digital clone selects products from a virtual store based on the user's purchase history and preference data, and recommends the most suitable products to the user.

[0192] 6. Means of responding to user questions

[0193] When users ask questions through chat, the digital clone answers in real time.

[0194] Specific examples

[0195] 1. Example of user data acquisition

[0196] A user logs into the application and provides data such as "I like casual clothes" and "I like the color blue."

[0197] 2. Example of vector representation generation

[0198] The server parses the provided data, tokenizes the text with TfidfVectorizer, and generates a vector representation.

[0199] 3. Specific examples of product endorsements

[0200] Based on vector data, the system recommends the most suitable item from a virtual store, such as an elegant black dress, casual blue jeans, or a minimalist white T-shirt.

[0201] 4. Example of responding to user questions

[0202] When the user asks, "What is my favorite color?" the digital clone responds, "Your favorite color is blue."

[0203] Prompt Sentence Examples

[0204] "Based on your data, our virtual shopping assistant will recommend the best products for you. Please let us know if you have a preference for casual clothing or specific colors."

[0205] The above is a description of a specific embodiment for carrying out the present invention. This system enables a user to have a highly personalized shopping experience, thereby significantly improving user satisfaction.

[0206] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0207] Step 1:

[0208] Retrieving User Data

[0209] When a user logs in to a smartphone application and agrees to provide data, the device collects the user's message data, emails, search queries, etc. The input is the data provided by the user, and the output is the collected raw user data.

[0210] Step 2:

[0211] Data analysis and vector representation generation

[0212] The server receives the collected user data and performs analysis. Specifically, it uses Python and scikit-learn's TfidfVectorizer to tokenize the data and generate a vector representation. The input is raw user data, and the output is user data represented as a numeric vector.

[0213] Step 3:

[0214] Saving vector data

[0215] The generated vector representation is saved in a database on the server, making it available for subsequent tasks. The input is a numeric vector, and the output is vector data stored in the database.

[0216] Step 4:

[0217] Creating a digital clone

[0218] When a user requests the creation of a digital clone, the server uses the stored vector data to create the digital clone. The input is the vector data stored in the database, and the output is a digital clone that mimics the user's characteristics.

[0219] Step 5:

[0220] Executing a task

[0221] The digital clone performs a given task on behalf of the user, for example, making product recommendations in a virtual store: the input is data about the user's purchasing history and preferences, and the output is a list of products recommended to the user.

[0222] Step 6:

[0223] Question Processing

[0224] When a user inputs a question through a smartphone application, the server instantly generates an answer using the digital clone. The input is the user's question, and the output is the generated answer. For example, if the user asks, "What is my favorite color?", the digital clone will respond, "Your favorite color is blue."

[0225] Step 7:

[0226] Best product recommendations

[0227] The server compares the user's vector data with the product data of the virtual store and recommends the best products using cosine_similarity. The input is the user vector data and the product data vector, and the output is a list of products that are judged to be best for the user.

[0228] The above are the specific processing steps of the system for implementing the present invention, which provides users with an optimal shopping experience and question responses.

[0229] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0230] The present invention relates to a system that efficiently utilizes user data to generate a digital clone to perform various tasks. Furthermore, by combining it with an emotion engine that analyzes user data and recognizes the user's emotions, it is possible to provide more advanced services. This system includes a means for acquiring user data, analyzing it, and converting it into a vector representation to generate a digital clone that captures the user's characteristics and emotions. The generated digital clone is then used for specific tasks such as answering surveys, conducting interviews, and matching relationships. The system also includes a means for providing notifications and email reminders to the user.

[0231] System program processing explanation

[0232] Retrieving User Data

[0233] The user logs in and agrees to provide LINE chat data, emails, search queries, etc. The device collects this data with the user's consent.

[0234] Analyzing data and generating vector representations

[0235] The server analyzes the collected user data and recognizes the user's emotions using an emotion engine. This emotion information and user data are analyzed comprehensively to generate a vector representation. A natural language processing engine is used for the analysis, tokenizing the data and performing preprocessing to understand the context and emotion. The preprocessed data is then input into an embedding model to generate a vector representation. The generated vector is then stored in a database by the server.

[0236] Creating a digital clone and performing tasks

[0237] If a user wishes to create a digital clone, the server will use the stored vector data to generate a digital clone that reflects the user's characteristics and emotions and performs specific tasks on the user's behalf.

[0238] For example, in the case of a survey, the server inputs the survey contents into the digital clone, generates and sends appropriate answers, and in the case of love matching, the server compares multiple digital clones, finds compatible pairs, and notifies the user of the results.

[0239] Notifications and reminders

[0240] The server identifies new messages and emails that the user is likely to be interested in, analyzes their importance, and then sends push notifications to the device to remind the user of the important content.

[0241] Specific examples

[0242] 1. The user logs in to the system and agrees to provide data. The device collects LINE chat data, emails, and search queries.

[0243] 2. The server analyzes the collected data, uses an emotion engine to recognize the user's emotions, and reflects them in a vector representation.

[0244] 3. The server stores the generated vector in a database.

[0245] 4. The user requests the creation of a digital clone, and the server uses the generated vector data to create the digital clone.

[0246] 5. To answer the survey, the server passes the survey contents to the digital clone, which generates and transmits the appropriate answers.

[0247] 6. For love matching, the server compares multiple clones and suggests compatible pairs.

[0248] 7. The server analyzes new messages and emails and sends important information via push notification to the device to remind the user.

[0249] In this way, the present invention can provide a multi-functional system for improving the user experience by effectively utilizing user data and further recognizing the user's emotions.

[0250] The processing flow will be explained below.

[0251] Specific flow of program processing

[0252] Processing from acquiring user data to utilizing digital clones

[0253] Step 1:

[0254] A user logs in to the system.

[0255] The user enters their authentication information and accesses the login screen.

[0256] The server verifies the authentication information and establishes a session for the user.

[0257] Step 2:

[0258] The user agrees to provide the data.

[0259] The user reads the terms of use and selects a checkbox to agree to providing data.

[0260] The terminal transmits the user's consent to the server.

[0261] Step 3:

[0262] The terminal collects user data.

[0263] The device downloads the chat data through the LINE app's API.

[0264] The terminal retrieves received emails using the mail server's API.

[0265] The device collects search queries using the browser's history API.

[0266] Step 4:

[0267] The server parses the user data and generates a vector representation.

[0268] The server sends the collected messages, emails, and search queries to a natural language processing engine.

[0269] A natural language processing engine tokenizes the data and analyzes its contextual information.

[0270] The server inputs the preprocessed data into an embedding model (e.g., BERT, GPT) to generate vectors.

[0271] Step 5:

[0272] The server uses an emotion engine to recognize the user's emotion.

[0273] The server inputs the preprocessed data into an emotion engine to extract emotion information (e.g., joy, sadness, anger).

[0274] The server integrates the emotion information obtained from the emotion engine into a vector representation.

[0275] Step 6:

[0276] The server stores the generated vector in a database.

[0277] The server stores the vector data generated for each user ID in a database.

[0278] A database associates the vector data with a user profile.

[0279] Step 7:

[0280] A user wants to create a digital clone.

[0281] The user selects the digital cloning option on the settings page and sends a request to the server.

[0282] Step 8:

[0283] The server uses the stored vector data to generate a digital clone.

[0284] The server inputs the user's vector data into an LLM (Large Scale Language Model).

[0285] LLM creates a digital clone that reflects the user's characteristics and emotions.

[0286] Step 9:

[0287] The server uses a digital clone to perform tasks.

[0288] The server passes the survey results to the digital clone.

[0289] The digital clone generates appropriate answers to the questions and sends them to the survey system.

[0290] Step 10:

[0291] A user registers for a dating service.

[0292] Users enter profile information and preferences and submit them to the server.

[0293] Step 11:

[0294] The server compares multiple digital clones and finds compatible pairs.

[0295] The server analyzes the digital clones of all registered users and compares their feature vectors.

[0296] The server pairs compatible clones using a matching algorithm.

[0297] Step 12:

[0298] The server analyzes the importance of new messages and emails.

[0299] The device collects new notifications and emails in real time.

[0300] The server inputs the collected data into a machine learning model to calculate an importance score.

[0301] Step 13:

[0302] The server reminds the user of important content.

[0303] The server selects important notifications and emails and generates reminder content.

[0304] Send a push notification to the device and display a reminder to the user.

[0305] The above is the flow of specific processing steps related to the system of the present invention. In this way, by effectively utilizing user data and incorporating emotion information, it is possible to provide advanced services to users.

[0306] Example 2

[0307] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0308] Conventional systems that use user data have had difficulty in accurately recognizing user emotions and reflecting them in services. Furthermore, systems that notify and remind users have been unable to effectively deliver information of interest to users. Furthermore, there has been a lack of technology that can faithfully reflect a user's characteristics and emotions when performing tasks using a digital clone.

[0309] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0310] In this invention, the server includes a means for acquiring user data, a means for analyzing the acquired data and generating a vector representation, and a means for creating a digital clone of the user using the generated vector representation. This makes it possible to recognize the user's emotions and reflect the analyzed emotional information in the vector representation. Furthermore, by including a means for providing notifications and reminders to the user, it is possible to accurately deliver information that interests the user.

[0311] "User Data" refers to any information related to a user's activities, including messages, emails, search queries, etc.

[0312] "Vector representation" is a representation of extracted data features as a numerical vector, and is used in data analysis and machine learning.

[0313] A "digital clone" is a virtual copy of a user that reflects their characteristics and emotions and can perform tasks on their behalf.

[0314] An "emotion engine" refers to a system or algorithm that recognizes and analyzes emotions from user data.

[0315] "Notification" means information sent by the system to the user to inform them of new messages or important information.

[0316] A "remind" is a notification or announcement that reminds users of important information or tasks that they tend to forget.

[0317] The present invention relates to a system that efficiently utilizes user data, generates digital clones, and executes various tasks. A specific embodiment of this system will be described below.

[0318] First, the user logs in to the system and agrees to provide data such as LINE chat data, emails, and search queries. After obtaining the user's consent, the device collects this data and sends it to the server. This data collection and transmission is encrypted using the HTTPS protocol.

[0319] The server then receives the collected user data. This data is temporarily stored in a database and prepared for analysis. The server is equipped with a natural language processing engine, which uses models such as BERT and GPT-3 to tokenize and preprocess the data. After tokenization, the data is split into sentences and words and unwanted noise is removed.

[0320] The server then uses an emotion engine to analyze the user's emotions. For emotion recognition, it uses an LSTM model or similar. The server then inputs the preprocessed data into an embedding model (e.g., Word2Vec or FastText) to generate a vector representation for each piece of data. The server then stores these vectors in a database and associates them with the user ID.

[0321] When a user requests the creation of a digital clone, the server retrieves vector data associated with the user ID from the database. The server then inputs the vector data into a generative AI model (e.g., GPT-3) to generate a digital clone that reflects the user's characteristics and emotions. This digital clone can then perform specific tasks on the user's behalf.

[0322] For example, when answering a questionnaire, the server provides the digital clone with the questionnaire content, which generates and transmits the appropriate answer. In the case of love matching, the server compares multiple digital clones to find the most compatible pair.

[0323] Finally, the server periodically checks for new messages and emails, analyzes their content using Natural Language Understanding (NLU), identifies content that may be of interest to the user, scores its importance, and sends push notifications about the identified important content to the device. The device receives the push notifications and reminds the user.

[0324] As a concrete example, the following prompt sentence is input to the generative AI model:

[0325] "User: Please create a digital clone of yourself that will answer the following survey:

[0326] 1. What is your favorite food?

[0327] 2. What are your hobbies?

[0328] 3. Tell us about your most recent travel destination.

[0329] Following these prompts, the generative AI model generates appropriate answers that reflect the user's characteristics and emotions.

[0330] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0331] Step 1: Get User Data

[0332] 1. The user logs into the system and enters their credentials.

[0333] Input: User authentication information (user ID, password).

[0334] Output: Authentication success message, user ID.

[0335] 2. The user agrees to provide LINE chat data, emails, and search queries.

[0336] Input: User consent.

[0337] Output: Consent confirmation message.

[0338] 3. After obtaining the user's consent, the device collects LINE chat data, emails, and search history and sends them to the server. The transmission uses the HTTPS protocol and the data is encrypted.

[0339] Input: User's LINE chat data, emails, and search history.

[0340] Output: Encrypted user data.

[0341] Step 2: Analyze the data and generate a vector representation

[0342] 1. The server receives the collected user data and temporarily stores it in a database.

[0343] Input: Encrypted user data.

[0344] Output: Raw data stored in a database.

[0345] 2. The server uses a natural language processing engine to tokenize and preprocess the data, specifically splitting the text data into sentences and words and filtering out unnecessary noise.

[0346] Input: Stored raw data.

[0347] Output: Tokenized data.

[0348] 3. The emotion engine analyzes the tokenized data and recognizes the user's emotions. It uses an LSTM model or similar.

[0349] Input: Tokenized data.

[0350] Output: Emotional information.

[0351] 4. The natural language processing engine inputs the preprocessed data into an embedding model (e.g., Word2Vec or FastText) to generate a vector representation.

[0352] Input: Tokenized data and sentiment information.

[0353] Output: Vector representation.

[0354] 5. The generated vector is stored in the database and associated with the user ID.

[0355] Input: Vector representation and user ID.

[0356] Output: Vector data stored in a database.

[0357] Step 3: Generate a digital clone and perform the task

[0358] 1. If a user wishes to create a digital clone, they submit a request.

[0359] Input: Digital cloning request.

[0360] Output: Request received message.

[0361] 2. The server retrieves vector data associated with the user ID from the database.

[0362] Input: User ID.

[0363] Output: Vector data.

[0364] 3. The server inputs the vector data into a generative AI model (e.g., GPT-3) to generate a digital clone.

[0365] Input: Vector data.

[0366] Output: The generated digital clone.

[0367] 4. The digital clone performs specific tasks.

[0368] In the case of a survey response: The server provides the survey content to the digital clone, which generates and transmits the response.

[0369] Input: Survey contents.

[0370] Output: The generated answer.

[0371] In the case of love matching: The server compares multiple digital clones and finds compatible pairs.

[0372] Enter: multiple digital clones.

[0373] Output: Results of compatible pairs.

[0374] Step 4: Notifications and reminders

[0375] 1. The server periodically checks for new messages and emails.

[0376] Input: New Message, Email.

[0377] Output: A list of messages and emails.

[0378] 2. The server uses NLU to analyze the content of the message or email to identify content that the user is likely to be interested in. An importance score is calculated.

[0379] Input: message, email.

[0380] Output: Importance score.

[0381] 3. The server selects content with a high importance score and sends a notification to the device using the push notification API.

[0382] Input: Content with high importance score.

[0383] Output: Push notification.

[0384] 4. The device receives a push notification and reminds the user.

[0385] Enter: push notification.

[0386] Output: Remind notification.

[0387] At each step, specific actions are performed, and the entire system functions by repeating the input, processing, and output of data.

[0388] (Application example 2)

[0389] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0390] While conventional digital clone generation systems have a certain degree of accuracy in analyzing user data and performing tasks, they have limitations in providing more personalized services by incorporating the user's emotional state and real-time visual information. Furthermore, they have the problem of being unable to suggest products based on the user's emotions and interests when it comes to purchasing behavior in physical stores.

[0391] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, means for the digital clone to perform a predetermined task, means for recognizing the user's emotions, means for acquiring visual data from a visual device worn by the user, and means for suggesting products based on the visual data and the user's emotions. This makes it possible to provide highly personalized services based on the user's emotional state and real-time visual information.

[0392] "User Data" is information provided by a user, including messages, emails, search queries, and purchase history data.

[0393] A "vector representation" is a representation of acquired user data quantified and expressed as a multidimensional vector.

[0394] A "digital clone" is a virtual being that reflects the user's characteristics and emotions and performs various tasks on the user's behalf.

[0395] An "emotion recognition means" is a device or software that uses algorithms or models to analyze a user's emotional state.

[0396] A "visual device" is a device worn by a user to capture visual data, such as smart glasses.

[0397] "Visual data" refers to image or video information obtained from a visual device.

[0398] A "means for suggesting products" is a device or software that uses algorithms or models to suggest optimal products to a user based on the user's emotional state and visual data.

[0399] The present invention provides a system that efficiently utilizes user data and suggests products in physical stores based on the user's emotional state and visual information. This system acquires and analyzes user data to generate a digital clone, and further recognizes the user's emotions using an emotion engine, thereby realizing more advanced services.

[0400] Overall system configuration

[0401] The system consists of the following main components:

[0402] 1. User data acquisition means: A means for collecting messages, emails, search queries, and purchase history data provided by users. This means also acquires visual data through smart glasses.

[0403] 2. Data analysis means: Analyzes collected user data and recognizes user emotions using an emotion engine. Specifically, it uses an NLP model (natural language processing engine) and an emotion recognition engine.

[0404] 3. Vector representation generation: This is a method for generating vector representations that reflect the user's characteristics and emotions based on the analyzed data. An embedding model is used.

[0405] 4. Digital clone generation means: A means for creating a digital clone of the user using the generated vector representation. The digital clone performs tasks on behalf of the user.

[0406] 5. Task execution means: This is the means by which the digital clone performs tasks such as making purchasing suggestions on behalf of the user.

[0407] 6. Notifications and reminders: This is a method of sending product suggestions and new message reminders to users' devices via push notifications.

[0408] Specific examples of programs

[0409] 1. User data acquisition

[0410] With the user's consent, the server collects LINE chat data, emails, search queries, and purchase history data. It also collects visual data of users wearing smart glasses in real time.

[0411] 2. Data analysis and vector representation generation

[0412] The server analyzes the collected user data and converts the user's emotions and characteristics into vector representations using an emotion engine and NLP model.

[0413] 3. Creating a digital clone and performing tasks

[0414] When a user requests the creation of a digital clone, the server uses the stored vector data to create a digital clone that reflects the user's characteristics and emotions. The digital clone can then perform tasks such as product recommendations.

[0415] 4. Notifications and Reminders

[0416] The server sends important information and suggestions to the user's device, such as smart glasses, via push notifications to remind the user.

[0417] Hardware and software used

[0418] Hardware: Smart glasses, device (smartphone)

[0419] Software: Emotion recognition engine (e.g., DistilBERT), NLP model, digital clone generation engine, embedded model, external API (user data acquisition API, product recommendation API)

[0420] Specific examples

[0421] For example, imagine a user is in the cosmetics section. The smart glasses detect the specific cosmetics the user is interested in and, based on the user's past purchase data and real-time emotional data, suggest other users' reviews of that cosmetics, similar products, and special offers. This information is displayed via push notification on the smart glasses.

[0422] Prompt Sentence Examples

[0423] User ID: user_123

[0424] Store ID: store_789

[0425] Visual data: path / to / sample_image.jpg

[0426] [the purpose]

[0427] Appropriate sentiment analysis is performed on the products and locations that the user is interested in, and the most suitable products are suggested for the user.

[0428] [Provided data]

[0429] 1. User purchase history data

[0430] 2. Real-time user visual data

[0431] 3. In-store product data

[0432] [Expected output]

[0433] 1. The user's emotional state

[0434] 2. List of recommended products

[0435] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0436] Step 1:

[0437] The user logs into the system and agrees to provide data.

[0438] Input: User-provided messages, emails, search queries, and purchase history data

[0439] Output: Collected user data

[0440] Specific operation: When the user clicks the consent button, the device will collect LINE chat data, email content, past search queries, and purchase history data and send them to the server.

[0441] Step 2:

[0442] The server analyzes the collected user data and recognizes the user's emotions using an emotion engine (e.g., DistilBERT).

[0443] Input: Collected user data

[0444] Output: Analysis results including emotional information

[0445] What it does: The server tokenizes the collected data and uses an emotion recognition algorithm to determine the emotional state of each message or email, then stores the results in a database.

[0446] Step 3:

[0447] The server uses the parsed data to generate a vector representation.

[0448] Input: Analysis results including emotion information

[0449] Output: A vector representation reflecting the user's features and emotions

[0450] How it works: The server uses the NLP engine and embedding model to convert user data into multidimensional vectors as semantic vectors, which are then stored in a database.

[0451] Step 4:

[0452] When a user requests the creation of a digital clone, the server uses the stored vector data to generate a digital clone that reflects the user's characteristics and emotions.

[0453] Input: Vector representation

[0454] Output: Digital Clone

[0455] How it works: The server uses the stored vector data to generate a digital clone that mimics the user's behavioral patterns and emotions.

[0456] Step 5:

[0457] When a user puts on the smart glasses and enters a physical store, the device captures visual data in real time.

[0458] Input: Visual data

[0459] Output: Captured image and video data

[0460] How it works: The camera in the smart glasses captures visual data of the surroundings and transmits it to a server in real time.

[0461] Step 6:

[0462] The server uses real-time visual data and vector-expressed user data to suggest optimal products to users.

[0463] Input: visual data, vector representation

[0464] Output: A list of product suggestions

[0465] How it works: The server analyzes visual data to identify products the user is interested in. It then analyzes the user's emotional state and purchasing history to generate suggestions for similar products and review information.

[0466] Step 7:

[0467] The server sends the suggested product information to the smart glasses via push notification to notify the user.

[0468] Input: Product suggestion list

[0469] Output: Push notification product information

[0470] Specific operation: The server sends the generated product suggestions to the smart glasses as a push notification and displays them in the user's field of view, allowing the user to check product information in real time.

[0471] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0472] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0473] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0474] [Second embodiment]

[0475] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0476] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0477] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0478] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0479] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0480] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0481] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0482] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0483] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0484] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0485] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0486] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0487] The present invention relates to a system that efficiently utilizes user data to generate a digital clone to perform various tasks. The system includes a means for acquiring user data and converting it into a vector representation to generate a digital clone that captures the user's characteristics. The generated digital clone is then used for specific tasks such as answering questionnaires, conducting interviews, and matching relationships. The system also includes a means for providing notifications and email reminders to the user.

[0488] System program processing explanation

[0489] Retrieving User Data

[0490] The user logs in and agrees to provide LINE chat data, emails, search queries, etc. The device collects this data with the user's consent.

[0491] Analyzing data and generating vector representations

[0492] The server analyzes the collected user data and generates vector representations. This analysis uses a natural language processing engine to tokenize the data and preprocess it to understand its context. The preprocessed data is then fed into an embedding model to generate vector representations. The server stores the generated vectors in a database.

[0493] Creating a digital clone and performing tasks

[0494] If a user wishes to create a digital clone, the server uses the stored vector data to generate a digital clone that reflects the user's characteristics and performs specific tasks on the user's behalf.

[0495] For example, in the case of a survey, the server inputs the survey contents into the digital clone, generates and sends appropriate answers, and in the case of love matching, the server compares multiple digital clones, finds compatible pairs, and notifies the user of the results.

[0496] Notifications and reminders

[0497] The server identifies new messages and emails that the user is likely to be interested in, analyzes their importance, and then sends push notifications to the device to remind the user of the important content.

[0498] Specific examples

[0499] 1. The user logs in to the system and agrees to provide data. The device collects LINE chat data, emails, and search queries.

[0500] 2. The server analyzes the collected data, generates a vector representation, and stores it in a database.

[0501] 3. The user requests the creation of a digital clone, and the server uses the generated vector data to create the digital clone.

[0502] 4. To answer the survey, the server inputs the survey details into the digital clone, generates the appropriate answers, and sends them.

[0503] 5. For love matching, the server compares multiple clones and suggests compatible pairs.

[0504] 6. The server analyzes new messages and emails and sends important information via push notification to the device to remind the user.

[0505] Thus, the present invention provides a versatile system for effectively utilizing user data and enhancing the user experience.

[0506] The processing flow will be explained below.

[0507] Specific flow of program processing

[0508] Processing from acquiring user data to utilizing digital clones

[0509] Step 1:

[0510] A user logs in to the system.

[0511] The user enters their authentication information and accesses the login screen.

[0512] The server verifies the authentication information and establishes a session for the user.

[0513] Step 2:

[0514] The user agrees to provide the data.

[0515] The user reads the terms of use and selects a checkbox to agree to providing data.

[0516] The terminal transmits the user's consent to the server.

[0517] Step 3:

[0518] The terminal collects user data.

[0519] The device downloads the chat data through the LINE app's API.

[0520] The terminal retrieves received emails using the mail server's API.

[0521] The device collects search queries using the browser's history API.

[0522] Step 4:

[0523] The server parses the user data and generates a vector representation.

[0524] The server sends the collected messages, emails, and search queries to a natural language processing engine.

[0525] A natural language processing engine tokenizes the data and analyzes its contextual information.

[0526] The server inputs the preprocessed data into an embedding model (e.g., BERT, GPT) to generate vectors.

[0527] Step 5:

[0528] The server stores the generated vector in a database.

[0529] The server stores the vector data generated for each user ID in a database.

[0530] A database associates the vector data with a user profile.

[0531] Step 6:

[0532] A user wants to create a digital clone.

[0533] The user selects the digital cloning option on the settings page and sends a request to the server.

[0534] Step 7:

[0535] The server uses the stored vector data to generate a digital clone.

[0536] The server inputs the user's vector data into an LLM (Large Scale Language Model).

[0537] LLM generates a digital clone that reflects the user's characteristics.

[0538] Step 8:

[0539] The server uses a digital clone to perform tasks.

[0540] The server passes the survey results to the digital clone.

[0541] The digital clone generates appropriate answers to the questions and sends them to the survey system.

[0542] Step 9:

[0543] A user registers for a dating service.

[0544] Users enter profile information and preferences and submit them to the server.

[0545] Step 10:

[0546] The server compares multiple digital clones and finds compatible pairs.

[0547] The server analyzes the digital clones of all registered users and compares their feature vectors.

[0548] The server pairs compatible clones using a matching algorithm.

[0549] Step 11:

[0550] The server analyzes the importance of new messages and emails.

[0551] The device collects new notifications and emails in real time.

[0552] The server inputs the collected data into a machine learning model to calculate an importance score.

[0553] Step 12:

[0554] The server reminds the user of important content.

[0555] The server selects important notifications and emails and generates reminder content.

[0556] Send a push notification to the device and display a reminder to the user.

[0557] The above is the processing flow related to the system of the present invention. In this way, user data can be effectively utilized and functions suited to the user can be provided.

[0558] Example 1

[0559] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0560] In today's information society, the amount of data generated by users every day is steadily increasing. However, it is not easy to efficiently analyze this vast amount of data and provide services and notifications that meet user needs. In particular, there is a lack of systems that can generate digital clones that perform tasks on behalf of users and that can provide important notifications in a timely manner. There is a need for systems that can solve these technical challenges and improve user convenience.

[0561] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0562] In this invention, the server includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, means for the digital clone to perform a predetermined task, means for analyzing the content of newly arrived messages and emails and identifying important content, and means for sending the identified important content to the terminal by push notification. This makes it possible to effectively utilize user data and provide services that meet the user's needs.

[0563] "User Data" means information generated or provided by a user, including messages, emails, search queries, and the like.

[0564] "Analysis" refers to the process of analyzing collected data and converting it into meaningful information.

[0565] "Vector representation" is a format in which data is quantified and represented in a multidimensional space. This format makes it possible to efficiently handle the characteristics and relationships of data.

[0566] A "digital clone" is a virtual agent that reflects the user's characteristics and can perform specific tasks on the user's behalf.

[0567] "Tasks" refer to specific tasks or activities that digital clones perform, such as answering surveys or matching romantic partners.

[0568] "New messages" refers to messages or notifications that the user has recently received.

[0569] "Email" means a digital message sent or received over the Internet.

[0570] A "push notification" is an instantaneous notification sent from a server to a terminal, and is a means of quickly conveying important information to a user.

[0571] The present invention relates to a system that efficiently utilizes user data to generate a digital clone to perform various tasks. The system includes a means for acquiring user data and converting it into a vector representation to generate a digital clone that captures the user's characteristics. The generated digital clone is then used for specific tasks such as answering questionnaires, conducting interviews, and matching relationships. The system also includes a means for providing notifications and email reminders to the user.

[0572] Specifically, to obtain user data, users must log in to the system and agree to provide LINE chat data, emails, search queries, etc. The device collects this data with the user's consent. The collected data is encrypted and sent to the server.

[0573] The server then analyzes the collected user data and generates vector representations. This analysis involves using the Google NLP API to tokenize the data and preprocess it to understand its context. The preprocessed data is then fed into an embedding model (e.g., Word2Vec or BERT) to generate vector representations. The server then stores the generated vectors in a database (e.g., MySQL).

[0574] If a user wishes to create a digital clone, the server uses the stored vector data to generate a digital clone that reflects the user's characteristics and can perform specific tasks (e.g., answering surveys or matching relationships) on the user's behalf.

[0575] For example, in the case of a survey, the server inputs the survey contents into the digital clone, generates and sends appropriate answers, and in the case of love matching, the server compares multiple digital clones, finds compatible pairs, and notifies the user of the results.

[0576] The server also identifies content that may be of interest to the user from among newly received messages and emails, analyzes its importance, and sends push notifications to the device to remind the user of the identified important content.

[0577] Thus, the present invention provides a versatile system for effectively utilizing user data and improving user experience.

[0578] Examples of prompt statements

[0579] 1. Survey prompt:

[0580] "Please share your opinion on the following survey question: 'How do you like to spend your holidays?'"

[0581] 2. Romance Matching Prompt:

[0582] "What is your favorite movie genre and why?"

[0583] 3. Prompt for notification reminder:

[0584] "Check for the new message below: 'Tomorrow's meeting time has changed.'"

[0585] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0586] Step 1:

[0587] The user logs in to the system and agrees to provide data. User ID and password are required as input. The output is a successful login status and confirmation of data provision agreement. Specifically, the user enters authentication information into the login form and checks the checkbox for data provision.

[0588] Step 2:

[0589] With the user's consent, the device collects user data such as LINE chat data, emails, and search queries. Raw data provided by various apps and browsers is required as input. The collected data is encrypted and sent to the server as output. Specifically, the device collects data in the background, encrypts it with RSA, and uploads it to the server.

[0590] Step 3:

[0591] The server receives and decrypts the collected encrypted data. It requires encrypted user data as input and decrypts the decrypted text data as output. Specifically, the server uses an RSA decryption key to decrypt the received data and convert it into an analyzable format.

[0592] Step 4:

[0593] The server tokenizes and preprocesses the data using the Google NLP API. The input requires decrypted user data. The output is tokenized data. Specifically, the server calls the NLP API to split sentences and generate tokens.

[0594] Step 5:

[0595] The server inputs the preprocessed data into an embedding model (such as Word2Vec or BERT) to generate a vector representation. Tokenized data is required as input, and a vector representation is obtained as output. Specifically, the server calls the embedding model library to vectorize the data.

[0596] Step 6:

[0597] The server saves the generated vector to the database. It requires a vector representation as input and a confirmation of saving to the database as output. Specifically, the server connects to MySQL and uses the INSERT statement to insert the vector data into a table.

[0598] Step 7:

[0599] A user wishes to create a digital clone and submits a request. The input required is a clone creation request from the user. The output is a confirmation of receipt of the clone creation request. The specific operation is when the user clicks the clone creation button on a dedicated dashboard.

[0600] Step 8:

[0601] The server receives the request and generates a digital clone using the stored vector data. The input is vector data. The output is a digital clone. Specifically, the server reads the vector data and runs the clone generation algorithm.

[0602] Step 9:

[0603] The digital clone performs tasks such as answering surveys and matching. Survey questions and matching conditions are required as input. The output is the results of the task execution and matching. Specifically, the server inputs the survey questions into the clone, generates answers, and sends them.

[0604] Step 10:

[0605] The server identifies new messages or emails that are likely to be of interest to users and analyzes their importance. The input required is the new messages or emails. The output is the identified important content. Specifically, the server uses a text analysis algorithm to evaluate the importance of the messages.

[0606] Step 11:

[0607] The server sends a push notification to the device to remind the user of the identified important information. The input required is the identified important information. The output is a confirmation of the push notification. Specifically, the server sends the notification to the device using the Firebase Cloud Messaging (FCM) API.

[0608] (Application example 1)

[0609] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0610] Conventional virtual store shopping experiences often fail to fully address individual user preferences, potentially resulting in low user satisfaction. Furthermore, users' questions are not answered in real time, limiting the user experience. Therefore, there is a need for a system that can recommend products based on user preferences and quickly respond to user questions.

[0611] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0612] In this invention, the server includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, means for the digital clone to perform a predetermined task, means for recommending products based on the user's preferences, and means for responding to user questions, thereby enabling product recommendations based on the user's individual preferences, improving the shopping experience in the virtual store, and enabling quick responses to user questions.

[0613] "User Data" refers to any information about a user, including messages, emails, search queries, and other data.

[0614] "Vector representation" refers to the technique of converting text and other data into numerical vectors, a format that can be processed by a computer.

[0615] A "digital clone" is a virtual model that captures a user's characteristics and performs various tasks on their behalf.

[0616] "Prescribed tasks" refer to specific tasks that the digital clone will perform, such as answering surveys or matching romantic partners.

[0617] "Product recommendation tools" refers to algorithms and systems that suggest appropriate products based on a user's preferences and past behavior.

[0618] "Means for responding to questions" refers to a function for providing appropriate answers to inquiries and questions from users.

[0619] This invention is a system that utilizes user data to generate a digital clone that captures the user's characteristics and automatically executes various tasks. A specific embodiment of this system will be described below.

[0620] System Overview

[0621] The system mainly includes the following elements:

[0622] 1. How to obtain user data

[0623] 2. Data analysis and vector representation generation

[0624] 3. How to create a digital clone

[0625] 4. A means of accomplishing a specific task

[0626] 5. A way to recommend products based on user preferences

[0627] 6. Means of responding to user questions

[0628] Hardware and software used

[0629] Hardware: Smartphones, servers

[0630] Software: Python, scikit-learn (natural language processing engine), TfidfVectorizer (text vectorization), cosine_similarity (similarity calculation)

[0631] Program processing overview

[0632] 1. How to obtain user data

[0633] Users log in through a smartphone application and agree to provide data, which is then collected, including messages, emails, and search queries.

[0634] 2. Data analysis and vector representation generation

[0635] The server analyzes the collected user data, tokenizing it using TfidfVectorizer to generate vector representations that represent the user's characteristics in numerical form.

[0636] 3. How to create a digital clone

[0637] A digital clone is generated based on the analyzed vector data, and the digital clone can perform various tasks while reflecting the user's characteristics.

[0638] 4. A means of accomplishing a specific task

[0639] Digital clones can automatically perform tasks such as answering surveys and matching people for love.

[0640] 5. A way to recommend products based on user preferences

[0641] The digital clone selects products from a virtual store based on the user's purchase history and preference data, and recommends the most suitable products to the user.

[0642] 6. Means of responding to user questions

[0643] When users ask questions through chat, the digital clone answers in real time.

[0644] Specific examples

[0645] 1. Example of user data acquisition

[0646] A user logs into the application and provides data such as "I like casual clothes" and "I like the color blue."

[0647] 2. Example of vector representation generation

[0648] The server parses the provided data, tokenizes the text with TfidfVectorizer, and generates a vector representation.

[0649] 3. Specific examples of product endorsements

[0650] Based on vector data, the system recommends the most suitable item from a virtual store, such as an elegant black dress, casual blue jeans, or a minimalist white T-shirt.

[0651] 4. Example of responding to user questions

[0652] When the user asks, "What is my favorite color?" the digital clone responds, "Your favorite color is blue."

[0653] Prompt Sentence Examples

[0654] "Based on your data, our virtual shopping assistant will recommend the best products for you. Please let us know if you have a preference for casual clothing or specific colors."

[0655] The above is a description of a specific embodiment for carrying out the present invention. This system enables a user to have a highly personalized shopping experience, thereby significantly improving user satisfaction.

[0656] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0657] Step 1:

[0658] Retrieving User Data

[0659] When a user logs in to a smartphone application and agrees to provide data, the device collects the user's message data, emails, search queries, etc. The input is the data provided by the user, and the output is the collected raw user data.

[0660] Step 2:

[0661] Data analysis and vector representation generation

[0662] The server receives the collected user data and performs analysis. Specifically, it uses Python and scikit-learn's TfidfVectorizer to tokenize the data and generate a vector representation. The input is raw user data, and the output is user data represented as a numeric vector.

[0663] Step 3:

[0664] Saving vector data

[0665] The generated vector representation is saved in a database on the server, making it available for subsequent tasks. The input is a numeric vector, and the output is vector data stored in the database.

[0666] Step 4:

[0667] Creating a digital clone

[0668] When a user requests the creation of a digital clone, the server uses the stored vector data to create the digital clone. The input is the vector data stored in the database, and the output is a digital clone that mimics the user's characteristics.

[0669] Step 5:

[0670] Executing a task

[0671] The digital clone performs a given task on behalf of the user, for example, making product recommendations in a virtual store: the input is data about the user's purchasing history and preferences, and the output is a list of products recommended to the user.

[0672] Step 6:

[0673] Question Processing

[0674] When a user inputs a question through a smartphone application, the server instantly generates an answer using the digital clone. The input is the user's question, and the output is the generated answer. For example, if the user asks, "What is my favorite color?", the digital clone will respond, "Your favorite color is blue."

[0675] Step 7:

[0676] Best product recommendations

[0677] The server compares the user's vector data with the product data of the virtual store and recommends the best products using cosine_similarity. The input is the user vector data and the product data vector, and the output is a list of products that are judged to be best for the user.

[0678] The above are the specific processing steps of the system for implementing the present invention, which provides users with an optimal shopping experience and question responses.

[0679] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0680] The present invention relates to a system that efficiently utilizes user data to generate a digital clone to perform various tasks. Furthermore, by combining it with an emotion engine that analyzes user data and recognizes the user's emotions, it is possible to provide more advanced services. This system includes a means for acquiring user data, analyzing it, and converting it into a vector representation to generate a digital clone that captures the user's characteristics and emotions. The generated digital clone is then used for specific tasks such as answering surveys, conducting interviews, and matching relationships. The system also includes a means for providing notifications and email reminders to the user.

[0681] System program processing explanation

[0682] Retrieving User Data

[0683] The user logs in and agrees to provide LINE chat data, emails, search queries, etc. The device collects this data with the user's consent.

[0684] Analyzing data and generating vector representations

[0685] The server analyzes the collected user data and recognizes the user's emotions using an emotion engine. This emotion information and user data are analyzed comprehensively to generate a vector representation. A natural language processing engine is used for the analysis, tokenizing the data and performing preprocessing to understand the context and emotion. The preprocessed data is then input into an embedding model to generate a vector representation. The generated vector is then stored in a database by the server.

[0686] Creating a digital clone and performing tasks

[0687] If a user wishes to create a digital clone, the server will use the stored vector data to generate a digital clone that reflects the user's characteristics and emotions and performs specific tasks on the user's behalf.

[0688] For example, in the case of a survey, the server inputs the survey contents into the digital clone, generates and sends appropriate answers, and in the case of love matching, the server compares multiple digital clones, finds compatible pairs, and notifies the user of the results.

[0689] Notifications and reminders

[0690] The server identifies new messages and emails that the user is likely to be interested in, analyzes their importance, and then sends push notifications to the device to remind the user of the important content.

[0691] Specific examples

[0692] 1. The user logs in to the system and agrees to provide data. The device collects LINE chat data, emails, and search queries.

[0693] 2. The server analyzes the collected data, uses an emotion engine to recognize the user's emotions, and reflects them in a vector representation.

[0694] 3. The server stores the generated vector in a database.

[0695] 4. The user requests the creation of a digital clone, and the server uses the generated vector data to create the digital clone.

[0696] 5. To answer the survey, the server passes the survey contents to the digital clone, which generates and transmits the appropriate answers.

[0697] 6. For love matching, the server compares multiple clones and suggests compatible pairs.

[0698] 7. The server analyzes new messages and emails and sends important information via push notification to the device to remind the user.

[0699] In this way, the present invention can provide a multi-functional system for improving the user experience by effectively utilizing user data and further recognizing the user's emotions.

[0700] The processing flow will be explained below.

[0701] Specific flow of program processing

[0702] Processing from acquiring user data to utilizing digital clones

[0703] Step 1:

[0704] A user logs in to the system.

[0705] The user enters their authentication information and accesses the login screen.

[0706] The server verifies the authentication information and establishes a session for the user.

[0707] Step 2:

[0708] The user agrees to provide the data.

[0709] The user reads the terms of use and selects a checkbox to agree to providing data.

[0710] The terminal transmits the user's consent to the server.

[0711] Step 3:

[0712] The terminal collects user data.

[0713] The device downloads the chat data through the LINE app's API.

[0714] The terminal retrieves received emails using the mail server's API.

[0715] The device collects search queries using the browser's history API.

[0716] Step 4:

[0717] The server parses the user data and generates a vector representation.

[0718] The server sends the collected messages, emails, and search queries to a natural language processing engine.

[0719] A natural language processing engine tokenizes the data and analyzes its contextual information.

[0720] The server inputs the preprocessed data into an embedding model (e.g., BERT, GPT) to generate vectors.

[0721] Step 5:

[0722] The server uses an emotion engine to recognize the user's emotion.

[0723] The server inputs the preprocessed data into an emotion engine to extract emotion information (e.g., joy, sadness, anger).

[0724] The server integrates the emotion information obtained from the emotion engine into a vector representation.

[0725] Step 6:

[0726] The server stores the generated vector in a database.

[0727] The server stores the vector data generated for each user ID in a database.

[0728] A database associates the vector data with a user profile.

[0729] Step 7:

[0730] A user wants to create a digital clone.

[0731] The user selects the digital cloning option on the settings page and sends a request to the server.

[0732] Step 8:

[0733] The server uses the stored vector data to generate a digital clone.

[0734] The server inputs the user's vector data into an LLM (Large Scale Language Model).

[0735] LLM creates a digital clone that reflects the user's characteristics and emotions.

[0736] Step 9:

[0737] The server uses a digital clone to perform tasks.

[0738] The server passes the survey results to the digital clone.

[0739] The digital clone generates appropriate answers to the questions and sends them to the survey system.

[0740] Step 10:

[0741] A user registers for a dating service.

[0742] Users enter profile information and preferences and submit them to the server.

[0743] Step 11:

[0744] The server compares multiple digital clones and finds compatible pairs.

[0745] The server analyzes the digital clones of all registered users and compares their feature vectors.

[0746] The server pairs compatible clones using a matching algorithm.

[0747] Step 12:

[0748] The server analyzes the importance of new messages and emails.

[0749] The device collects new notifications and emails in real time.

[0750] The server inputs the collected data into a machine learning model to calculate an importance score.

[0751] Step 13:

[0752] The server reminds the user of important content.

[0753] The server selects important notifications and emails and generates reminder content.

[0754] Send a push notification to the device and display a reminder to the user.

[0755] The above is the flow of specific processing steps related to the system of the present invention. In this way, by effectively utilizing user data and incorporating emotion information, it is possible to provide advanced services to users.

[0756] Example 2

[0757] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0758] Conventional systems that use user data have had difficulty in accurately recognizing user emotions and reflecting them in services. Furthermore, systems that notify and remind users have been unable to effectively deliver information of interest to users. Furthermore, there has been a lack of technology that can faithfully reflect a user's characteristics and emotions when performing tasks using a digital clone.

[0759] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0760] In this invention, the server includes a means for acquiring user data, a means for analyzing the acquired data and generating a vector representation, and a means for creating a digital clone of the user using the generated vector representation. This makes it possible to recognize the user's emotions and reflect the analyzed emotional information in the vector representation. Furthermore, by including a means for providing notifications and reminders to the user, it is possible to accurately deliver information that interests the user.

[0761] "User Data" refers to any information related to a user's activities, including messages, emails, search queries, etc.

[0762] "Vector representation" is a representation of extracted data features as a numerical vector, and is used in data analysis and machine learning.

[0763] A "digital clone" is a virtual copy of a user that reflects their characteristics and emotions and can perform tasks on their behalf.

[0764] An "emotion engine" refers to a system or algorithm that recognizes and analyzes emotions from user data.

[0765] "Notification" means information sent by the system to the user to inform them of new messages or important information.

[0766] A "remind" is a notification or announcement that reminds users of important information or tasks that they tend to forget.

[0767] The present invention relates to a system that efficiently utilizes user data, generates digital clones, and executes various tasks. A specific embodiment of this system will be described below.

[0768] First, the user logs in to the system and agrees to provide data such as LINE chat data, emails, and search queries. After obtaining the user's consent, the device collects this data and sends it to the server. This data collection and transmission is encrypted using the HTTPS protocol.

[0769] The server then receives the collected user data. This data is temporarily stored in a database and prepared for analysis. The server is equipped with a natural language processing engine, which uses models such as BERT and GPT-3 to tokenize and preprocess the data. After tokenization, the data is split into sentences and words and unwanted noise is removed.

[0770] The server then uses an emotion engine to analyze the user's emotions. For emotion recognition, it uses an LSTM model or similar. The server then inputs the preprocessed data into an embedding model (e.g., Word2Vec or FastText) to generate a vector representation for each piece of data. The server then stores these vectors in a database and associates them with the user ID.

[0771] When a user requests the creation of a digital clone, the server retrieves vector data associated with the user ID from the database. The server then inputs the vector data into a generative AI model (e.g., GPT-3) to generate a digital clone that reflects the user's characteristics and emotions. This digital clone can then perform specific tasks on the user's behalf.

[0772] For example, when answering a questionnaire, the server provides the digital clone with the questionnaire content, which generates and transmits the appropriate answer. In the case of love matching, the server compares multiple digital clones to find the most compatible pair.

[0773] Finally, the server periodically checks for new messages and emails, analyzes their content using Natural Language Understanding (NLU), identifies content that may be of interest to the user, scores its importance, and sends push notifications about the identified important content to the device. The device receives the push notifications and reminds the user.

[0774] As a concrete example, the following prompt sentence is input to the generative AI model:

[0775] "User: Please create a digital clone of yourself that will answer the following survey:

[0776] 1. What is your favorite food?

[0777] 2. What are your hobbies?

[0778] 3. Tell us about your most recent travel destination.

[0779] Following these prompts, the generative AI model generates appropriate answers that reflect the user's characteristics and emotions.

[0780] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0781] Step 1: Get User Data

[0782] 1. The user logs into the system and enters their credentials.

[0783] Input: User authentication information (user ID, password).

[0784] Output: Authentication success message, user ID.

[0785] 2. The user agrees to provide LINE chat data, emails, and search queries.

[0786] Input: User consent.

[0787] Output: Consent confirmation message.

[0788] 3. After obtaining the user's consent, the device collects LINE chat data, emails, and search history and sends them to the server. The transmission uses the HTTPS protocol and the data is encrypted.

[0789] Input: User's LINE chat data, emails, and search history.

[0790] Output: Encrypted user data.

[0791] Step 2: Analyze the data and generate a vector representation

[0792] 1. The server receives the collected user data and temporarily stores it in a database.

[0793] Input: Encrypted user data.

[0794] Output: Raw data stored in a database.

[0795] 2. The server uses a natural language processing engine to tokenize and preprocess the data, specifically splitting the text data into sentences and words and filtering out unnecessary noise.

[0796] Input: Stored raw data.

[0797] Output: Tokenized data.

[0798] 3. The emotion engine analyzes the tokenized data and recognizes the user's emotions. It uses an LSTM model or similar.

[0799] Input: Tokenized data.

[0800] Output: Emotional information.

[0801] 4. The natural language processing engine inputs the preprocessed data into an embedding model (e.g., Word2Vec or FastText) to generate a vector representation.

[0802] Input: Tokenized data and sentiment information.

[0803] Output: Vector representation.

[0804] 5. The generated vector is stored in the database and associated with the user ID.

[0805] Input: Vector representation and user ID.

[0806] Output: Vector data stored in a database.

[0807] Step 3: Generate a digital clone and perform the task

[0808] 1. If a user wishes to create a digital clone, they submit a request.

[0809] Input: Digital cloning request.

[0810] Output: Request received message.

[0811] 2. The server retrieves vector data associated with the user ID from the database.

[0812] Input: User ID.

[0813] Output: Vector data.

[0814] 3. The server inputs the vector data into a generative AI model (e.g., GPT-3) to generate a digital clone.

[0815] Input: Vector data.

[0816] Output: The generated digital clone.

[0817] 4. The digital clone performs specific tasks.

[0818] In the case of a survey response: The server provides the survey content to the digital clone, which generates and transmits the response.

[0819] Input: Survey contents.

[0820] Output: The generated answer.

[0821] In the case of love matching: The server compares multiple digital clones and finds compatible pairs.

[0822] Enter: multiple digital clones.

[0823] Output: Results of compatible pairs.

[0824] Step 4: Notifications and reminders

[0825] 1. The server periodically checks for new messages and emails.

[0826] Input: New Message, Email.

[0827] Output: A list of messages and emails.

[0828] 2. The server uses NLU to analyze the content of the message or email to identify content that the user is likely to be interested in. An importance score is calculated.

[0829] Input: message, email.

[0830] Output: Importance score.

[0831] 3. The server selects content with a high importance score and sends a notification to the device using the push notification API.

[0832] Input: Content with high importance score.

[0833] Output: Push notification.

[0834] 4. The device receives a push notification and reminds the user.

[0835] Enter: push notification.

[0836] Output: Remind notification.

[0837] At each step, specific actions are performed, and the entire system functions by repeating the input, processing, and output of data.

[0838] (Application example 2)

[0839] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0840] While conventional digital clone generation systems have a certain degree of accuracy in analyzing user data and performing tasks, they have limitations in providing more personalized services by incorporating the user's emotional state and real-time visual information. Furthermore, they have the problem of being unable to suggest products based on the user's emotions and interests when it comes to purchasing behavior in physical stores.

[0841] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, means for the digital clone to perform a predetermined task, means for recognizing the user's emotions, means for acquiring visual data from a visual device worn by the user, and means for suggesting products based on the visual data and the user's emotions. This makes it possible to provide highly personalized services based on the user's emotional state and real-time visual information.

[0842] "User Data" is information provided by a user, including messages, emails, search queries, and purchase history data.

[0843] A "vector representation" is a representation of acquired user data quantified and expressed as a multidimensional vector.

[0844] A "digital clone" is a virtual being that reflects the user's characteristics and emotions and performs various tasks on the user's behalf.

[0845] An "emotion recognition means" is a device or software that uses algorithms or models to analyze a user's emotional state.

[0846] A "visual device" is a device worn by a user to capture visual data, such as smart glasses.

[0847] "Visual data" refers to image or video information obtained from a visual device.

[0848] A "means for suggesting products" is a device or software that uses algorithms or models to suggest optimal products to a user based on the user's emotional state and visual data.

[0849] The present invention provides a system that efficiently utilizes user data and suggests products in physical stores based on the user's emotional state and visual information. This system acquires and analyzes user data to generate a digital clone, and further recognizes the user's emotions using an emotion engine, thereby realizing more advanced services.

[0850] Overall system configuration

[0851] The system consists of the following main components:

[0852] 1. User data acquisition means: A means for collecting messages, emails, search queries, and purchase history data provided by users. This means also acquires visual data through smart glasses.

[0853] 2. Data analysis means: Analyzes collected user data and recognizes user emotions using an emotion engine. Specifically, it uses an NLP model (natural language processing engine) and an emotion recognition engine.

[0854] 3. Vector representation generation: This is a method for generating vector representations that reflect the user's characteristics and emotions based on the analyzed data. An embedding model is used.

[0855] 4. Digital clone generation means: A means for creating a digital clone of the user using the generated vector representation. The digital clone performs tasks on behalf of the user.

[0856] 5. Task execution means: This is the means by which the digital clone performs tasks such as making purchasing suggestions on behalf of the user.

[0857] 6. Notifications and reminders: This is a method of sending product suggestions and new message reminders to users' devices via push notifications.

[0858] Specific examples of programs

[0859] 1. User data acquisition

[0860] With the user's consent, the server collects LINE chat data, emails, search queries, and purchase history data. It also collects visual data of users wearing smart glasses in real time.

[0861] 2. Data analysis and vector representation generation

[0862] The server analyzes the collected user data and converts the user's emotions and characteristics into vector representations using an emotion engine and NLP model.

[0863] 3. Creating a digital clone and performing tasks

[0864] When a user requests the creation of a digital clone, the server uses the stored vector data to create a digital clone that reflects the user's characteristics and emotions. The digital clone can then perform tasks such as product recommendations.

[0865] 4. Notifications and Reminders

[0866] The server sends important information and suggestions to the user's device, such as smart glasses, via push notifications to remind the user.

[0867] Hardware and software used

[0868] Hardware: Smart glasses, device (smartphone)

[0869] Software: Emotion recognition engine (e.g., DistilBERT), NLP model, digital clone generation engine, embedded model, external API (user data acquisition API, product recommendation API)

[0870] Specific examples

[0871] For example, imagine a user is in the cosmetics section. The smart glasses detect the specific cosmetics the user is interested in and, based on the user's past purchase data and real-time emotional data, suggest other users' reviews of that cosmetics, similar products, and special offers. This information is displayed via push notification on the smart glasses.

[0872] Prompt Sentence Examples

[0873] User ID: user_123

[0874] Store ID: store_789

[0875] Visual data: path / to / sample_image.jpg

[0876] [the purpose]

[0877] Appropriate sentiment analysis is performed on the products and locations that the user is interested in, and the most suitable products are suggested for the user.

[0878] [Provided data]

[0879] 1. User purchase history data

[0880] 2. Real-time user visual data

[0881] 3. In-store product data

[0882] [Expected output]

[0883] 1. The user's emotional state

[0884] 2. List of recommended products

[0885] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0886] Step 1:

[0887] The user logs into the system and agrees to provide data.

[0888] Input: User-provided messages, emails, search queries, and purchase history data

[0889] Output: Collected user data

[0890] Specific operation: When the user clicks the consent button, the device will collect LINE chat data, email content, past search queries, and purchase history data and send them to the server.

[0891] Step 2:

[0892] The server analyzes the collected user data and recognizes the user's emotions using an emotion engine (e.g., DistilBERT).

[0893] Input: Collected user data

[0894] Output: Analysis results including emotional information

[0895] What it does: The server tokenizes the collected data and uses an emotion recognition algorithm to determine the emotional state of each message or email, then stores the results in a database.

[0896] Step 3:

[0897] The server uses the parsed data to generate a vector representation.

[0898] Input: Analysis results including emotion information

[0899] Output: A vector representation reflecting the user's features and emotions

[0900] How it works: The server uses the NLP engine and embedding model to convert user data into multidimensional vectors as semantic vectors, which are then stored in a database.

[0901] Step 4:

[0902] When a user requests the creation of a digital clone, the server uses the stored vector data to generate a digital clone that reflects the user's characteristics and emotions.

[0903] Input: Vector representation

[0904] Output: Digital Clone

[0905] How it works: The server uses the stored vector data to generate a digital clone that mimics the user's behavioral patterns and emotions.

[0906] Step 5:

[0907] When a user puts on the smart glasses and enters a physical store, the device captures visual data in real time.

[0908] Input: Visual data

[0909] Output: Captured image and video data

[0910] How it works: The camera in the smart glasses captures visual data of the surroundings and transmits it to a server in real time.

[0911] Step 6:

[0912] The server uses real-time visual data and vector-expressed user data to suggest optimal products to users.

[0913] Input: visual data, vector representation

[0914] Output: A list of product suggestions

[0915] How it works: The server analyzes visual data to identify products the user is interested in. It then analyzes the user's emotional state and purchasing history to generate suggestions for similar products and review information.

[0916] Step 7:

[0917] The server sends the suggested product information to the smart glasses via push notification to notify the user.

[0918] Input: Product suggestion list

[0919] Output: Push notification product information

[0920] Specific operation: The server sends the generated product suggestions to the smart glasses as a push notification and displays them in the user's field of view, allowing the user to check product information in real time.

[0921] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0922] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0923] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0924] [Third embodiment]

[0925] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0926] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0927] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0928] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0929] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0930] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0931] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0932] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0933] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0934] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0935] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0936] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0937] The present invention relates to a system that efficiently utilizes user data to generate a digital clone to perform various tasks. The system includes a means for acquiring user data and converting it into a vector representation to generate a digital clone that captures the user's characteristics. The generated digital clone is then used for specific tasks such as answering questionnaires, conducting interviews, and matching relationships. The system also includes a means for providing notifications and email reminders to the user.

[0938] System program processing explanation

[0939] Retrieving User Data

[0940] The user logs in and agrees to provide LINE chat data, emails, search queries, etc. The device collects this data with the user's consent.

[0941] Analyzing data and generating vector representations

[0942] The server analyzes the collected user data and generates vector representations. This analysis uses a natural language processing engine to tokenize the data and preprocess it to understand its context. The preprocessed data is then fed into an embedding model to generate vector representations. The server stores the generated vectors in a database.

[0943] Creating a digital clone and performing tasks

[0944] If a user wishes to create a digital clone, the server uses the stored vector data to generate a digital clone that reflects the user's characteristics and performs specific tasks on the user's behalf.

[0945] For example, in the case of a survey, the server inputs the survey contents into the digital clone, generates and sends appropriate answers, and in the case of love matching, the server compares multiple digital clones, finds compatible pairs, and notifies the user of the results.

[0946] Notifications and reminders

[0947] The server identifies new messages and emails that the user is likely to be interested in, analyzes their importance, and then sends push notifications to the device to remind the user of the important content.

[0948] Specific examples

[0949] 1. The user logs in to the system and agrees to provide data. The device collects LINE chat data, emails, and search queries.

[0950] 2. The server analyzes the collected data, generates a vector representation, and stores it in a database.

[0951] 3. The user requests the creation of a digital clone, and the server uses the generated vector data to create the digital clone.

[0952] 4. To answer the survey, the server inputs the survey details into the digital clone, generates the appropriate answers, and sends them.

[0953] 5. For love matching, the server compares multiple clones and suggests compatible pairs.

[0954] 6. The server analyzes new messages and emails and sends important information via push notification to the device to remind the user.

[0955] Thus, the present invention provides a versatile system for effectively utilizing user data and enhancing the user experience.

[0956] The processing flow will be explained below.

[0957] Specific flow of program processing

[0958] Processing from acquiring user data to utilizing digital clones

[0959] Step 1:

[0960] A user logs in to the system.

[0961] The user enters their authentication information and accesses the login screen.

[0962] The server verifies the authentication information and establishes a session for the user.

[0963] Step 2:

[0964] The user agrees to provide the data.

[0965] The user reads the terms of use and selects a checkbox to agree to providing data.

[0966] The terminal transmits the user's consent to the server.

[0967] Step 3:

[0968] The terminal collects user data.

[0969] The device downloads the chat data through the LINE app's API.

[0970] The terminal retrieves received emails using the mail server's API.

[0971] The device collects search queries using the browser's history API.

[0972] Step 4:

[0973] The server parses the user data and generates a vector representation.

[0974] The server sends the collected messages, emails, and search queries to a natural language processing engine.

[0975] A natural language processing engine tokenizes the data and analyzes its contextual information.

[0976] The server inputs the preprocessed data into an embedding model (e.g., BERT, GPT) to generate vectors.

[0977] Step 5:

[0978] The server stores the generated vector in a database.

[0979] The server stores the vector data generated for each user ID in a database.

[0980] A database associates the vector data with a user profile.

[0981] Step 6:

[0982] A user wants to create a digital clone.

[0983] The user selects the digital cloning option on the settings page and sends a request to the server.

[0984] Step 7:

[0985] The server uses the stored vector data to generate a digital clone.

[0986] The server inputs the user's vector data into an LLM (Large Scale Language Model).

[0987] LLM generates a digital clone that reflects the user's characteristics.

[0988] Step 8:

[0989] The server uses a digital clone to perform tasks.

[0990] The server passes the survey results to the digital clone.

[0991] The digital clone generates appropriate answers to the questions and sends them to the survey system.

[0992] Step 9:

[0993] A user registers for a dating service.

[0994] Users enter profile information and preferences and submit them to the server.

[0995] Step 10:

[0996] The server compares multiple digital clones and finds compatible pairs.

[0997] The server analyzes the digital clones of all registered users and compares their feature vectors.

[0998] The server pairs compatible clones using a matching algorithm.

[0999] Step 11:

[1000] The server analyzes the importance of new messages and emails.

[1001] The device collects new notifications and emails in real time.

[1002] The server inputs the collected data into a machine learning model to calculate an importance score.

[1003] Step 12:

[1004] The server reminds the user of important content.

[1005] The server selects important notifications and emails and generates reminder content.

[1006] Send a push notification to the device and display a reminder to the user.

[1007] The above is the processing flow related to the system of the present invention. In this way, user data can be effectively utilized and functions suited to the user can be provided.

[1008] Example 1

[1009] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1010] In today's information society, the amount of data generated by users every day is steadily increasing. However, it is not easy to efficiently analyze this vast amount of data and provide services and notifications that meet user needs. In particular, there is a lack of systems that can generate digital clones that perform tasks on behalf of users and that can provide important notifications in a timely manner. There is a need for systems that can solve these technical challenges and improve user convenience.

[1011] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1012] In this invention, the server includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, means for the digital clone to perform a predetermined task, means for analyzing the content of newly arrived messages and emails and identifying important content, and means for sending the identified important content to the terminal by push notification. This makes it possible to effectively utilize user data and provide services that meet the user's needs.

[1013] "User Data" means information generated or provided by a user, including messages, emails, search queries, and the like.

[1014] "Analysis" refers to the process of analyzing collected data and converting it into meaningful information.

[1015] "Vector representation" is a format in which data is quantified and represented in a multidimensional space. This format makes it possible to efficiently handle the characteristics and relationships of data.

[1016] A "digital clone" is a virtual agent that reflects the user's characteristics and can perform specific tasks on the user's behalf.

[1017] "Tasks" refer to specific tasks or activities that digital clones perform, such as answering surveys or matching romantic partners.

[1018] "New messages" refers to messages or notifications that the user has recently received.

[1019] "Email" means a digital message sent or received over the Internet.

[1020] A "push notification" is an instantaneous notification sent from a server to a terminal, and is a means of quickly conveying important information to a user.

[1021] The present invention relates to a system that efficiently utilizes user data to generate a digital clone to perform various tasks. The system includes a means for acquiring user data and converting it into a vector representation to generate a digital clone that captures the user's characteristics. The generated digital clone is then used for specific tasks such as answering questionnaires, conducting interviews, and matching relationships. The system also includes a means for providing notifications and email reminders to the user.

[1022] Specifically, to obtain user data, users must log in to the system and agree to provide LINE chat data, emails, search queries, etc. The device collects this data with the user's consent. The collected data is encrypted and sent to the server.

[1023] The server then analyzes the collected user data and generates vector representations. This analysis involves using the Google NLP API to tokenize the data and preprocess it to understand its context. The preprocessed data is then fed into an embedding model (e.g., Word2Vec or BERT) to generate vector representations. The server then stores the generated vectors in a database (e.g., MySQL).

[1024] If a user wishes to create a digital clone, the server uses the stored vector data to generate a digital clone that reflects the user's characteristics and can perform specific tasks (e.g., answering surveys or matching relationships) on the user's behalf.

[1025] For example, in the case of a survey, the server inputs the survey contents into the digital clone, generates and sends appropriate answers, and in the case of love matching, the server compares multiple digital clones, finds compatible pairs, and notifies the user of the results.

[1026] The server also identifies content that may be of interest to the user from among newly received messages and emails, analyzes its importance, and sends push notifications to the device to remind the user of the identified important content.

[1027] Thus, the present invention provides a versatile system for effectively utilizing user data and improving user experience.

[1028] Examples of prompt statements

[1029] 1. Survey prompt:

[1030] "Please share your opinion on the following survey question: 'How do you like to spend your holidays?'"

[1031] 2. Romance Matching Prompt:

[1032] "What is your favorite movie genre and why?"

[1033] 3. Prompt for notification reminder:

[1034] "Check for the new message below: 'Tomorrow's meeting time has changed.'"

[1035] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1036] Step 1:

[1037] The user logs in to the system and agrees to provide data. User ID and password are required as input. The output is a successful login status and confirmation of data provision agreement. Specifically, the user enters authentication information into the login form and checks the checkbox for data provision.

[1038] Step 2:

[1039] With the user's consent, the device collects user data such as LINE chat data, emails, and search queries. Raw data provided by various apps and browsers is required as input. The collected data is encrypted and sent to the server as output. Specifically, the device collects data in the background, encrypts it with RSA, and uploads it to the server.

[1040] Step 3:

[1041] The server receives and decrypts the collected encrypted data. It requires encrypted user data as input and decrypts the decrypted text data as output. Specifically, the server uses an RSA decryption key to decrypt the received data and convert it into an analyzable format.

[1042] Step 4:

[1043] The server tokenizes and preprocesses the data using the Google NLP API. The input requires decrypted user data. The output is tokenized data. Specifically, the server calls the NLP API to split sentences and generate tokens.

[1044] Step 5:

[1045] The server inputs the preprocessed data into an embedding model (such as Word2Vec or BERT) to generate a vector representation. Tokenized data is required as input, and a vector representation is obtained as output. Specifically, the server calls the embedding model library to vectorize the data.

[1046] Step 6:

[1047] The server saves the generated vector to the database. It requires a vector representation as input and a confirmation of saving to the database as output. Specifically, the server connects to MySQL and uses the INSERT statement to insert the vector data into a table.

[1048] Step 7:

[1049] A user wishes to create a digital clone and submits a request. The input required is a clone creation request from the user. The output is a confirmation of receipt of the clone creation request. The specific operation is when the user clicks the clone creation button on a dedicated dashboard.

[1050] Step 8:

[1051] The server receives the request and generates a digital clone using the stored vector data. The input is vector data. The output is a digital clone. Specifically, the server reads the vector data and runs the clone generation algorithm.

[1052] Step 9:

[1053] The digital clone performs tasks such as answering surveys and matching. Survey questions and matching conditions are required as input. The output is the results of the task execution and matching. Specifically, the server inputs the survey questions into the clone, generates answers, and sends them.

[1054] Step 10:

[1055] The server identifies new messages or emails that are likely to be of interest to users and analyzes their importance. The input required is the new messages or emails. The output is the identified important content. Specifically, the server uses a text analysis algorithm to evaluate the importance of the messages.

[1056] Step 11:

[1057] The server sends a push notification to the device to remind the user of the identified important information. The input required is the identified important information. The output is a confirmation of the push notification. Specifically, the server sends the notification to the device using the Firebase Cloud Messaging (FCM) API.

[1058] (Application example 1)

[1059] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1060] Conventional virtual store shopping experiences often fail to fully address individual user preferences, potentially resulting in low user satisfaction. Furthermore, users' questions are not answered in real time, limiting the user experience. Therefore, there is a need for a system that can recommend products based on user preferences and quickly respond to user questions.

[1061] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1062] In this invention, the server includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, means for the digital clone to perform a predetermined task, means for recommending products based on the user's preferences, and means for responding to user questions, thereby enabling product recommendations based on the user's individual preferences, improving the shopping experience in the virtual store, and enabling quick responses to user questions.

[1063] "User Data" refers to any information about a user, including messages, emails, search queries, and other data.

[1064] "Vector representation" refers to the technique of converting text and other data into numerical vectors, a format that can be processed by a computer.

[1065] A "digital clone" is a virtual model that captures a user's characteristics and performs various tasks on their behalf.

[1066] "Prescribed tasks" refer to specific tasks that the digital clone will perform, such as answering surveys or matching romantic partners.

[1067] "Product recommendation tools" refers to algorithms and systems that suggest appropriate products based on a user's preferences and past behavior.

[1068] "Means for responding to questions" refers to a function for providing appropriate answers to inquiries and questions from users.

[1069] This invention is a system that utilizes user data to generate a digital clone that captures the user's characteristics and automatically executes various tasks. A specific embodiment of this system will be described below.

[1070] System Overview

[1071] The system mainly includes the following elements:

[1072] 1. How to obtain user data

[1073] 2. Data analysis and vector representation generation

[1074] 3. How to create a digital clone

[1075] 4. A means of accomplishing a specific task

[1076] 5. A way to recommend products based on user preferences

[1077] 6. Means of responding to user questions

[1078] Hardware and software used

[1079] Hardware: Smartphones, servers

[1080] Software: Python, scikit-learn (natural language processing engine), TfidfVectorizer (text vectorization), cosine_similarity (similarity calculation)

[1081] Program processing overview

[1082] 1. How to obtain user data

[1083] Users log in through a smartphone application and agree to provide data, which is then collected, including messages, emails, and search queries.

[1084] 2. Data analysis and vector representation generation

[1085] The server analyzes the collected user data, tokenizing it using TfidfVectorizer to generate vector representations that represent the user's characteristics in numerical form.

[1086] 3. How to create a digital clone

[1087] A digital clone is generated based on the analyzed vector data, and the digital clone can perform various tasks while reflecting the user's characteristics.

[1088] 4. A means of accomplishing a specific task

[1089] Digital clones can automatically perform tasks such as answering surveys and matching people for love.

[1090] 5. A way to recommend products based on user preferences

[1091] The digital clone selects products from a virtual store based on the user's purchase history and preference data, and recommends the most suitable products to the user.

[1092] 6. Means of responding to user questions

[1093] When users ask questions through chat, the digital clone answers in real time.

[1094] Specific examples

[1095] 1. Example of user data acquisition

[1096] A user logs into the application and provides data such as "I like casual clothes" and "I like the color blue."

[1097] 2. Example of vector representation generation

[1098] The server parses the provided data, tokenizes the text with TfidfVectorizer, and generates a vector representation.

[1099] 3. Specific examples of product endorsements

[1100] Based on vector data, the system recommends the most suitable item from a virtual store, such as an elegant black dress, casual blue jeans, or a minimalist white T-shirt.

[1101] 4. Example of responding to user questions

[1102] When the user asks, "What is my favorite color?" the digital clone responds, "Your favorite color is blue."

[1103] Prompt Sentence Examples

[1104] "Based on your data, our virtual shopping assistant will recommend the best products for you. Please let us know if you have a preference for casual clothing or specific colors."

[1105] The above is a description of a specific embodiment for carrying out the present invention. This system enables a user to have a highly personalized shopping experience, thereby significantly improving user satisfaction.

[1106] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1107] Step 1:

[1108] Retrieving User Data

[1109] When a user logs in to a smartphone application and agrees to provide data, the device collects the user's message data, emails, search queries, etc. The input is the data provided by the user, and the output is the collected raw user data.

[1110] Step 2:

[1111] Data analysis and vector representation generation

[1112] The server receives the collected user data and performs analysis. Specifically, it uses Python and scikit-learn's TfidfVectorizer to tokenize the data and generate a vector representation. The input is raw user data, and the output is user data represented as a numeric vector.

[1113] Step 3:

[1114] Saving vector data

[1115] The generated vector representation is saved in a database on the server, making it available for subsequent tasks. The input is a numeric vector, and the output is vector data stored in the database.

[1116] Step 4:

[1117] Creating a digital clone

[1118] When a user requests the creation of a digital clone, the server uses the stored vector data to create the digital clone. The input is the vector data stored in the database, and the output is a digital clone that mimics the user's characteristics.

[1119] Step 5:

[1120] Executing a task

[1121] The digital clone performs a given task on behalf of the user, for example, making product recommendations in a virtual store: the input is data about the user's purchasing history and preferences, and the output is a list of products recommended to the user.

[1122] Step 6:

[1123] Question Processing

[1124] When a user inputs a question through a smartphone application, the server instantly generates an answer using the digital clone. The input is the user's question, and the output is the generated answer. For example, if the user asks, "What is my favorite color?", the digital clone will respond, "Your favorite color is blue."

[1125] Step 7:

[1126] Best product recommendations

[1127] The server compares the user's vector data with the product data of the virtual store and recommends the best products using cosine_similarity. The input is the user vector data and the product data vector, and the output is a list of products that are judged to be best for the user.

[1128] The above are the specific processing steps of the system for implementing the present invention, which provides users with an optimal shopping experience and question responses.

[1129] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1130] The present invention relates to a system that efficiently utilizes user data to generate a digital clone to perform various tasks. Furthermore, by combining it with an emotion engine that analyzes user data and recognizes the user's emotions, it is possible to provide more advanced services. This system includes a means for acquiring user data, analyzing it, and converting it into a vector representation to generate a digital clone that captures the user's characteristics and emotions. The generated digital clone is then used for specific tasks such as answering surveys, conducting interviews, and matching relationships. The system also includes a means for providing notifications and email reminders to the user.

[1131] System program processing explanation

[1132] Retrieving User Data

[1133] The user logs in and agrees to provide LINE chat data, emails, search queries, etc. The device collects this data with the user's consent.

[1134] Analyzing data and generating vector representations

[1135] The server analyzes the collected user data and recognizes the user's emotions using an emotion engine. This emotion information and user data are analyzed comprehensively to generate a vector representation. A natural language processing engine is used for the analysis, tokenizing the data and performing preprocessing to understand the context and emotion. The preprocessed data is then input into an embedding model to generate a vector representation. The generated vector is then stored in a database by the server.

[1136] Creating a digital clone and performing tasks

[1137] If a user wishes to create a digital clone, the server will use the stored vector data to generate a digital clone that reflects the user's characteristics and emotions and performs specific tasks on the user's behalf.

[1138] For example, in the case of a survey, the server inputs the survey contents into the digital clone, generates and sends appropriate answers, and in the case of love matching, the server compares multiple digital clones, finds compatible pairs, and notifies the user of the results.

[1139] Notifications and reminders

[1140] The server identifies new messages and emails that the user is likely to be interested in, analyzes their importance, and then sends push notifications to the device to remind the user of the important content.

[1141] Specific examples

[1142] 1. The user logs in to the system and agrees to provide data. The device collects LINE chat data, emails, and search queries.

[1143] 2. The server analyzes the collected data, uses an emotion engine to recognize the user's emotions, and reflects them in a vector representation.

[1144] 3. The server stores the generated vector in a database.

[1145] 4. The user requests the creation of a digital clone, and the server uses the generated vector data to create the digital clone.

[1146] 5. To answer the survey, the server passes the survey contents to the digital clone, which generates and transmits the appropriate answers.

[1147] 6. For love matching, the server compares multiple clones and suggests compatible pairs.

[1148] 7. The server analyzes new messages and emails and sends important information via push notification to the device to remind the user.

[1149] In this way, the present invention can provide a multi-functional system for improving the user experience by effectively utilizing user data and further recognizing the user's emotions.

[1150] The processing flow will be explained below.

[1151] Specific flow of program processing

[1152] Processing from acquiring user data to utilizing digital clones

[1153] Step 1:

[1154] A user logs in to the system.

[1155] The user enters their authentication information and accesses the login screen.

[1156] The server verifies the authentication information and establishes a session for the user.

[1157] Step 2:

[1158] The user agrees to provide the data.

[1159] The user reads the terms of use and selects a checkbox to agree to providing data.

[1160] The terminal transmits the user's consent to the server.

[1161] Step 3:

[1162] The terminal collects user data.

[1163] The device downloads the chat data through the LINE app's API.

[1164] The terminal retrieves received emails using the mail server's API.

[1165] The device collects search queries using the browser's history API.

[1166] Step 4:

[1167] The server parses the user data and generates a vector representation.

[1168] The server sends the collected messages, emails, and search queries to a natural language processing engine.

[1169] A natural language processing engine tokenizes the data and analyzes its contextual information.

[1170] The server inputs the preprocessed data into an embedding model (e.g., BERT, GPT) to generate vectors.

[1171] Step 5:

[1172] The server uses an emotion engine to recognize the user's emotion.

[1173] The server inputs the preprocessed data into an emotion engine to extract emotion information (e.g., joy, sadness, anger).

[1174] The server integrates the emotion information obtained from the emotion engine into a vector representation.

[1175] Step 6:

[1176] The server stores the generated vector in a database.

[1177] The server stores the vector data generated for each user ID in a database.

[1178] A database associates the vector data with a user profile.

[1179] Step 7:

[1180] A user wants to create a digital clone.

[1181] The user selects the digital cloning option on the settings page and sends a request to the server.

[1182] Step 8:

[1183] The server uses the stored vector data to generate a digital clone.

[1184] The server inputs the user's vector data into an LLM (Large Scale Language Model).

[1185] LLM creates a digital clone that reflects the user's characteristics and emotions.

[1186] Step 9:

[1187] The server uses a digital clone to perform tasks.

[1188] The server passes the survey results to the digital clone.

[1189] The digital clone generates appropriate answers to the questions and sends them to the survey system.

[1190] Step 10:

[1191] A user registers for a dating service.

[1192] Users enter profile information and preferences and submit them to the server.

[1193] Step 11:

[1194] The server compares multiple digital clones and finds compatible pairs.

[1195] The server analyzes the digital clones of all registered users and compares their feature vectors.

[1196] The server pairs compatible clones using a matching algorithm.

[1197] Step 12:

[1198] The server analyzes the importance of new messages and emails.

[1199] The device collects new notifications and emails in real time.

[1200] The server inputs the collected data into a machine learning model to calculate an importance score.

[1201] Step 13:

[1202] The server reminds the user of important content.

[1203] The server selects important notifications and emails and generates reminder content.

[1204] Send a push notification to the device and display a reminder to the user.

[1205] The above is the flow of specific processing steps related to the system of the present invention. In this way, by effectively utilizing user data and incorporating emotion information, it is possible to provide advanced services to users.

[1206] Example 2

[1207] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1208] Conventional systems that use user data have had difficulty in accurately recognizing user emotions and reflecting them in services. Furthermore, systems that notify and remind users have been unable to effectively deliver information of interest to users. Furthermore, there has been a lack of technology that can faithfully reflect a user's characteristics and emotions when performing tasks using a digital clone.

[1209] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1210] In this invention, the server includes a means for acquiring user data, a means for analyzing the acquired data and generating a vector representation, and a means for creating a digital clone of the user using the generated vector representation. This makes it possible to recognize the user's emotions and reflect the analyzed emotional information in the vector representation. Furthermore, by including a means for providing notifications and reminders to the user, it is possible to accurately deliver information that interests the user.

[1211] "User Data" refers to any information related to a user's activities, including messages, emails, search queries, etc.

[1212] "Vector representation" is a representation of extracted data features as a numerical vector, and is used in data analysis and machine learning.

[1213] A "digital clone" is a virtual copy of a user that reflects their characteristics and emotions and can perform tasks on their behalf.

[1214] An "emotion engine" refers to a system or algorithm that recognizes and analyzes emotions from user data.

[1215] "Notification" means information sent by the system to the user to inform them of new messages or important information.

[1216] A "remind" is a notification or announcement that reminds users of important information or tasks that they tend to forget.

[1217] The present invention relates to a system that efficiently utilizes user data, generates digital clones, and executes various tasks. A specific embodiment of this system will be described below.

[1218] First, the user logs in to the system and agrees to provide data such as LINE chat data, emails, and search queries. After obtaining the user's consent, the device collects this data and sends it to the server. This data collection and transmission is encrypted using the HTTPS protocol.

[1219] The server then receives the collected user data. This data is temporarily stored in a database and prepared for analysis. The server is equipped with a natural language processing engine, which uses models such as BERT and GPT-3 to tokenize and preprocess the data. After tokenization, the data is split into sentences and words and unwanted noise is removed.

[1220] The server then uses an emotion engine to analyze the user's emotions. For emotion recognition, it uses an LSTM model or similar. The server then inputs the preprocessed data into an embedding model (e.g., Word2Vec or FastText) to generate a vector representation for each piece of data. The server then stores these vectors in a database and associates them with the user ID.

[1221] When a user requests the creation of a digital clone, the server retrieves vector data associated with the user ID from the database. The server then inputs the vector data into a generative AI model (e.g., GPT-3) to generate a digital clone that reflects the user's characteristics and emotions. This digital clone can then perform specific tasks on the user's behalf.

[1222] For example, when answering a questionnaire, the server provides the digital clone with the questionnaire content, which generates and transmits the appropriate answer. In the case of love matching, the server compares multiple digital clones to find the most compatible pair.

[1223] Finally, the server periodically checks for new messages and emails, analyzes their content using Natural Language Understanding (NLU), identifies content that may be of interest to the user, scores its importance, and sends push notifications about the identified important content to the device. The device receives the push notifications and reminds the user.

[1224] As a concrete example, the following prompt sentence is input to the generative AI model:

[1225] "User: Please create a digital clone of yourself that will answer the following survey:

[1226] 1. What is your favorite food?

[1227] 2. What are your hobbies?

[1228] 3. Tell us about your most recent travel destination.

[1229] Following these prompts, the generative AI model generates appropriate answers that reflect the user's characteristics and emotions.

[1230] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1231] Step 1: Get User Data

[1232] 1. The user logs into the system and enters their credentials.

[1233] Input: User authentication information (user ID, password).

[1234] Output: Authentication success message, user ID.

[1235] 2. The user agrees to provide LINE chat data, emails, and search queries.

[1236] Input: User consent.

[1237] Output: Consent confirmation message.

[1238] 3. After obtaining the user's consent, the device collects LINE chat data, emails, and search history and sends them to the server. The transmission uses the HTTPS protocol and the data is encrypted.

[1239] Input: User's LINE chat data, emails, and search history.

[1240] Output: Encrypted user data.

[1241] Step 2: Analyze the data and generate a vector representation

[1242] 1. The server receives the collected user data and temporarily stores it in a database.

[1243] Input: Encrypted user data.

[1244] Output: Raw data stored in a database.

[1245] 2. The server uses a natural language processing engine to tokenize and preprocess the data, specifically splitting the text data into sentences and words and filtering out unnecessary noise.

[1246] Input: Stored raw data.

[1247] Output: Tokenized data.

[1248] 3. The emotion engine analyzes the tokenized data and recognizes the user's emotions. It uses an LSTM model or similar.

[1249] Input: Tokenized data.

[1250] Output: Emotional information.

[1251] 4. The natural language processing engine inputs the preprocessed data into an embedding model (e.g., Word2Vec or FastText) to generate a vector representation.

[1252] Input: Tokenized data and sentiment information.

[1253] Output: Vector representation.

[1254] 5. The generated vector is stored in the database and associated with the user ID.

[1255] Input: Vector representation and user ID.

[1256] Output: Vector data stored in a database.

[1257] Step 3: Generate a digital clone and perform the task

[1258] 1. If a user wishes to create a digital clone, they submit a request.

[1259] Input: Digital cloning request.

[1260] Output: Request received message.

[1261] 2. The server retrieves vector data associated with the user ID from the database.

[1262] Input: User ID.

[1263] Output: Vector data.

[1264] 3. The server inputs the vector data into a generative AI model (e.g., GPT-3) to generate a digital clone.

[1265] Input: Vector data.

[1266] Output: The generated digital clone.

[1267] 4. The digital clone performs specific tasks.

[1268] In the case of a survey response: The server provides the survey content to the digital clone, which generates and transmits the response.

[1269] Input: Survey contents.

[1270] Output: The generated answer.

[1271] In the case of love matching: The server compares multiple digital clones and finds compatible pairs.

[1272] Enter: multiple digital clones.

[1273] Output: Results of compatible pairs.

[1274] Step 4: Notifications and reminders

[1275] 1. The server periodically checks for new messages and emails.

[1276] Input: New Message, Email.

[1277] Output: A list of messages and emails.

[1278] 2. The server uses NLU to analyze the content of the message or email to identify content that the user is likely to be interested in. An importance score is calculated.

[1279] Input: message, email.

[1280] Output: Importance score.

[1281] 3. The server selects content with a high importance score and sends a notification to the device using the push notification API.

[1282] Input: Content with high importance score.

[1283] Output: Push notification.

[1284] 4. The device receives a push notification and reminds the user.

[1285] Enter: push notification.

[1286] Output: Remind notification.

[1287] At each step, specific actions are performed, and the entire system functions by repeating the input, processing, and output of data.

[1288] (Application example 2)

[1289] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1290] While conventional digital clone generation systems have a certain degree of accuracy in analyzing user data and performing tasks, they have limitations in providing more personalized services by incorporating the user's emotional state and real-time visual information. Furthermore, they have the problem of being unable to suggest products based on the user's emotions and interests when it comes to purchasing behavior in physical stores.

[1291] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, means for the digital clone to perform a predetermined task, means for recognizing the user's emotions, means for acquiring visual data from a visual device worn by the user, and means for suggesting products based on the visual data and the user's emotions. This makes it possible to provide highly personalized services based on the user's emotional state and real-time visual information.

[1292] "User Data" is information provided by a user, including messages, emails, search queries, and purchase history data.

[1293] A "vector representation" is a representation of acquired user data quantified and expressed as a multidimensional vector.

[1294] A "digital clone" is a virtual being that reflects the user's characteristics and emotions and performs various tasks on the user's behalf.

[1295] An "emotion recognition means" is a device or software that uses algorithms or models to analyze a user's emotional state.

[1296] A "visual device" is a device worn by a user to capture visual data, such as smart glasses.

[1297] "Visual data" refers to image or video information obtained from a visual device.

[1298] A "means for suggesting products" is a device or software that uses algorithms or models to suggest optimal products to a user based on the user's emotional state and visual data.

[1299] The present invention provides a system that efficiently utilizes user data and suggests products in physical stores based on the user's emotional state and visual information. This system acquires and analyzes user data to generate a digital clone, and further recognizes the user's emotions using an emotion engine, thereby realizing more advanced services.

[1300] Overall system configuration

[1301] The system consists of the following main components:

[1302] 1. User data acquisition means: A means for collecting messages, emails, search queries, and purchase history data provided by users. This means also acquires visual data through smart glasses.

[1303] 2. Data analysis means: Analyzes collected user data and recognizes user emotions using an emotion engine. Specifically, it uses an NLP model (natural language processing engine) and an emotion recognition engine.

[1304] 3. Vector representation generation: This is a method for generating vector representations that reflect the user's characteristics and emotions based on the analyzed data. An embedding model is used.

[1305] 4. Digital clone generation means: A means for creating a digital clone of the user using the generated vector representation. The digital clone performs tasks on behalf of the user.

[1306] 5. Task execution means: This is the means by which the digital clone performs tasks such as making purchasing suggestions on behalf of the user.

[1307] 6. Notifications and reminders: This is a method of sending product suggestions and new message reminders to users' devices via push notifications.

[1308] Specific examples of programs

[1309] 1. User data acquisition

[1310] With the user's consent, the server collects LINE chat data, emails, search queries, and purchase history data. It also collects visual data of users wearing smart glasses in real time.

[1311] 2. Data analysis and vector representation generation

[1312] The server analyzes the collected user data and converts the user's emotions and characteristics into vector representations using an emotion engine and NLP model.

[1313] 3. Creating a digital clone and performing tasks

[1314] When a user requests the creation of a digital clone, the server uses the stored vector data to create a digital clone that reflects the user's characteristics and emotions. The digital clone can then perform tasks such as product recommendations.

[1315] 4. Notifications and Reminders

[1316] The server sends important information and suggestions to the user's device, such as smart glasses, via push notifications to remind the user.

[1317] Hardware and software used

[1318] Hardware: Smart glasses, device (smartphone)

[1319] Software: Emotion recognition engine (e.g., DistilBERT), NLP model, digital clone generation engine, embedded model, external API (user data acquisition API, product recommendation API)

[1320] Specific examples

[1321] For example, imagine a user is in the cosmetics section. The smart glasses detect the specific cosmetics the user is interested in and, based on the user's past purchase data and real-time emotional data, suggest other users' reviews of that cosmetics, similar products, and special offers. This information is displayed via push notification on the smart glasses.

[1322] Prompt Sentence Examples

[1323] User ID: user_123

[1324] Store ID: store_789

[1325] Visual data: path / to / sample_image.jpg

[1326] [the purpose]

[1327] Appropriate sentiment analysis is performed on the products and locations that the user is interested in, and the most suitable products are suggested for the user.

[1328] [Provided data]

[1329] 1. User purchase history data

[1330] 2. Real-time user visual data

[1331] 3. In-store product data

[1332] [Expected output]

[1333] 1. The user's emotional state

[1334] 2. List of recommended products

[1335] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1336] Step 1:

[1337] The user logs into the system and agrees to provide data.

[1338] Input: User-provided messages, emails, search queries, and purchase history data

[1339] Output: Collected user data

[1340] Specific operation: When the user clicks the consent button, the device will collect LINE chat data, email content, past search queries, and purchase history data and send them to the server.

[1341] Step 2:

[1342] The server analyzes the collected user data and recognizes the user's emotions using an emotion engine (e.g., DistilBERT).

[1343] Input: Collected user data

[1344] Output: Analysis results including emotional information

[1345] What it does: The server tokenizes the collected data and uses an emotion recognition algorithm to determine the emotional state of each message or email, then stores the results in a database.

[1346] Step 3:

[1347] The server uses the parsed data to generate a vector representation.

[1348] Input: Analysis results including emotion information

[1349] Output: A vector representation reflecting the user's features and emotions

[1350] How it works: The server uses the NLP engine and embedding model to convert user data into multidimensional vectors as semantic vectors, which are then stored in a database.

[1351] Step 4:

[1352] When a user requests the creation of a digital clone, the server uses the stored vector data to generate a digital clone that reflects the user's characteristics and emotions.

[1353] Input: Vector representation

[1354] Output: Digital Clone

[1355] How it works: The server uses the stored vector data to generate a digital clone that mimics the user's behavioral patterns and emotions.

[1356] Step 5:

[1357] When a user puts on the smart glasses and enters a physical store, the device captures visual data in real time.

[1358] Input: Visual data

[1359] Output: Captured image and video data

[1360] How it works: The camera in the smart glasses captures visual data of the surroundings and transmits it to a server in real time.

[1361] Step 6:

[1362] The server uses real-time visual data and vector-expressed user data to suggest optimal products to users.

[1363] Input: visual data, vector representation

[1364] Output: A list of product suggestions

[1365] How it works: The server analyzes visual data to identify products the user is interested in. It then analyzes the user's emotional state and purchasing history to generate suggestions for similar products and review information.

[1366] Step 7:

[1367] The server sends the suggested product information to the smart glasses via push notification to notify the user.

[1368] Input: Product suggestion list

[1369] Output: Push notification product information

[1370] Specific operation: The server sends the generated product suggestions to the smart glasses as a push notification and displays them in the user's field of view, allowing the user to check product information in real time.

[1371] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1372] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1373] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1374] [Fourth embodiment]

[1375] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1376] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1377] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1378] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1379] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1380] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1381] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1382] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1383] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1384] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1385] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1386] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1387] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1388] The present invention relates to a system that efficiently utilizes user data to generate a digital clone to perform various tasks. The system includes a means for acquiring user data and converting it into a vector representation to generate a digital clone that captures the user's characteristics. The generated digital clone is then used for specific tasks such as answering questionnaires, conducting interviews, and matching relationships. The system also includes a means for providing notifications and email reminders to the user.

[1389] System program processing explanation

[1390] Retrieving User Data

[1391] The user logs in and agrees to provide LINE chat data, emails, search queries, etc. The device collects this data with the user's consent.

[1392] Analyzing data and generating vector representations

[1393] The server analyzes the collected user data and generates vector representations. This analysis uses a natural language processing engine to tokenize the data and preprocess it to understand its context. The preprocessed data is then fed into an embedding model to generate vector representations. The server stores the generated vectors in a database.

[1394] Creating a digital clone and performing tasks

[1395] If a user wishes to create a digital clone, the server uses the stored vector data to generate a digital clone that reflects the user's characteristics and performs specific tasks on the user's behalf.

[1396] For example, in the case of a survey, the server inputs the survey contents into the digital clone, generates and sends appropriate answers, and in the case of love matching, the server compares multiple digital clones, finds compatible pairs, and notifies the user of the results.

[1397] Notifications and reminders

[1398] The server identifies new messages and emails that the user is likely to be interested in, analyzes their importance, and then sends push notifications to the device to remind the user of the important content.

[1399] Specific examples

[1400] 1. The user logs in to the system and agrees to provide data. The device collects LINE chat data, emails, and search queries.

[1401] 2. The server analyzes the collected data, generates a vector representation, and stores it in a database.

[1402] 3. The user requests the creation of a digital clone, and the server uses the generated vector data to create the digital clone.

[1403] 4. To answer the survey, the server inputs the survey details into the digital clone, generates the appropriate answers, and sends them.

[1404] 5. For love matching, the server compares multiple clones and suggests compatible pairs.

[1405] 6. The server analyzes new messages and emails and sends important information via push notification to the device to remind the user.

[1406] Thus, the present invention provides a versatile system for effectively utilizing user data and enhancing the user experience.

[1407] The processing flow will be explained below.

[1408] Specific flow of program processing

[1409] Processing from acquiring user data to utilizing digital clones

[1410] Step 1:

[1411] A user logs in to the system.

[1412] The user enters their authentication information and accesses the login screen.

[1413] The server verifies the authentication information and establishes a session for the user.

[1414] Step 2:

[1415] The user agrees to provide the data.

[1416] The user reads the terms of use and selects a checkbox to agree to providing data.

[1417] The terminal transmits the user's consent to the server.

[1418] Step 3:

[1419] The terminal collects user data.

[1420] The device downloads the chat data through the LINE app's API.

[1421] The terminal retrieves received emails using the mail server's API.

[1422] The device collects search queries using the browser's history API.

[1423] Step 4:

[1424] The server parses the user data and generates a vector representation.

[1425] The server sends the collected messages, emails, and search queries to a natural language processing engine.

[1426] A natural language processing engine tokenizes the data and analyzes its contextual information.

[1427] The server inputs the preprocessed data into an embedding model (e.g., BERT, GPT) to generate vectors.

[1428] Step 5:

[1429] The server stores the generated vector in a database.

[1430] The server stores the vector data generated for each user ID in a database.

[1431] A database associates the vector data with a user profile.

[1432] Step 6:

[1433] A user wants to create a digital clone.

[1434] The user selects the digital cloning option on the settings page and sends a request to the server.

[1435] Step 7:

[1436] The server uses the stored vector data to generate a digital clone.

[1437] The server inputs the user's vector data into an LLM (Large Scale Language Model).

[1438] LLM generates a digital clone that reflects the user's characteristics.

[1439] Step 8:

[1440] The server uses a digital clone to perform tasks.

[1441] The server passes the survey results to the digital clone.

[1442] The digital clone generates appropriate answers to the questions and sends them to the survey system.

[1443] Step 9:

[1444] A user registers for a dating service.

[1445] Users enter profile information and preferences and submit them to the server.

[1446] Step 10:

[1447] The server compares multiple digital clones and finds compatible pairs.

[1448] The server analyzes the digital clones of all registered users and compares their feature vectors.

[1449] The server pairs compatible clones using a matching algorithm.

[1450] Step 11:

[1451] The server analyzes the importance of new messages and emails.

[1452] The device collects new notifications and emails in real time.

[1453] The server inputs the collected data into a machine learning model to calculate an importance score.

[1454] Step 12:

[1455] The server reminds the user of important content.

[1456] The server selects important notifications and emails and generates reminder content.

[1457] Send a push notification to the device and display a reminder to the user.

[1458] The above is the processing flow related to the system of the present invention. In this way, user data can be effectively utilized and functions suited to the user can be provided.

[1459] Example 1

[1460] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1461] In today's information society, the amount of data generated by users every day is steadily increasing. However, it is not easy to efficiently analyze this vast amount of data and provide services and notifications that meet user needs. In particular, there is a lack of systems that can generate digital clones that perform tasks on behalf of users and that can provide important notifications in a timely manner. There is a need for systems that can solve these technical challenges and improve user convenience.

[1462] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1463] In this invention, the server includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, means for the digital clone to perform a predetermined task, means for analyzing the content of newly arrived messages and emails and identifying important content, and means for sending the identified important content to the terminal by push notification. This makes it possible to effectively utilize user data and provide services that meet the user's needs.

[1464] "User Data" means information generated or provided by a user, including messages, emails, search queries, and the like.

[1465] "Analysis" refers to the process of analyzing collected data and converting it into meaningful information.

[1466] "Vector representation" is a format in which data is quantified and represented in a multidimensional space. This format makes it possible to efficiently handle the characteristics and relationships of data.

[1467] A "digital clone" is a virtual agent that reflects the user's characteristics and can perform specific tasks on the user's behalf.

[1468] "Tasks" refer to specific tasks or activities that digital clones perform, such as answering surveys or matching romantic partners.

[1469] "New messages" refers to messages or notifications that the user has recently received.

[1470] "Email" means a digital message sent or received over the Internet.

[1471] A "push notification" is an instantaneous notification sent from a server to a terminal, and is a means of quickly conveying important information to a user.

[1472] The present invention relates to a system that efficiently utilizes user data to generate a digital clone to perform various tasks. The system includes a means for acquiring user data and converting it into a vector representation to generate a digital clone that captures the user's characteristics. The generated digital clone is then used for specific tasks such as answering questionnaires, conducting interviews, and matching relationships. The system also includes a means for providing notifications and email reminders to the user.

[1473] Specifically, to obtain user data, users must log in to the system and agree to provide LINE chat data, emails, search queries, etc. The device collects this data with the user's consent. The collected data is encrypted and sent to the server.

[1474] The server then analyzes the collected user data and generates vector representations. This analysis involves using the Google NLP API to tokenize the data and preprocess it to understand its context. The preprocessed data is then fed into an embedding model (e.g., Word2Vec or BERT) to generate vector representations. The server then stores the generated vectors in a database (e.g., MySQL).

[1475] If a user wishes to create a digital clone, the server uses the stored vector data to generate a digital clone that reflects the user's characteristics and can perform specific tasks (e.g., answering surveys or matching relationships) on the user's behalf.

[1476] For example, in the case of a survey, the server inputs the survey contents into the digital clone, generates and sends appropriate answers, and in the case of love matching, the server compares multiple digital clones, finds compatible pairs, and notifies the user of the results.

[1477] The server also identifies content that may be of interest to the user from among newly received messages and emails, analyzes its importance, and sends push notifications to the device to remind the user of the identified important content.

[1478] Thus, the present invention provides a versatile system for effectively utilizing user data and improving user experience.

[1479] Examples of prompt statements

[1480] 1. Survey prompt:

[1481] "Please share your opinion on the following survey question: 'How do you like to spend your holidays?'"

[1482] 2. Romance Matching Prompt:

[1483] "What is your favorite movie genre and why?"

[1484] 3. Prompt for notification reminder:

[1485] "Check for the new message below: 'Tomorrow's meeting time has changed.'"

[1486] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1487] Step 1:

[1488] The user logs in to the system and agrees to provide data. User ID and password are required as input. The output is a successful login status and confirmation of data provision agreement. Specifically, the user enters authentication information into the login form and checks the checkbox for data provision.

[1489] Step 2:

[1490] With the user's consent, the device collects user data such as LINE chat data, emails, and search queries. Raw data provided by various apps and browsers is required as input. The collected data is encrypted and sent to the server as output. Specifically, the device collects data in the background, encrypts it with RSA, and uploads it to the server.

[1491] Step 3:

[1492] The server receives and decrypts the collected encrypted data. It requires encrypted user data as input and decrypts the decrypted text data as output. Specifically, the server uses an RSA decryption key to decrypt the received data and convert it into an analyzable format.

[1493] Step 4:

[1494] The server tokenizes and preprocesses the data using the Google NLP API. The input requires decrypted user data. The output is tokenized data. Specifically, the server calls the NLP API to split sentences and generate tokens.

[1495] Step 5:

[1496] The server inputs the preprocessed data into an embedding model (such as Word2Vec or BERT) to generate a vector representation. Tokenized data is required as input, and a vector representation is obtained as output. Specifically, the server calls the embedding model library to vectorize the data.

[1497] Step 6:

[1498] The server saves the generated vector to the database. It requires a vector representation as input and a confirmation of saving to the database as output. Specifically, the server connects to MySQL and uses the INSERT statement to insert the vector data into a table.

[1499] Step 7:

[1500] A user wishes to create a digital clone and submits a request. The input required is a clone creation request from the user. The output is a confirmation of receipt of the clone creation request. The specific operation is when the user clicks the clone creation button on a dedicated dashboard.

[1501] Step 8:

[1502] The server receives the request and generates a digital clone using the stored vector data. The input is vector data. The output is a digital clone. Specifically, the server reads the vector data and runs the clone generation algorithm.

[1503] Step 9:

[1504] The digital clone performs tasks such as answering surveys and matching. Survey questions and matching conditions are required as input. The output is the results of the task execution and matching. Specifically, the server inputs the survey questions into the clone, generates answers, and sends them.

[1505] Step 10:

[1506] The server identifies new messages or emails that are likely to be of interest to users and analyzes their importance. The input required is the new messages or emails. The output is the identified important content. Specifically, the server uses a text analysis algorithm to evaluate the importance of the messages.

[1507] Step 11:

[1508] The server sends a push notification to the device to remind the user of the identified important information. The input required is the identified important information. The output is a confirmation of the push notification. Specifically, the server sends the notification to the device using the Firebase Cloud Messaging (FCM) API.

[1509] (Application example 1)

[1510] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1511] Conventional virtual store shopping experiences often fail to fully address individual user preferences, potentially resulting in low user satisfaction. Furthermore, users' questions are not answered in real time, limiting the user experience. Therefore, there is a need for a system that can recommend products based on user preferences and quickly respond to user questions.

[1512] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1513] In this invention, the server includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, means for the digital clone to perform a predetermined task, means for recommending products based on the user's preferences, and means for responding to user questions, thereby enabling product recommendations based on the user's individual preferences, improving the shopping experience in the virtual store, and enabling quick responses to user questions.

[1514] "User Data" refers to any information about a user, including messages, emails, search queries, and other data.

[1515] "Vector representation" refers to the technique of converting text and other data into numerical vectors, a format that can be processed by a computer.

[1516] A "digital clone" is a virtual model that captures a user's characteristics and performs various tasks on their behalf.

[1517] "Prescribed tasks" refer to specific tasks that the digital clone will perform, such as answering surveys or matching romantic partners.

[1518] "Product recommendation tools" refers to algorithms and systems that suggest appropriate products based on a user's preferences and past behavior.

[1519] "Means for responding to questions" refers to a function for providing appropriate answers to inquiries and questions from users.

[1520] This invention is a system that utilizes user data to generate a digital clone that captures the user's characteristics and automatically executes various tasks. A specific embodiment of this system will be described below.

[1521] System Overview

[1522] The system mainly includes the following elements:

[1523] 1. How to obtain user data

[1524] 2. Data analysis and vector representation generation

[1525] 3. How to create a digital clone

[1526] 4. A means of accomplishing a specific task

[1527] 5. A way to recommend products based on user preferences

[1528] 6. Means of responding to user questions

[1529] Hardware and software used

[1530] Hardware: Smartphones, servers

[1531] Software: Python, scikit-learn (natural language processing engine), TfidfVectorizer (text vectorization), cosine_similarity (similarity calculation)

[1532] Program processing overview

[1533] 1. How to obtain user data

[1534] Users log in through a smartphone application and agree to provide data, which is then collected, including messages, emails, and search queries.

[1535] 2. Data analysis and vector representation generation

[1536] The server analyzes the collected user data, tokenizing it using TfidfVectorizer to generate vector representations that represent the user's characteristics in numerical form.

[1537] 3. How to create a digital clone

[1538] A digital clone is generated based on the analyzed vector data, and the digital clone can perform various tasks while reflecting the user's characteristics.

[1539] 4. A means of accomplishing a specific task

[1540] Digital clones can automatically perform tasks such as answering surveys and matching people for love.

[1541] 5. A way to recommend products based on user preferences

[1542] The digital clone selects products from a virtual store based on the user's purchase history and preference data, and recommends the most suitable products to the user.

[1543] 6. Means of responding to user questions

[1544] When users ask questions through chat, the digital clone answers in real time.

[1545] Specific examples

[1546] 1. Example of user data acquisition

[1547] A user logs into the application and provides data such as "I like casual clothes" and "I like the color blue."

[1548] 2. Example of vector representation generation

[1549] The server parses the provided data, tokenizes the text with TfidfVectorizer, and generates a vector representation.

[1550] 3. Specific examples of product endorsements

[1551] Based on vector data, the system recommends the most suitable item from a virtual store, such as an elegant black dress, casual blue jeans, or a minimalist white T-shirt.

[1552] 4. Example of responding to user questions

[1553] When the user asks, "What is my favorite color?" the digital clone responds, "Your favorite color is blue."

[1554] Prompt Sentence Examples

[1555] "Based on your data, our virtual shopping assistant will recommend the best products for you. Please let us know if you have a preference for casual clothing or specific colors."

[1556] The above is a description of a specific embodiment for carrying out the present invention. This system enables a user to have a highly personalized shopping experience, thereby significantly improving user satisfaction.

[1557] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1558] Step 1:

[1559] Retrieving User Data

[1560] When a user logs in to a smartphone application and agrees to provide data, the device collects the user's message data, emails, search queries, etc. The input is the data provided by the user, and the output is the collected raw user data.

[1561] Step 2:

[1562] Data analysis and vector representation generation

[1563] The server receives the collected user data and performs analysis. Specifically, it uses Python and scikit-learn's TfidfVectorizer to tokenize the data and generate a vector representation. The input is raw user data, and the output is user data represented as a numeric vector.

[1564] Step 3:

[1565] Saving vector data

[1566] The generated vector representation is saved in a database on the server, making it available for subsequent tasks. The input is a numeric vector, and the output is vector data stored in the database.

[1567] Step 4:

[1568] Creating a digital clone

[1569] When a user requests the creation of a digital clone, the server uses the stored vector data to create the digital clone. The input is the vector data stored in the database, and the output is a digital clone that mimics the user's characteristics.

[1570] Step 5:

[1571] Executing a task

[1572] The digital clone performs a given task on behalf of the user, for example, making product recommendations in a virtual store: the input is data about the user's purchasing history and preferences, and the output is a list of products recommended to the user.

[1573] Step 6:

[1574] Question Processing

[1575] When a user inputs a question through a smartphone application, the server instantly generates an answer using the digital clone. The input is the user's question, and the output is the generated answer. For example, if the user asks, "What is my favorite color?", the digital clone will respond, "Your favorite color is blue."

[1576] Step 7:

[1577] Best product recommendations

[1578] The server compares the user's vector data with the product data of the virtual store and recommends the best products using cosine_similarity. The input is the user vector data and the product data vector, and the output is a list of products that are judged to be best for the user.

[1579] The above are the specific processing steps of the system for implementing the present invention, which provides users with an optimal shopping experience and question responses.

[1580] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1581] The present invention relates to a system that efficiently utilizes user data to generate a digital clone to perform various tasks. Furthermore, by combining it with an emotion engine that analyzes user data and recognizes the user's emotions, it is possible to provide more advanced services. This system includes a means for acquiring user data, analyzing it, and converting it into a vector representation to generate a digital clone that captures the user's characteristics and emotions. The generated digital clone is then used for specific tasks such as answering surveys, conducting interviews, and matching relationships. The system also includes a means for providing notifications and email reminders to the user.

[1582] System program processing explanation

[1583] Retrieving User Data

[1584] The user logs in and agrees to provide LINE chat data, emails, search queries, etc. The device collects this data with the user's consent.

[1585] Analyzing data and generating vector representations

[1586] The server analyzes the collected user data and recognizes the user's emotions using an emotion engine. This emotion information and user data are analyzed comprehensively to generate a vector representation. A natural language processing engine is used for the analysis, tokenizing the data and performing preprocessing to understand the context and emotion. The preprocessed data is then input into an embedding model to generate a vector representation. The generated vector is then stored in a database by the server.

[1587] Creating a digital clone and performing tasks

[1588] If a user wishes to create a digital clone, the server will use the stored vector data to generate a digital clone that reflects the user's characteristics and emotions and performs specific tasks on the user's behalf.

[1589] For example, in the case of a survey, the server inputs the survey contents into the digital clone, generates and sends appropriate answers, and in the case of love matching, the server compares multiple digital clones, finds compatible pairs, and notifies the user of the results.

[1590] Notifications and reminders

[1591] The server identifies new messages and emails that the user is likely to be interested in, analyzes their importance, and then sends push notifications to the device to remind the user of the important content.

[1592] Specific examples

[1593] 1. The user logs in to the system and agrees to provide data. The device collects LINE chat data, emails, and search queries.

[1594] 2. The server analyzes the collected data, uses an emotion engine to recognize the user's emotions, and reflects them in a vector representation.

[1595] 3. The server stores the generated vector in a database.

[1596] 4. The user requests the creation of a digital clone, and the server uses the generated vector data to create the digital clone.

[1597] 5. To answer the survey, the server passes the survey contents to the digital clone, which generates and transmits the appropriate answers.

[1598] 6. For love matching, the server compares multiple clones and suggests compatible pairs.

[1599] 7. The server analyzes new messages and emails and sends important information via push notification to the device to remind the user.

[1600] In this way, the present invention can provide a multi-functional system for improving the user experience by effectively utilizing user data and further recognizing the user's emotions.

[1601] The processing flow will be explained below.

[1602] Specific flow of program processing

[1603] Processing from acquiring user data to utilizing digital clones

[1604] Step 1:

[1605] A user logs in to the system.

[1606] The user enters their authentication information and accesses the login screen.

[1607] The server verifies the authentication information and establishes a session for the user.

[1608] Step 2:

[1609] The user agrees to provide the data.

[1610] The user reads the terms of use and selects a checkbox to agree to providing data.

[1611] The terminal transmits the user's consent to the server.

[1612] Step 3:

[1613] The terminal collects user data.

[1614] The device downloads the chat data through the LINE app's API.

[1615] The terminal retrieves received emails using the mail server's API.

[1616] The device collects search queries using the browser's history API.

[1617] Step 4:

[1618] The server parses the user data and generates a vector representation.

[1619] The server sends the collected messages, emails, and search queries to a natural language processing engine.

[1620] A natural language processing engine tokenizes the data and analyzes its contextual information.

[1621] The server inputs the preprocessed data into an embedding model (e.g., BERT, GPT) to generate vectors.

[1622] Step 5:

[1623] The server uses an emotion engine to recognize the user's emotion.

[1624] The server inputs the preprocessed data into an emotion engine to extract emotion information (e.g., joy, sadness, anger).

[1625] The server integrates the emotion information obtained from the emotion engine into a vector representation.

[1626] Step 6:

[1627] The server stores the generated vector in a database.

[1628] The server stores the vector data generated for each user ID in a database.

[1629] A database associates the vector data with a user profile.

[1630] Step 7:

[1631] A user wants to create a digital clone.

[1632] The user selects the digital cloning option on the settings page and sends a request to the server.

[1633] Step 8:

[1634] The server uses the stored vector data to generate a digital clone.

[1635] The server inputs the user's vector data into an LLM (Large Scale Language Model).

[1636] LLM creates a digital clone that reflects the user's characteristics and emotions.

[1637] Step 9:

[1638] The server uses a digital clone to perform tasks.

[1639] The server passes the survey results to the digital clone.

[1640] The digital clone generates appropriate answers to the questions and sends them to the survey system.

[1641] Step 10:

[1642] A user registers for a dating service.

[1643] Users enter profile information and preferences and submit them to the server.

[1644] Step 11:

[1645] The server compares multiple digital clones and finds compatible pairs.

[1646] The server analyzes the digital clones of all registered users and compares their feature vectors.

[1647] The server pairs compatible clones using a matching algorithm.

[1648] Step 12:

[1649] The server analyzes the importance of new messages and emails.

[1650] The device collects new notifications and emails in real time.

[1651] The server inputs the collected data into a machine learning model to calculate an importance score.

[1652] Step 13:

[1653] The server reminds the user of important content.

[1654] The server selects important notifications and emails and generates reminder content.

[1655] Send a push notification to the device and display a reminder to the user.

[1656] The above is the flow of specific processing steps related to the system of the present invention. In this way, by effectively utilizing user data and incorporating emotion information, it is possible to provide advanced services to users.

[1657] Example 2

[1658] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1659] Conventional systems that use user data have had difficulty in accurately recognizing user emotions and reflecting them in services. Furthermore, systems that notify and remind users have been unable to effectively deliver information of interest to users. Furthermore, there has been a lack of technology that can faithfully reflect a user's characteristics and emotions when performing tasks using a digital clone.

[1660] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1661] In this invention, the server includes a means for acquiring user data, a means for analyzing the acquired data and generating a vector representation, and a means for creating a digital clone of the user using the generated vector representation. This makes it possible to recognize the user's emotions and reflect the analyzed emotional information in the vector representation. Furthermore, by including a means for providing notifications and reminders to the user, it is possible to accurately deliver information that interests the user.

[1662] "User Data" refers to any information related to a user's activities, including messages, emails, search queries, etc.

[1663] "Vector representation" is a representation of extracted data features as a numerical vector, and is used in data analysis and machine learning.

[1664] A "digital clone" is a virtual copy of a user that reflects their characteristics and emotions and can perform tasks on their behalf.

[1665] An "emotion engine" refers to a system or algorithm that recognizes and analyzes emotions from user data.

[1666] "Notification" means information sent by the system to the user to inform them of new messages or important information.

[1667] A "remind" is a notification or announcement that reminds users of important information or tasks that they tend to forget.

[1668] The present invention relates to a system that efficiently utilizes user data, generates digital clones, and executes various tasks. A specific embodiment of this system will be described below.

[1669] First, the user logs in to the system and agrees to provide data such as LINE chat data, emails, and search queries. After obtaining the user's consent, the device collects this data and sends it to the server. This data collection and transmission is encrypted using the HTTPS protocol.

[1670] The server then receives the collected user data. This data is temporarily stored in a database and prepared for analysis. The server is equipped with a natural language processing engine, which uses models such as BERT and GPT-3 to tokenize and preprocess the data. After tokenization, the data is split into sentences and words and unwanted noise is removed.

[1671] The server then uses an emotion engine to analyze the user's emotions. For emotion recognition, it uses an LSTM model or similar. The server then inputs the preprocessed data into an embedding model (e.g., Word2Vec or FastText) to generate a vector representation for each piece of data. The server then stores these vectors in a database and associates them with the user ID.

[1672] When a user requests the creation of a digital clone, the server retrieves vector data associated with the user ID from the database. The server then inputs the vector data into a generative AI model (e.g., GPT-3) to generate a digital clone that reflects the user's characteristics and emotions. This digital clone can then perform specific tasks on the user's behalf.

[1673] For example, when answering a questionnaire, the server provides the digital clone with the questionnaire content, which generates and transmits the appropriate answer. In the case of love matching, the server compares multiple digital clones to find the most compatible pair.

[1674] Finally, the server periodically checks for new messages and emails, analyzes their content using Natural Language Understanding (NLU), identifies content that may be of interest to the user, scores its importance, and sends push notifications about the identified important content to the device. The device receives the push notifications and reminds the user.

[1675] As a concrete example, the following prompt sentence is input to the generative AI model:

[1676] "User: Please create a digital clone of yourself that will answer the following survey:

[1677] 1. What is your favorite food?

[1678] 2. What are your hobbies?

[1679] 3. Tell us about your most recent travel destination.

[1680] Following these prompts, the generative AI model generates appropriate answers that reflect the user's characteristics and emotions.

[1681] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1682] Step 1: Get User Data

[1683] 1. The user logs into the system and enters their credentials.

[1684] Input: User authentication information (user ID, password).

[1685] Output: Authentication success message, user ID.

[1686] 2. The user agrees to provide LINE chat data, emails, and search queries.

[1687] Input: User consent.

[1688] Output: Consent confirmation message.

[1689] 3. After obtaining the user's consent, the device collects LINE chat data, emails, and search history and sends them to the server. The transmission uses the HTTPS protocol and the data is encrypted.

[1690] Input: User's LINE chat data, emails, and search history.

[1691] Output: Encrypted user data.

[1692] Step 2: Analyze the data and generate a vector representation

[1693] 1. The server receives the collected user data and temporarily stores it in a database.

[1694] Input: Encrypted user data.

[1695] Output: Raw data stored in a database.

[1696] 2. The server uses a natural language processing engine to tokenize and preprocess the data, specifically splitting the text data into sentences and words and filtering out unnecessary noise.

[1697] Input: Stored raw data.

[1698] Output: Tokenized data.

[1699] 3. The emotion engine analyzes the tokenized data and recognizes the user's emotions. It uses an LSTM model or similar.

[1700] Input: Tokenized data.

[1701] Output: Emotional information.

[1702] 4. The natural language processing engine inputs the preprocessed data into an embedding model (e.g., Word2Vec or FastText) to generate a vector representation.

[1703] Input: Tokenized data and sentiment information.

[1704] Output: Vector representation.

[1705] 5. The generated vector is stored in the database and associated with the user ID.

[1706] Input: Vector representation and user ID.

[1707] Output: Vector data stored in a database.

[1708] Step 3: Generate a digital clone and perform the task

[1709] 1. If a user wishes to create a digital clone, they submit a request.

[1710] Input: Digital cloning request.

[1711] Output: Request received message.

[1712] 2. The server retrieves vector data associated with the user ID from the database.

[1713] Input: User ID.

[1714] Output: Vector data.

[1715] 3. The server inputs the vector data into a generative AI model (e.g., GPT-3) to generate a digital clone.

[1716] Input: Vector data.

[1717] Output: The generated digital clone.

[1718] 4. The digital clone performs specific tasks.

[1719] In the case of a survey response: The server provides the survey content to the digital clone, which generates and transmits the response.

[1720] Input: Survey contents.

[1721] Output: The generated answer.

[1722] In the case of love matching: The server compares multiple digital clones and finds compatible pairs.

[1723] Enter: multiple digital clones.

[1724] Output: Results of compatible pairs.

[1725] Step 4: Notifications and reminders

[1726] 1. The server periodically checks for new messages and emails.

[1727] Input: New Message, Email.

[1728] Output: A list of messages and emails.

[1729] 2. The server uses NLU to analyze the content of the message or email to identify content that the user is likely to be interested in. An importance score is calculated.

[1730] Input: message, email.

[1731] Output: Importance score.

[1732] 3. The server selects content with a high importance score and sends a notification to the device using the push notification API.

[1733] Input: Content with high importance score.

[1734] Output: Push notification.

[1735] 4. The device receives a push notification and reminds the user.

[1736] Enter: push notification.

[1737] Output: Remind notification.

[1738] At each step, specific actions are performed, and the entire system functions by repeating the input, processing, and output of data.

[1739] (Application example 2)

[1740] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1741] While conventional digital clone generation systems have a certain degree of accuracy in analyzing user data and performing tasks, they have limitations in providing more personalized services by incorporating the user's emotional state and real-time visual information. Furthermore, they have the problem of being unable to suggest products based on the user's emotions and interests when it comes to purchasing behavior in physical stores.

[1742] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user data, means for analyzing the acquired data and generating a vector representation, means for creating a digital clone of the user using the generated vector representation, means for the digital clone to perform a predetermined task, means for recognizing the user's emotions, means for acquiring visual data from a visual device worn by the user, and means for suggesting products based on the visual data and the user's emotions. This makes it possible to provide highly personalized services based on the user's emotional state and real-time visual information.

[1743] "User Data" is information provided by a user, including messages, emails, search queries, and purchase history data.

[1744] A "vector representation" is a representation of acquired user data quantified and expressed as a multidimensional vector.

[1745] A "digital clone" is a virtual being that reflects the user's characteristics and emotions and performs various tasks on the user's behalf.

[1746] An "emotion recognition means" is a device or software that uses algorithms or models to analyze a user's emotional state.

[1747] A "visual device" is a device worn by a user to capture visual data, such as smart glasses.

[1748] "Visual data" refers to image or video information obtained from a visual device.

[1749] A "means for suggesting products" is a device or software that uses algorithms or models to suggest optimal products to a user based on the user's emotional state and visual data.

[1750] The present invention provides a system that efficiently utilizes user data and suggests products in physical stores based on the user's emotional state and visual information. This system acquires and analyzes user data to generate a digital clone, and further recognizes the user's emotions using an emotion engine, thereby realizing more advanced services.

[1751] Overall system configuration

[1752] The system consists of the following main components:

[1753] 1. User data acquisition means: A means for collecting messages, emails, search queries, and purchase history data provided by users. This means also acquires visual data through smart glasses.

[1754] 2. Data analysis means: Analyzes collected user data and recognizes user emotions using an emotion engine. Specifically, it uses an NLP model (natural language processing engine) and an emotion recognition engine.

[1755] 3. Vector representation generation: This is a method for generating vector representations that reflect the user's characteristics and emotions based on the analyzed data. An embedding model is used.

[1756] 4. Digital clone generation means: A means for creating a digital clone of the user using the generated vector representation. The digital clone performs tasks on behalf of the user.

[1757] 5. Task execution means: This is the means by which the digital clone performs tasks such as making purchasing suggestions on behalf of the user.

[1758] 6. Notifications and reminders: This is a method of sending product suggestions and new message reminders to users' devices via push notifications.

[1759] Specific examples of programs

[1760] 1. User data acquisition

[1761] With the user's consent, the server collects LINE chat data, emails, search queries, and purchase history data. It also collects visual data of users wearing smart glasses in real time.

[1762] 2. Data analysis and vector representation generation

[1763] The server analyzes the collected user data and converts the user's emotions and characteristics into vector representations using an emotion engine and NLP model.

[1764] 3. Creating a digital clone and performing tasks

[1765] When a user requests the creation of a digital clone, the server uses the stored vector data to create a digital clone that reflects the user's characteristics and emotions. The digital clone can then perform tasks such as product recommendations.

[1766] 4. Notifications and Reminders

[1767] The server sends important information and suggestions to the user's device, such as smart glasses, via push notifications to remind the user.

[1768] Hardware and software used

[1769] Hardware: Smart glasses, device (smartphone)

[1770] Software: Emotion recognition engine (e.g., DistilBERT), NLP model, digital clone generation engine, embedded model, external API (user data acquisition API, product recommendation API)

[1771] Specific examples

[1772] For example, imagine a user is in the cosmetics section. The smart glasses detect the specific cosmetics the user is interested in and, based on the user's past purchase data and real-time emotional data, suggest other users' reviews of that cosmetics, similar products, and special offers. This information is displayed via push notification on the smart glasses.

[1773] Prompt Sentence Examples

[1774] User ID: user_123

[1775] Store ID: store_789

[1776] Visual data: path / to / sample_image.jpg

[1777] [the purpose]

[1778] Appropriate sentiment analysis is performed on the products and locations that the user is interested in, and the most suitable products are suggested for the user.

[1779] [Provided data]

[1780] 1. User purchase history data

[1781] 2. Real-time user visual data

[1782] 3. In-store product data

[1783] [Expected output]

[1784] 1. The user's emotional state

[1785] 2. List of recommended products

[1786] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1787] Step 1:

[1788] The user logs into the system and agrees to provide data.

[1789] Input: User-provided messages, emails, search queries, and purchase history data

[1790] Output: Collected user data

[1791] Specific operation: When the user clicks the consent button, the device will collect LINE chat data, email content, past search queries, and purchase history data and send them to the server.

[1792] Step 2:

[1793] The server analyzes the collected user data and recognizes the user's emotions using an emotion engine (e.g., DistilBERT).

[1794] Input: Collected user data

[1795] Output: Analysis results including emotional information

[1796] What it does: The server tokenizes the collected data and uses an emotion recognition algorithm to determine the emotional state of each message or email, then stores the results in a database.

[1797] Step 3:

[1798] The server uses the parsed data to generate a vector representation.

[1799] Input: Analysis results including emotion information

[1800] Output: A vector representation reflecting the user's features and emotions

[1801] How it works: The server uses the NLP engine and embedding model to convert user data into multidimensional vectors as semantic vectors, which are then stored in a database.

[1802] Step 4:

[1803] When a user requests the creation of a digital clone, the server uses the stored vector data to generate a digital clone that reflects the user's characteristics and emotions.

[1804] Input: Vector representation

[1805] Output: Digital Clone

[1806] How it works: The server uses the stored vector data to generate a digital clone that mimics the user's behavioral patterns and emotions.

[1807] Step 5:

[1808] When a user puts on the smart glasses and enters a physical store, the device captures visual data in real time.

[1809] Input: Visual data

[1810] Output: Captured image and video data

[1811] How it works: The camera in the smart glasses captures visual data of the surroundings and transmits it to a server in real time.

[1812] Step 6:

[1813] The server uses real-time visual data and vector-expressed user data to suggest optimal products to users.

[1814] Input: visual data, vector representation

[1815] Output: A list of product suggestions

[1816] How it works: The server analyzes visual data to identify products the user is interested in. It then analyzes the user's emotional state and purchasing history to generate suggestions for similar products and review information.

[1817] Step 7:

[1818] The server sends the suggested product information to the smart glasses via push notification to notify the user.

[1819] Input: Product suggestion list

[1820] Output: Push notification product information

[1821] Specific operation: The server sends the generated product suggestions to the smart glasses as a push notification and displays them in the user's field of view, allowing the user to check product information in real time.

[1822] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1823] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1824] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1825] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1826] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1827] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1828] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1829] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1830] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1831] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1832] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1833] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1834] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1835] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1836] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1837] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1838] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1839] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1840] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1841] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1842] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1843] The following is further disclosed regarding the above embodiment.

[1844] (Claim 1)

[1845] a means for obtaining user data;

[1846] means for analyzing the acquired data and generating a vector representation;

[1847] means for creating a digital clone of the user using the generated vector representation;

[1848] a means for the digital clone to operate on a predetermined task;

[1849] A system including:

[1850] (Claim 2)

[1851] 10. The system of claim 1, wherein the user data includes messages, emails, and search queries.

[1852] (Claim 3)

[1853] 3. The system of claim 1 or 2, further comprising means for storing the generated vector representation in a database.

[1854] (Claim 4)

[1855] 4. The system according to claim 1, further comprising means for using a digital clone to answer questionnaires and interviews on behalf of the user.

[1856] (Claim 5)

[1857] 5. The system according to claim 1, further comprising means for performing love matching between users using digital clones.

[1858] (Claim 6)

[1859] 6. The system according to claim 1, further comprising means for identifying content that may be of interest to the user from among newly arrived messages or e-mails and reminding the user of the content.

[1860] "Example 1"

[1861] (Claim 1)

[1862] a means for obtaining user data;

[1863] means for analyzing the acquired data and generating a vector representation;

[1864] means for creating a digital clone of the user using the generated vector representation;

[1865] a means for the digital clone to operate on a predetermined task;

[1866] A means of analyzing the content of new messages and emails to identify important content;

[1867] a means for sending the identified important information to the device via a push notification;

[1868] A system including:

[1869] (Claim 2)

[1870] 10. The system of claim 1, wherein the user data includes messages, emails, and search queries.

[1871] (Claim 3)

[1872] 3. The system of claim 1 or 2, further comprising means for storing the generated vector representation in a database.

[1873] "Application Example 1"

[1874] (Claim 1)

[1875] a means for obtaining user data;

[1876] means for analyzing the acquired data and generating a vector representation;

[1877] means for creating a digital clone of the user using the generated vector representation;

[1878] a means for the digital clone to operate on a predetermined task;

[1879] a means of recommending products based on user preferences;

[1880] means for responding to user queries;

[1881] A system including:

[1882] (Claim 2)

[1883] 10. The system of claim 1, wherein the user data includes messages, emails, and search queries.

[1884] (Claim 3)

[1885] 10. The system of claim 1, further comprising means for storing the generated vector representation in a database.

[1886] (Claim 4)

[1887] The system of claim 1, which recommends products that suit the user's preferences based on the generated vector representation and related data.

[1888] (Claim 5)

[1889] 10. The system of claim 1, wherein the digital clone generates responses to questions posed by the user.

[1890] "Example 2: Combining Emotion Engines"

[1891] (Claim 1)

[1892] a means for obtaining user data;

[1893] means for analyzing the acquired data and generating a vector representation;

[1894] means for creating a digital clone of the user using the generated vector representation;

[1895] a means for the digital clone to operate on a predetermined task;

[1896] means including an engine for recognizing a user's emotion;

[1897] A means for reflecting the analyzed emotion information in a vector representation;

[1898] a means for providing notifications and reminders to the user;

[1899] A system including:

[1900] (Claim 2)

[1901] 10. The system of claim 1, wherein the user data includes messages, emails, and search queries.

[1902] (Claim 3)

[1903] 10. The system of claim 1, further comprising means for storing the generated vector representation in a database.

[1904] "Application example 2 when combining emotion engines"

[1905] (Claim 1)

[1906] a means for obtaining user data;

[1907] means for analyzing the acquired data and generating a vector representation;

[1908] means for creating a digital clone of the user using the generated vector representation;

[1909] a means for the digital clone to operate on a predetermined task;

[1910] means for recognizing a user's emotion;

[1911] means for acquiring visual data from a visual device worn by a user;

[1912] A means for suggesting products based on visual data and user emotions;

[1913] A system including:

[1914] (Claim 2)

[1915] The system of claim 1, wherein the user data includes messages, emails, search queries, and purchase history data.

[1916] (Claim 3)

[1917] 10. The system of claim 1, further comprising means for storing the generated vector representation in a database. [Explanation of symbols]

[1918] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for obtaining user data; means for analyzing the acquired data and generating a vector representation; means for creating a digital clone of the user using the generated vector representation; a means for the digital clone to operate on a predetermined task; A system including:

2. The system of claim 1 , wherein the user data includes messages, emails, and search queries.

3. 3. A system according to claim 1 or 2, further comprising means for storing the generated vector representation in a database.

4. 4. The system according to claim 1, further comprising means for using the digital clone to answer questionnaires and interviews on behalf of the user.

5. The system according to any one of claims 1 to 4, further comprising means for performing love matching between users using digital clones.

6. 6. The system according to claim 1, further comprising means for identifying content that may be of interest to the user from among newly arrived messages or e-mails and reminding the user of the content.

Images